HuCurl: Human-induced Curriculum Discovery
Mohamed Elgaar Hadi Amiri
We introduce the problem of curriculum discovery and describe a curriculum
learning framework capable of discovering effective curricula in a curriculum
space based on prior knowledge about sample difficulty. Using annotation
entropy and loss as measures of difficulty, we show that (i): the
top-performing discovered curricula for a given model and dataset are often
non-monotonic as opposed to monotonic curricula in existing literature, (ii):
the prevailing easy-to-hard or hard-to-easy transition curricula are often at
the risk of underperforming, and (iii): the curricula discovered for smaller
datasets and models perform well on larger datasets and models respectively.
The proposed framework encompasses some of the existing curriculum learning
approaches and can discover curricula that outperform them across several NLP
tasks.