Ling-CL: Understanding NLP Models through Linguistic Curricula
Mohamed Elgaar Hadi Amiri
We employ a characterization of linguistic complexity from psycholinguistic
and language acquisition research to develop data-driven curricula to
understand the underlying linguistic knowledge that models learn to address NLP
tasks. The novelty of our approach is in the development of linguistic
curricula derived from data, existing knowledge about linguistic complexity,
and model behavior during training. By analyzing several benchmark NLP
datasets, our curriculum learning approaches identify sets of linguistic
metrics (indices) that inform the challenges and reasoning required to address
each task. Our work will inform future research in all NLP areas, allowing
linguistic complexity to be considered early in the research and development
process. In addition, our work prompts an examination of gold standards and
fair evaluation in NLP.