CreoleVal: Multilingual Multitask Benchmarks for Creoles
Heather Lent Kushal Tatariya Raj Dabre Yiyi Chen Marcell Fekete Esther Ploeger Li Zhou Hans Erik Heje Diptesh Kanojia Paul Belony Marcel Bollmann Loïc Grobol Miryam de Lhoneux Daniel Hershcovich Michel DeGraff Anders Søgaard Johannes Bjerva
Creoles represent an under-explored and marginalized group of languages, with
few available resources for NLP research. While the genealogical ties between
Creoles and other highly-resourced languages imply a significant potential for
transfer learning, this potential is hampered due to this lack of annotated
data. In this work we present CreoleVal, a collection of benchmark datasets
spanning 8 different NLP tasks, covering up to 28 Creole languages; it is an
aggregate of brand new development datasets for machine comprehension, relation
classification, and machine translation for Creoles, in addition to a practical
gateway to a handful of preexisting benchmarks. For each benchmark, we conduct
baseline experiments in a zero-shot setting in order to further ascertain the
capabilities and limitations of transfer learning for Creoles. Ultimately, the
goal of CreoleVal is to empower research on Creoles in NLP and computational
linguistics. We hope this resource will contribute to technological inclusion
for Creole language users around the globe.