MC^2: A Multilingual Corpus of Minority Languages in China
Chen Zhang Mingxu Tao Quzhe Huang Jiuheng Lin Zhibin Chen Yansong Feng
Large-scale corpora play a vital role in the construction of large language
models (LLMs). However, existing LLMs exhibit limited abilities in
understanding low-resource languages, including the minority languages in
China, due to a lack of training data. To improve the accessibility of these
languages, we present MC^2, a Multilingual Corpus of Minority Languages in
China, which is the largest open-source corpus so far. It encompasses four
underrepresented languages, i.e., Tibetan, Uyghur, Kazakh in the Kazakh Arabic
script, and Mongolian in the traditional Mongolian script. Notably, two writing
systems in MC^2 are long neglected in previous corpora. As we identify serious
contamination in the low-resource language split in the existing multilingual
corpora, we propose a quality-centric solution for collecting MC^2,
prioritizing quality and accuracy while enhancing representativeness and
diversity. By in-depth analysis, we demonstrate the new research challenges
MC^2 brings, such as long-text modeling and multiplicity of writing systems. We
hope MC^2 can help enhance the equity of the underrepresented languages in
China and provide a reliable data foundation for further research on
low-resource languages.