A Survey on Multilingual Large Language Models: Corpora, Alignment, and
  Bias
Yuemei Xu Ling Hu Jiayi Zhao Zihan Qiu Yuqi Ye Hanwen Gu
Based on the foundation of Large Language Models (LLMs), Multilingual Large
Language Models (MLLMs) have been developed to address the challenges of
multilingual natural language processing tasks, hoping to achieve knowledge
transfer from high-resource to low-resource languages. However, significant
limitations and challenges still exist, such as language imbalance,
multilingual alignment, and inherent bias. In this paper, we aim to provide a
comprehensive analysis of MLLMs, delving deeply into discussions surrounding
these critical issues. First of all, we start by presenting an overview of
MLLMs, covering their evolution, key techniques, and multilingual capacities.
Secondly, we explore widely utilized multilingual corpora for MLLMs' training
and multilingual datasets oriented for downstream tasks that are crucial for
enhancing the cross-lingual capability of MLLMs. Thirdly, we survey the
existing studies on multilingual representations and investigate whether the
current MLLMs can learn a universal language representation. Fourthly, we
discuss bias on MLLMs including its category and evaluation metrics, and
summarize the existing debiasing techniques. Finally, we discuss existing
challenges and point out promising research directions. By demonstrating these
aspects, this paper aims to facilitate a deeper understanding of MLLMs and
their potentiality in various domains.