Teaching Large Language Models an Unseen Language on the Fly
Chen Zhang Xiao Liu Jiuheng Lin Yansong Feng
Existing large language models struggle to support numerous low-resource
languages, particularly the extremely low-resource ones where there is minimal
training data available for effective parameter updating. We thus investigate
whether LLMs can learn a new language on the fly solely through prompting. To
study this question, we collect a research suite for Zhuang, a language
supported by no LLMs currently. We introduce \textsc{DiPMT++}, a framework for
adapting LLMs to unseen languages by in-context learning. Using a dictionary
and only 5K parallel sentences, \textsc{DiPMT++} significantly enhances the
performance of GPT-4 from 0 to 16 BLEU for Chinese-to-Zhuang translation and
achieves 32 BLEU for Zhuang-to-Chinese translation. Furthermore, we demonstrate
the practical utility of this framework in aiding humans to translate
completely unseen languages, which could contribute to the preservation of
linguistic diversity.