Zero-shot cross-lingual transfer in instruction tuning of large language
  model
Nadezhda Chirkova Vassilina Nikoulina
Instruction tuning (IT) is widely used to teach pretrained large language
models (LLMs) to follow arbitrary instructions, but is under-studied in
multilingual settings. In this work, we conduct a systematic study of zero-shot
cross-lingual transfer in IT, when an LLM is instruction-tuned on English-only
data and then tested on user prompts in other languages. We investigate the
influence of model configuration choices and devise a multi-facet evaluation
strategy for multilingual instruction following. We find that cross-lingual
transfer does happen successfully in IT even if all stages of model training
are English-centric, but only if multiliguality is taken into account in
hyperparameter tuning and with large enough IT data. English-trained LLMs are
capable of generating correct-language, comprehensive and helpful responses in
the other languages, but suffer from low factuality and may occasionally have
fluency errors.