Do Multilingual Language Models Think Better in English?
Julen Etxaniz Gorka Azkune Aitor Soroa Oier Lopez de Lacalle Mikel Artetxe
Translate-test is a popular technique to improve the performance of
multilingual language models. This approach works by translating the input into
English using an external machine translation system, and running inference
over the translated input. However, these improvements can be attributed to the
use of a separate translation system, which is typically trained on large
amounts of parallel data not seen by the language model. In this work, we
introduce a new approach called self-translate, which overcomes the need of an
external translation system by leveraging the few-shot translation capabilities
of multilingual language models. Experiments over 5 tasks show that
self-translate consistently outperforms direct inference, demonstrating that
language models are unable to leverage their full multilingual potential when
prompted in non-English languages. Our code is available at
https://github.com/juletx/self-translate.