Simple and Effective Input Reformulations for Translation
Brian Yu Hansen Lillemark Kurt Keutzer
Foundation language models learn from their finetuning input context in
different ways. In this paper, we reformulate inputs during finetuning for
challenging translation tasks, leveraging model strengths from pretraining in
novel ways to improve downstream performance. These reformulations are simple
data level modifications, require no additional collection of training data or
modification of data at inference time. They can be applied either on single
language pair translation tasks or massively multilingual translation tasks.
Experiments with these techniques demonstrate significant performance
improvements up to $\textbf{3.5 chrF++ on the Flores200 translation
benchmark}$. We hope our research accessibly improves finetuning data
efficiency, enabling more effective training to scalably improve
state-of-the-art performance. Our code is released
$\href{https://github.com/bri25yu/LanguageModelExperimentation}{here}.$