In-context Learning as Maintaining Coherency: A Study of On-the-fly
  Machine Translation Using Large Language Models
Suzanna Sia Kevin Duh
The phenomena of in-context learning has typically been thought of as
"learning from examples". In this work which focuses on Machine Translation, we
present a perspective of in-context learning as the desired generation task
maintaining coherency with its context, i.e., the prompt examples. We first
investigate randomly sampled prompts across 4 domains, and find that
translation performance improves when shown in-domain prompts. Next, we
investigate coherency for the in-domain setting, which uses prompt examples
from a moving window. We study this with respect to other factors that have
previously been identified in the literature such as length, surface similarity
and sentence embedding similarity. Our results across 3 models (GPTNeo2.7B,
Bloom3B, XGLM2.9B), and three translation directions
(\texttt{en}$\rightarrow$\{\texttt{pt, de, fr}\}) suggest that the long-term
coherency of the prompts and the test sentence is a good indicator of
downstream translation performance. In doing so, we demonstrate the efficacy of
In-context Machine Translation for on-the-fly adaptation.