MTCue: Learning Zero-Shot Control of Extra-Textual Attributes by
  Leveraging Unstructured Context in Neural Machine Translation
Sebastian Vincent Robert Flynn Carolina Scarton
Efficient utilisation of both intra- and extra-textual context remains one of
the critical gaps between machine and human translation. Existing research has
primarily focused on providing individual, well-defined types of context in
translation, such as the surrounding text or discrete external variables like
the speaker's gender. This work introduces MTCue, a novel neural machine
translation (NMT) framework that interprets all context (including discrete
variables) as text. MTCue learns an abstract representation of context,
enabling transferability across different data settings and leveraging similar
attributes in low-resource scenarios. With a focus on a dialogue domain with
access to document and metadata context, we extensively evaluate MTCue in four
language pairs in both translation directions. Our framework demonstrates
significant improvements in translation quality over a parameter-matched
non-contextual baseline, as measured by BLEU (+0.88) and Comet (+1.58).
Moreover, MTCue significantly outperforms a "tagging" baseline at translating
English text. Analysis reveals that the context encoder of MTCue learns a
representation space that organises context based on specific attributes, such
as formality, enabling effective zero-shot control. Pre-training on context
embeddings also improves MTCue's few-shot performance compared to the "tagging"
baseline. Finally, an ablation study conducted on model components and
contextual variables further supports the robustness of MTCue for context-based
NMT.