SCALE: Synergized Collaboration of Asymmetric Language Translation
  Engines
Xin Cheng Xun Wang Tao Ge Si-Qing Chen Furu Wei Dongyan Zhao Rui Yan
In this paper, we introduce SCALE, a collaborative framework that connects
compact Specialized Translation Models (STMs) and general-purpose Large
Language Models (LLMs) as one unified translation engine. By introducing
translation from STM into the triplet in-context demonstrations, SCALE unlocks
refinement and pivoting ability of LLM, thus mitigating language bias of LLM
and parallel data bias of STM, enhancing LLM speciality without sacrificing
generality, and facilitating continual learning without expensive LLM
fine-tuning. Our comprehensive experiments show that SCALE significantly
outperforms both few-shot LLMs (GPT-4) and specialized models (NLLB) in
challenging low-resource settings. Moreover, in Xhosa to English translation,
SCALE experiences consistent improvement by a 4 BLEURT score without tuning LLM
and surpasses few-shot GPT-4 by 2.5 COMET score and 3.8 BLEURT score when
equipped with a compact model consisting of merely 600M parameters. SCALE could
also effectively exploit the existing language bias of LLMs by using an
English-centric STM as a pivot for translation between any language pairs,
outperforming few-shot GPT-4 by an average of 6 COMET points across eight
translation directions. Furthermore we provide an in-depth analysis of SCALE's
robustness, translation characteristics, and latency costs, providing solid
foundation for future studies exploring the potential synergy between LLMs and
more specialized, task-specific models.