Language Model is a Branch Predictor for Simultaneous Machine
  Translation
Aoxiong Yin Tianyun Zhong Haoyuan Li Siliang Tang Zhou Zhao
The primary objective of simultaneous machine translation (SiMT) is to
minimize latency while preserving the quality of the final translation. Drawing
inspiration from CPU branch prediction techniques, we propose incorporating
branch prediction techniques in SiMT tasks to reduce translation latency.
Specifically, we utilize a language model as a branch predictor to predict
potential branch directions, namely, future source words. Subsequently, we
utilize the predicted source words to decode the output in advance. When the
actual source word deviates from the predicted source word, we use the real
source word to decode the output again, replacing the predicted output. To
further reduce computational costs, we share the parameters of the encoder and
the branch predictor, and utilize a pre-trained language model for
initialization. Our proposed method can be seamlessly integrated with any SiMT
model. Extensive experimental results demonstrate that our approach can improve
translation quality and latency at the same time. Our code is available at
https://github.com/YinAoXiong/simt_branch_predictor .