EBBS: An Ensemble with Bi-Level Beam Search for Zero-Shot Machine
  Translation
Yuqiao Wen Behzad Shayegh Chenyang Huang Yanshuai Cao Lili Mou
The ability of zero-shot translation emerges when we train a multilingual
model with certain translation directions; the model can then directly
translate in unseen directions. Alternatively, zero-shot translation can be
accomplished by pivoting through a third language (e.g., English). In our work,
we observe that both direct and pivot translations are noisy and achieve less
satisfactory performance. We propose EBBS, an ensemble method with a novel
bi-level beam search algorithm, where each ensemble component explores its own
prediction step by step at the lower level but they are synchronized by a "soft
voting" mechanism at the upper level. Results on two popular multilingual
translation datasets show that EBBS consistently outperforms direct and pivot
translations as well as existing ensemble techniques. Further, we can distill
the ensemble's knowledge back to the multilingual model to improve inference
efficiency; profoundly, our EBBS-based distillation does not sacrifice, or even
improves, the translation quality.