Variable-length Neural Interlingua Representations for Zero-shot Neural
  Machine Translation
Zhuoyuan Mao Haiyue Song Raj Dabre Chenhui Chu Sadao Kurohashi
The language-independency of encoded representations within multilingual
neural machine translation (MNMT) models is crucial for their generalization
ability on zero-shot translation. Neural interlingua representations have been
shown as an effective method for achieving this. However, fixed-length neural
interlingua representations introduced in previous work can limit its
flexibility and representation ability. In this study, we introduce a novel
method to enhance neural interlingua representations by making their length
variable, thereby overcoming the constraint of fixed-length neural interlingua
representations. Our empirical results on zero-shot translation on OPUS, IWSLT,
and Europarl datasets demonstrate stable model convergence and superior
zero-shot translation results compared to fixed-length neural interlingua
representations. However, our analysis reveals the suboptimal efficacy of our
approach in translating from certain source languages, wherein we pinpoint the
defective model component in our proposed method.