Disentangling the Roles of Target-Side Transfer and Regularization in
  Multilingual Machine Translation
Yan Meng Christof Monz
Multilingual Machine Translation (MMT) benefits from knowledge transfer
across different language pairs. However, improvements in one-to-many
translation compared to many-to-one translation are only marginal and sometimes
even negligible. This performance discrepancy raises the question of to what
extent positive transfer plays a role on the target-side for one-to-many MT. In
this paper, we conduct a large-scale study that varies the auxiliary target
side languages along two dimensions, i.e., linguistic similarity and corpus
size, to show the dynamic impact of knowledge transfer on the main language
pairs. We show that linguistically similar auxiliary target languages exhibit
strong ability to transfer positive knowledge. With an increasing size of
similar target languages, the positive transfer is further enhanced to benefit
the main language pairs. Meanwhile, we find distant auxiliary target languages
can also unexpectedly benefit main language pairs, even with minimal positive
transfer ability. Apart from transfer, we show distant auxiliary target
languages can act as a regularizer to benefit translation performance by
enhancing the generalization and model inference calibration.