DA-Net: A Disentangled and Adaptive Network for Multi-Source
  Cross-Lingual Transfer Learning
Ling Ge Chunming Hu Guanghui Ma Jihong Liu Hong Zhang
Multi-Source cross-lingual transfer learning deals with the transfer of task
knowledge from multiple labelled source languages to an unlabeled target
language under the language shift. Existing methods typically focus on
weighting the predictions produced by language-specific classifiers of
different sources that follow a shared encoder. However, all source languages
share the same encoder, which is updated by all these languages. The extracted
representations inevitably contain different source languages' information,
which may disturb the learning of the language-specific classifiers.
Additionally, due to the language gap, language-specific classifiers trained
with source labels are unable to make accurate predictions for the target
language. Both facts impair the model's performance. To address these
challenges, we propose a Disentangled and Adaptive Network (DA-Net). Firstly,
we devise a feedback-guided collaborative disentanglement method that seeks to
purify input representations of classifiers, thereby mitigating mutual
interference from multiple sources. Secondly, we propose a class-aware parallel
adaptation method that aligns class-level distributions for each source-target
language pair, thereby alleviating the language pairs' language gap.
Experimental results on three different tasks involving 38 languages validate
the effectiveness of our approach.