Zero-shot Cross-lingual Transfer without Parallel Corpus
Yuyang Zhang Xiaofeng Han Baojun Wang
Recently, although pre-trained language models have achieved great success on
multilingual NLP (Natural Language Processing) tasks, the lack of training data
on many tasks in low-resource languages still limits their performance. One
effective way of solving that problem is to transfer knowledge from
rich-resource languages to low-resource languages. However, many previous works
on cross-lingual transfer rely heavily on the parallel corpus or translation
models, which are often difficult to obtain. We propose a novel approach to
conduct zero-shot cross-lingual transfer with a pre-trained model. It consists
of a Bilingual Task Fitting module that applies task-related bilingual
information alignment; a self-training module generates pseudo soft and hard
labels for unlabeled data and utilizes them to conduct self-training. We got
the new SOTA on different tasks without any dependencies on the parallel corpus
or translation models.