Enhancing Cross-lingual Natural Language Inference by Soft Prompting
  with Multilingual Verbalizer
Shuang Li Xuming Hu Aiwei Liu Yawen Yang Fukun Ma Philip S. Yu Lijie Wen
Cross-lingual natural language inference is a fundamental problem in
cross-lingual language understanding. Many recent works have used prompt
learning to address the lack of annotated parallel corpora in XNLI. However,
these methods adopt discrete prompting by simply translating the templates to
the target language and need external expert knowledge to design the templates.
Besides, discrete prompts of human-designed template words are not trainable
vectors and can not be migrated to target languages in the inference stage
flexibly. In this paper, we propose a novel Soft prompt learning framework with
the Multilingual Verbalizer (SoftMV) for XNLI. SoftMV first constructs
cloze-style question with soft prompts for the input sample. Then we leverage
bilingual dictionaries to generate an augmented multilingual question for the
original question. SoftMV adopts a multilingual verbalizer to align the
representations of original and augmented multilingual questions into the same
semantic space with consistency regularization. Experimental results on XNLI
demonstrate that SoftMV can achieve state-of-the-art performance and
significantly outperform the previous methods under the few-shot and full-shot
cross-lingual transfer settings.