Boosting Cross-lingual Transferability in Multilingual Models via
  In-Context Learning
Sunkyoung Kim Dayeon Ki Yireun Kim Jinsik Lee
Existing cross-lingual transfer (CLT) prompting methods are only concerned
with monolingual demonstration examples in the source language. In this paper,
we propose In-CLT, a novel cross-lingual transfer prompting method that
leverages both source and target languages to construct the demonstration
examples. We conduct comprehensive evaluations on multilingual benchmarks,
focusing on question answering tasks. Experiment results show that In-CLT
prompt not only improves multilingual models' cross-lingual transferability,
but also demonstrates remarkable unseen language generalization ability. In-CLT
prompting, in particular, improves model performance by 10 to 20\% points on
average when compared to prior cross-lingual transfer approaches. We also
observe the surprising performance gain on the other multilingual benchmarks,
especially in reasoning tasks. Furthermore, we investigate the relationship
between lexical similarity and pre-training corpora in terms of the
cross-lingual transfer gap.