Language Representation Projection: Can We Transfer Factual Knowledge
  across Languages in Multilingual Language Models?
Shaoyang Xu Junzhuo Li Deyi Xiong
Multilingual pretrained language models serve as repositories of multilingual
factual knowledge. Nevertheless, a substantial performance gap of factual
knowledge probing exists between high-resource languages and low-resource
languages, suggesting limited implicit factual knowledge transfer across
languages in multilingual pretrained language models. This paper investigates
the feasibility of explicitly transferring relatively rich factual knowledge
from English to non-English languages. To accomplish this, we propose two
parameter-free $\textbf{L}$anguage $\textbf{R}$epresentation
$\textbf{P}$rojection modules (LRP2). The first module converts non-English
representations into English-like equivalents, while the second module reverts
English-like representations back into representations of the corresponding
non-English language. Experimental results on the mLAMA dataset demonstrate
that LRP2 significantly improves factual knowledge retrieval accuracy and
facilitates knowledge transferability across diverse non-English languages. We
further investigate the working mechanism of LRP2 from the perspectives of
representation space and cross-lingual knowledge neuron.