Investigating Bias in Multilingual Language Models: Cross-Lingual
  Transfer of Debiasing Techniques
Manon Reusens Philipp Borchert Margot Mieskes Jochen De Weerdt Bart Baesens
This paper investigates the transferability of debiasing techniques across
different languages within multilingual models. We examine the applicability of
these techniques in English, French, German, and Dutch. Using multilingual BERT
(mBERT), we demonstrate that cross-lingual transfer of debiasing techniques is
not only feasible but also yields promising results. Surprisingly, our findings
reveal no performance disadvantages when applying these techniques to
non-English languages. Using translations of the CrowS-Pairs dataset, our
analysis identifies SentenceDebias as the best technique across different
languages, reducing bias in mBERT by an average of 13%. We also find that
debiasing techniques with additional pretraining exhibit enhanced cross-lingual
effectiveness for the languages included in the analyses, particularly in
lower-resource languages. These novel insights contribute to a deeper
understanding of bias mitigation in multilingual language models and provide
practical guidance for debiasing techniques in different language contexts.