Comparing Biases and the Impact of Multilingual Training across Multiple
  Languages
Sharon Levy Neha Anna John Ling Liu Yogarshi Vyas Jie Ma Yoshinari Fujinuma Miguel Ballesteros Vittorio Castelli Dan Roth
Studies in bias and fairness in natural language processing have primarily
examined social biases within a single language and/or across few attributes
(e.g. gender, race). However, biases can manifest differently across various
languages for individual attributes. As a result, it is critical to examine
biases within each language and attribute. Of equal importance is to study how
these biases compare across languages and how the biases are affected when
training a model on multilingual data versus monolingual data. We present a
bias analysis across Italian, Chinese, English, Hebrew, and Spanish on the
downstream sentiment analysis task to observe whether specific demographics are
viewed more positively. We study bias similarities and differences across these
languages and investigate the impact of multilingual vs. monolingual training
data. We adapt existing sentiment bias templates in English to Italian,
Chinese, Hebrew, and Spanish for four attributes: race, religion, nationality,
and gender. Our results reveal similarities in bias expression such as
favoritism of groups that are dominant in each language's culture (e.g.
majority religions and nationalities). Additionally, we find an increased
variation in predictions across protected groups, indicating bias
amplification, after multilingual finetuning in comparison to multilingual
pretraining.