Systematic Offensive Stereotyping (SOS) Bias in Language Models
Fatma Elsafoury
Research has shown that language models (LMs) are socially biased. However,
toxicity and offensive stereotyping bias in LMs are understudied. In this
paper, we investigate the systematic offensive stereotype (SOS) bias in LMs. We
propose a method to measure it. Then, we validate the SOS bias and investigate
the effectiveness of debias methods from the literature on removing it.
Finally, we investigate the impact of the SOS bias in LMs on their performance
and their fairness on the task of hate speech detection. Our results suggest
that all the inspected LMs are SOS biased. The results suggest that the SOS
bias in LMs is reflective of the hate experienced online by the inspected
marginalized groups. The results indicate that removing the SOS bias in LMs,
using a popular debias method from the literature, leads to worse SOS bias
scores. Finally, Our results show no strong evidence that the SOS bias in LMs
is impactful on their performance on hate speech detection. On the other hand,
there is evidence that the SOS bias in LMs is impactful on their fairness.