Identifying and Adapting Transformer-Components Responsible for Gender
  Bias in an English Language Model
Abhijith Chintam Rahel Beloch Willem Zuidema Michael Hanna Oskar van der Wal
Language models (LMs) exhibit and amplify many types of undesirable biases
learned from the training data, including gender bias. However, we lack tools
for effectively and efficiently changing this behavior without hurting general
language modeling performance. In this paper, we study three methods for
identifying causal relations between LM components and particular output:
causal mediation analysis, automated circuit discovery and our novel, efficient
method called DiffMask+ based on differential masking. We apply the methods to
GPT-2 small and the problem of gender bias, and use the discovered sets of
components to perform parameter-efficient fine-tuning for bias mitigation. Our
results show significant overlap in the identified components (despite huge
differences in the computational requirements of the methods) as well as
success in mitigating gender bias, with less damage to general language
modeling compared to full model fine-tuning. However, our work also underscores
the difficulty of defining and measuring bias, and the sensitivity of causal
discovery procedures to dataset choice. We hope our work can contribute to more
attention for dataset development, and lead to more effective mitigation
strategies for other types of bias.