Bias Mitigation in Fine-tuning Pre-trained Models for Enhanced Fairness
  and Efficiency
Yixuan Zhang Feng Zhou
Fine-tuning pre-trained models is a widely employed technique in numerous
real-world applications. However, fine-tuning these models on new tasks can
lead to unfair outcomes. This is due to the absence of generalization
guarantees for fairness properties, regardless of whether the original
pre-trained model was developed with fairness considerations. To tackle this
issue, we introduce an efficient and robust fine-tuning framework specifically
designed to mitigate biases in new tasks. Our empirical analysis shows that the
parameters in the pre-trained model that affect predictions for different
demographic groups are different, so based on this observation, we employ a
transfer learning strategy that neutralizes the importance of these influential
weights, determined using Fisher information across demographic groups.
Additionally, we integrate this weight importance neutralization strategy with
a matrix factorization technique, which provides a low-rank approximation of
the weight matrix using fewer parameters, reducing the computational demands.
Experiments on multiple pre-trained models and new tasks demonstrate the
effectiveness of our method.