Evaluating Bias and Fairness in Gender-Neutral Pretrained
  Vision-and-Language Models
Laura Cabello Emanuele Bugliarello Stephanie Brandl Desmond Elliott
Pretrained machine learning models are known to perpetuate and even amplify
existing biases in data, which can result in unfair outcomes that ultimately
impact user experience. Therefore, it is crucial to understand the mechanisms
behind those prejudicial biases to ensure that model performance does not
result in discriminatory behaviour toward certain groups or populations. In
this work, we define gender bias as our case study. We quantify bias
amplification in pretraining and after fine-tuning on three families of
vision-and-language models. We investigate the connection, if any, between the
two learning stages, and evaluate how bias amplification reflects on model
performance. Overall, we find that bias amplification in pretraining and after
fine-tuning are independent. We then examine the effect of continued
pretraining on gender-neutral data, finding that this reduces group
disparities, i.e., promotes fairness, on VQAv2 and retrieval tasks without
significantly compromising task performance.