Addressing Racial Bias in Facial Emotion Recognition
Alex Fan Xingshuo Xiao Peter Washington
Fairness in deep learning models trained with high-dimensional inputs and
subjective labels remains a complex and understudied area. Facial emotion
recognition, a domain where datasets are often racially imbalanced, can lead to
models that yield disparate outcomes across racial groups. This study focuses
on analyzing racial bias by sub-sampling training sets with varied racial
distributions and assessing test performance across these simulations. Our
findings indicate that smaller datasets with posed faces improve on both
fairness and performance metrics as the simulations approach racial balance.
Notably, the F1-score increases by $27.2\%$ points, and demographic parity
increases by $15.7\%$ points on average across the simulations. However, in
larger datasets with greater facial variation, fairness metrics generally
remain constant, suggesting that racial balance by itself is insufficient to
achieve parity in test performance across different racial groups.