Toward Fair Facial Expression Recognition with Improved Distribution
  Alignment
Mojtaba Kolahdouzi Ali Etemad
We present a novel approach to mitigate bias in facial expression recognition
(FER) models. Our method aims to reduce sensitive attribute information such as
gender, age, or race, in the embeddings produced by FER models. We employ a
kernel mean shrinkage estimator to estimate the kernel mean of the
distributions of the embeddings associated with different sensitive attribute
groups, such as young and old, in the Hilbert space. Using this estimation, we
calculate the maximum mean discrepancy (MMD) distance between the distributions
and incorporate it in the classifier loss along with an adversarial loss, which
is then minimized through the learning process to improve the distribution
alignment. Our method makes sensitive attributes less recognizable for the
model, which in turn promotes fairness. Additionally, for the first time, we
analyze the notion of attractiveness as an important sensitive attribute in FER
models and demonstrate that FER models can indeed exhibit biases towards more
attractive faces. To prove the efficacy of our model in reducing bias regarding
different sensitive attributes (including the newly proposed attractiveness
attribute), we perform several experiments on two widely used datasets, CelebA
and RAF-DB. The results in terms of both accuracy and fairness measures
outperform the state-of-the-art in most cases, demonstrating the effectiveness
of the proposed method.