Leveraging Synthetic Data for Generalizable and Fair Facial Action Unit
  Detection
Liupei Lu Yufeng Yin Yuming Gu Yizhen Wu Pratusha Prasad Yajie Zhao Mohammad Soleymani
Facial action unit (AU) detection is a fundamental block for objective facial
expression analysis. Supervised learning approaches require a large amount of
manual labeling which is costly. The limited labeled data are also not diverse
in terms of gender which can affect model fairness. In this paper, we propose
to use synthetically generated data and multi-source domain adaptation (MSDA)
to address the problems of the scarcity of labeled data and the diversity of
subjects. Specifically, we propose to generate a diverse dataset through
synthetic facial expression re-targeting by transferring the expressions from
real faces to synthetic avatars. Then, we use MSDA to transfer the AU detection
knowledge from a real dataset and the synthetic dataset to a target dataset.
Instead of aligning the overall distributions of different domains, we propose
Paired Moment Matching (PM2) to align the features of the paired real and
synthetic data with the same facial expression. To further improve gender
fairness, PM2 matches the features of the real data with a female and a male
synthetic image. Our results indicate that synthetic data and the proposed
model improve both AU detection performance and fairness across genders,
demonstrating its potential to solve AU detection in-the-wild.