Contrastive Learning of Person-independent Representations for Facial
  Action Unit Detection
Yong Li Shiguang Shan
Facial action unit (AU) detection, aiming to classify AU present in the
facial image, has long suffered from insufficient AU annotations. In this
paper, we aim to mitigate this data scarcity issue by learning AU
representations from a large number of unlabelled facial videos in a
contrastive learning paradigm. We formulate the self-supervised AU
representation learning signals in two-fold: (1) AU representation should be
frame-wisely discriminative within a short video clip; (2) Facial frames
sampled from different identities but show analogous facial AUs should have
consistent AU representations. As to achieve these goals, we propose to
contrastively learn the AU representation within a video clip and devise a
cross-identity reconstruction mechanism to learn the person-independent
representations. Specially, we adopt a margin-based temporal contrastive
learning paradigm to perceive the temporal AU coherence and evolution
characteristics within a clip that consists of consecutive input facial frames.
Moreover, the cross-identity reconstruction mechanism facilitates pushing the
faces from different identities but show analogous AUs close in the latent
embedding space. Experimental results on three public AU datasets demonstrate
that the learned AU representation is discriminative for AU detection. Our
method outperforms other contrastive learning methods and significantly closes
the performance gap between the self-supervised and supervised AU detection
approaches.