Domain-Adaptive Full-Face Gaze Estimation via Novel-View-Synthesis and
  Feature Disentanglement
Jiawei Qin Takuru Shimoyama Xucong Zhang Yusuke Sugano
Along with the recent development of deep neural networks, appearance-based
gaze estimation has succeeded considerably when training and testing within the
same domain. Compared to the within-domain task, the variance of different
domains makes the cross-domain performance drop severely, preventing gaze
estimation deployment in real-world applications. Among all the factors, ranges
of head pose and gaze are believed to play a significant role in the final
performance of gaze estimation, while collecting large ranges of data is
expensive. This work proposes an effective model training pipeline consisting
of a training data synthesis and a gaze estimation model for unsupervised
domain adaptation. The proposed data synthesis leverages the single-image 3D
reconstruction to expand the range of the head poses from the source domain
without requiring a 3D facial shape dataset. To bridge the inevitable gap
between synthetic and real images, we further propose an unsupervised domain
adaptation method suitable for synthetic full-face data. We propose a
disentangling autoencoder network to separate gaze-related features and
introduce background augmentation consistency loss to utilize the
characteristics of the synthetic source domain. Through comprehensive
experiments, we show that the model only using monocular-reconstructed
synthetic training data can perform comparably to real data with a large label
range. Our proposed domain adaptation approach further improves the performance
on multiple target domains. The code and data will be available at
\url{https://github.com/ut-vision/AdaptiveGaze}.