3D Face Alignment Through Fusion of Head Pose Information and Features
Jaehyun So Youngjoon Han
The ability of humans to infer head poses from face shapes, and vice versa,
indicates a strong correlation between the two. Accordingly, recent studies on
face alignment have employed head pose information to predict facial landmarks
in computer vision tasks. In this study, we propose a novel method that employs
head pose information to improve face alignment performance by fusing said
information with the feature maps of a face alignment network, rather than
simply using it to initialize facial landmarks. Furthermore, the proposed
network structure performs robust face alignment through a dual-dimensional
network using multidimensional features represented by 2D feature maps and a 3D
heatmap. For effective dense face alignment, we also propose a prediction
method for facial geometric landmarks through training based on knowledge
distillation using predicted keypoints. We experimentally assessed the
correlation between the predicted facial landmarks and head pose information,
as well as variations in the accuracy of facial landmarks with respect to the
quality of head pose information. In addition, we demonstrated the
effectiveness of the proposed method through a competitive performance
comparison with state-of-the-art methods on the AFLW2000-3D, AFLW, and BIWI
datasets.