DSFNet: Dual Space Fusion Network for Occlusion-Robust 3D Dense Face
  Alignment
Heyuan Li Bo Wang Yu Cheng Mohan Kankanhalli Robby T. Tan
Sensitivity to severe occlusion and large view angles limits the usage
scenarios of the existing monocular 3D dense face alignment methods. The
state-of-the-art 3DMM-based method, directly regresses the model's
coefficients, underutilizing the low-level 2D spatial and semantic information,
which can actually offer cues for face shape and orientation. In this work, we
demonstrate how modeling 3D facial geometry in image and model space jointly
can solve the occlusion and view angle problems. Instead of predicting the
whole face directly, we regress image space features in the visible facial
region by dense prediction first. Subsequently, we predict our model's
coefficients based on the regressed feature of the visible regions, leveraging
the prior knowledge of whole face geometry from the morphable models to
complete the invisible regions. We further propose a fusion network that
combines the advantages of both the image and model space predictions to
achieve high robustness and accuracy in unconstrained scenarios. Thanks to the
proposed fusion module, our method is robust not only to occlusion and large
pitch and roll view angles, which is the benefit of our image space approach,
but also to noise and large yaw angles, which is the benefit of our model space
method. Comprehensive evaluations demonstrate the superior performance of our
method compared with the state-of-the-art methods. On the 3D dense face
alignment task, we achieve 3.80% NME on the AFLW2000-3D dataset, which
outperforms the state-of-the-art method by 5.5%. Code is available at
https://github.com/lhyfst/DSFNet.