Towards Geometric-Photometric Joint Alignment for Facial Mesh
  Registration
Xizhi Wang Yaxiong Wang Mengjian Li
This paper presents a Geometric-Photometric Joint Alignment(GPJA) method, for
accurately aligning human expressions by combining geometry and photometric
information. Common practices for registering human heads typically involve
aligning landmarks with facial template meshes using geometry processing
approaches, but often overlook photometric consistency. GPJA overcomes this
limitation by leveraging differentiable rendering to align vertices with target
expressions, achieving joint alignment in geometry and photometric appearances
automatically, without the need for semantic annotation or aligned meshes for
training. It features a holistic rendering alignment strategy and a multiscale
regularized optimization for robust and fast convergence. The method utilizes
derivatives at vertex positions for supervision and employs a gradient-based
algorithm which guarantees smoothness and avoids topological defects during the
geometry evolution. Experimental results demonstrate faithful alignment under
various expressions, surpassing the conventional ICP-based methods and the
state-of-the-art deep learning based method. In practical, our method enhances
the efficiency of obtaining topology-consistent face models from multi-view
stereo facial scanning.