Skull-to-Face: Anatomy-Guided 3D Facial Reconstruction and Editing
Yongqing Liang Congyi Zhang Junli Zhao Wenping Wang Xin Li
Deducing the 3D face from a skull is an essential but challenging task in
forensic science and archaeology. Existing methods for automated facial
reconstruction yield inaccurate results, suffering from the non-determinative
nature of the problem that a skull with a sparse set of tissue depth cannot
fully determine the skinned face. Additionally, their texture-less results
require further post-processing stages to achieve a photo-realistic appearance.
This paper proposes an end-to-end 3D face reconstruction and exploration tool,
providing textured 3D faces for reference. With the help of state-of-the-art
text-to-image diffusion models and image-based facial reconstruction
techniques, we generate an initial reference 3D face, whose biological profile
aligns with the given skull. We then adapt these initial faces to meet the
statistical expectations of extruded anatomical landmarks on the skull through
an optimization process. The joint statistical distribution of tissue depths is
learned on a small set of anatomical landmarks on the skull. To support further
adjustment, we propose an efficient face adaptation tool to assist users in
tuning tissue depths, either globally or at local regions, while observing
plausible visual feedback. Experiments conducted on a real skull-face dataset
demonstrated the effectiveness of our proposed pipeline in terms of
reconstruction accuracy, diversity, and stability.