Text-Driven Diverse Facial Texture Generation via Progressive
  Latent-Space Refinement
Chi Wang Junming Huang Rong Zhang Qi Wang Haotian Yang Haibin Huang Chongyang Ma Weiwei Xu
Automatic 3D facial texture generation has gained significant interest
recently. Existing approaches may not support the traditional physically based
rendering pipeline or rely on 3D data captured by Light Stage. Our key
contribution is a progressive latent space refinement approach that can
bootstrap from 3D Morphable Models (3DMMs)-based texture maps generated from
facial images to generate high-quality and diverse PBR textures, including
albedo, normal, and roughness. It starts with enhancing Generative Adversarial
Networks (GANs) for text-guided and diverse texture generation. To this end, we
design a self-supervised paradigm to overcome the reliance on ground truth 3D
textures and train the generative model with only entangled texture maps.
Besides, we foster mutual enhancement between GANs and Score Distillation
Sampling (SDS). SDS boosts GANs with more generative modes, while GANs promote
more efficient optimization of SDS. Furthermore, we introduce an edge-aware SDS
for multi-view consistent facial structure. Experiments demonstrate that our
method outperforms existing 3D texture generation methods regarding
photo-realistic quality, diversity, and efficiency.