Self-supervised Learning for Enhancing Geometrical Modeling in 3D-Aware
  Generative Adversarial Network
Jiarong Guo Xiaogang Xu Hengshuang Zhao
3D-aware Generative Adversarial Networks (3D-GANs) currently exhibit
artifacts in their 3D geometrical modeling, such as mesh imperfections and
holes. These shortcomings are primarily attributed to the limited availability
of annotated 3D data, leading to a constrained "valid latent area" for
satisfactory modeling. To address this, we present a Self-Supervised Learning
(SSL) technique tailored as an auxiliary loss for any 3D-GAN, designed to
improve its 3D geometrical modeling capabilities. Our approach pioneers an
inversion technique for 3D-GANs, integrating an encoder that performs adaptive
spatially-varying range operations. Utilizing this inversion, we introduce the
Cyclic Generative Constraint (CGC), aiming to densify the valid latent space.
The CGC operates via augmented local latent vectors that maintain the same
geometric form, and it imposes constraints on the cycle path outputs,
specifically the generator-encoder-generator sequence. This SSL methodology
seamlessly integrates with the inherent GAN loss, ensuring the integrity of
pre-existing 3D-GAN architectures without necessitating alterations. We
validate our approach with comprehensive experiments across various datasets
and architectures, underscoring its efficacy. Our project website:
https://3dgan-ssl.github.io