SphereHead: Stable 3D Full-head Synthesis with Spherical Tri-plane
  Representation
Heyuan Li Ce Chen Tianhao Shi Yuda Qiu Sizhe An Guanying Chen Xiaoguang Han
While recent advances in 3D-aware Generative Adversarial Networks (GANs) have
aided the development of near-frontal view human face synthesis, the challenge
of comprehensively synthesizing a full 3D head viewable from all angles still
persists. Although PanoHead proves the possibilities of using a large-scale
dataset with images of both frontal and back views for full-head synthesis, it
often causes artifacts for back views. Based on our in-depth analysis, we found
the reasons are mainly twofold. First, from network architecture perspective,
we found each plane in the utilized tri-plane/tri-grid representation space
tends to confuse the features from both sides, causing "mirroring" artifacts
(e.g., the glasses appear in the back). Second, from data supervision aspect,
we found that existing discriminator training in 3D GANs mainly focuses on the
quality of the rendered image itself, and does not care much about its
plausibility with the perspective from which it was rendered. This makes it
possible to generate "face" in non-frontal views, due to its easiness to fool
the discriminator. In response, we propose SphereHead, a novel tri-plane
representation in the spherical coordinate system that fits the human head's
geometric characteristics and efficiently mitigates many of the generated
artifacts. We further introduce a view-image consistency loss for the
discriminator to emphasize the correspondence of the camera parameters and the
images. The combination of these efforts results in visually superior outcomes
with significantly fewer artifacts. Our code and dataset are publicly available
at https://lhyfst.github.io/spherehead.