Preface: A Data-driven Volumetric Prior for Few-shot Ultra
  High-resolution Face Synthesis
Marcel C. Bühler Kripasindhu Sarkar Tanmay Shah Gengyan Li Daoye Wang Leonhard Helminger Sergio Orts-Escolano Dmitry Lagun Otmar Hilliges Thabo Beeler Abhimitra Meka
NeRFs have enabled highly realistic synthesis of human faces including
complex appearance and reflectance effects of hair and skin. These methods
typically require a large number of multi-view input images, making the process
hardware intensive and cumbersome, limiting applicability to unconstrained
settings. We propose a novel volumetric human face prior that enables the
synthesis of ultra high-resolution novel views of subjects that are not part of
the prior's training distribution. This prior model consists of an
identity-conditioned NeRF, trained on a dataset of low-resolution multi-view
images of diverse humans with known camera calibration. A simple sparse
landmark-based 3D alignment of the training dataset allows our model to learn a
smooth latent space of geometry and appearance despite a limited number of
training identities. A high-quality volumetric representation of a novel
subject can be obtained by model fitting to 2 or 3 camera views of arbitrary
resolution. Importantly, our method requires as few as two views of casually
captured images as input at inference time.