Artist-Friendly Relightable and Animatable Neural Heads
Yingyan Xu Prashanth Chandran Sebastian Weiss Markus Gross Gaspard Zoss Derek Bradley
An increasingly common approach for creating photo-realistic digital avatars
is through the use of volumetric neural fields. The original neural radiance
field (NeRF) allowed for impressive novel view synthesis of static heads when
trained on a set of multi-view images, and follow up methods showed that these
neural representations can be extended to dynamic avatars. Recently, new
variants also surpassed the usual drawback of baked-in illumination in neural
representations, showing that static neural avatars can be relit in any
environment. In this work we simultaneously tackle both the motion and
illumination problem, proposing a new method for relightable and animatable
neural heads. Our method builds on a proven dynamic avatar approach based on a
mixture of volumetric primitives, combined with a recently-proposed lightweight
hardware setup for relightable neural fields, and includes a novel architecture
that allows relighting dynamic neural avatars performing unseen expressions in
any environment, even with nearfield illumination and viewpoints.