Democratizing the Creation of Animatable Facial Avatars
Yilin Zhu Dalton Omens Haodi He Ron Fedkiw
In high-end visual effects pipelines, a customized (and expensive) light
stage system is (typically) used to scan an actor in order to acquire both
geometry and texture for various expressions. Aiming towards democratization,
we propose a novel pipeline for obtaining geometry and texture as well as
enough expression information to build a customized person-specific animation
rig without using a light stage or any other high-end hardware (or manual
cleanup). A key novel idea consists of warping real-world images to align with
the geometry of a template avatar and subsequently projecting the warped image
into the template avatar's texture; importantly, this allows us to leverage
baked-in real-world lighting/texture information in order to create surrogate
facial features (and bridge the domain gap) for the sake of geometry
reconstruction. Not only can our method be used to obtain a neutral expression
geometry and de-lit texture, but it can also be used to improve avatars after
they have been imported into an animation system (noting that such imports tend
to be lossy, while also hallucinating various features). Since a default
animation rig will contain template expressions that do not correctly
correspond to those of a particular individual, we use a Simon Says approach to
capture various expressions and build a person-specific animation rig (that
moves like they do). Our aforementioned warping/projection method has high
enough efficacy to reconstruct geometry corresponding to each expressions.