DELIFFAS: Deformable Light Fields for Fast Avatar Synthesis
Youngjoong Kwon Lingjie Liu Henry Fuchs Marc Habermann Christian Theobalt
Generating controllable and photorealistic digital human avatars is a
long-standing and important problem in Vision and Graphics. Recent methods have
shown great progress in terms of either photorealism or inference speed while
the combination of the two desired properties still remains unsolved. To this
end, we propose a novel method, called DELIFFAS, which parameterizes the
appearance of the human as a surface light field that is attached to a
controllable and deforming human mesh model. At the core, we represent the
light field around the human with a deformable two-surface parameterization,
which enables fast and accurate inference of the human appearance. This allows
perceptual supervision on the full image compared to previous approaches that
could only supervise individual pixels or small patches due to their slow
runtime. Our carefully designed human representation and supervision strategy
leads to state-of-the-art synthesis results and inference time. The video
results and code are available at
https://vcai.mpi-inf.mpg.de/projects/DELIFFAS.