Facial Reenactment Through a Personalized Generator
Ariel Elazary Yotam Nitzan Daniel Cohen-Or
In recent years, the role of image generative models in facial reenactment
has been steadily increasing. Such models are usually subject-agnostic and
trained on domain-wide datasets. The appearance of the reenacted individual is
learned from a single image, and hence, the entire breadth of the individual's
appearance is not entirely captured, leading these methods to resort to
unfaithful hallucination. Thanks to recent advancements, it is now possible to
train a personalized generative model tailored specifically to a given
individual. In this paper, we propose a novel method for facial reenactment
using a personalized generator. We train the generator using frames from a
short, yet varied, self-scan video captured using a simple commodity camera.
Images synthesized by the personalized generator are guaranteed to preserve
identity. The premise of our work is that the task of reenactment is thus
reduced to accurately mimicking head poses and expressions. To this end, we
locate the desired frames in the latent space of the personalized generator
using carefully designed latent optimization. Through extensive evaluation, we
demonstrate state-of-the-art performance for facial reenactment. Furthermore,
we show that since our reenactment takes place in a semantic latent space, it
can be semantically edited and stylized in post-processing.