IFaceUV: Intuitive Motion Facial Image Generation by Identity
  Preservation via UV map
Hansol Lee Yunhoe Ku Eunseo Kim Seungryul Baek
Reenacting facial images is an important task that can find numerous
applications. We proposed IFaceUV, a fully differentiable pipeline that
properly combines 2D and 3D information to conduct the facial reenactment task.
The three-dimensional morphable face models (3DMMs) and corresponding UV maps
are utilized to intuitively control facial motions and textures, respectively.
Two-dimensional techniques based on 2D image warping is further required to
compensate for missing components of the 3DMMs such as backgrounds, ear, hair
and etc. In our pipeline, we first extract 3DMM parameters and corresponding UV
maps from source and target images. Then, initial UV maps are refined by the UV
map refinement network and it is rendered to the image with the motion
manipulated 3DMM parameters. In parallel, we warp the source image according to
the 2D flow field obtained from the 2D warping network. Rendered and warped
images are combined in the final editing network to generate the final
reenactment image. Additionally, we tested our model for the audio-driven
facial reenactment task. Extensive qualitative and quantitative experiments
illustrate the remarkable performance of our method compared to other
state-of-the-art methods.