Emotional Talking Head Generation based on Memory-Sharing and
  Attention-Augmented Networks
Jianrong Wang Yaxin Zhao Li Liu Tianyi Xu Qi Li Sen Li
Given an audio clip and a reference face image, the goal of the talking head
generation is to generate a high-fidelity talking head video. Although some
audio-driven methods of generating talking head videos have made some
achievements in the past, most of them only focused on lip and audio
synchronization and lack the ability to reproduce the facial expressions of the
target person. To this end, we propose a talking head generation model
consisting of a Memory-Sharing Emotion Feature extractor (MSEF) and an
Attention-Augmented Translator based on U-net (AATU). Firstly, MSEF can extract
implicit emotional auxiliary features from audio to estimate more accurate
emotional face landmarks.~Secondly, AATU acts as a translator between the
estimated landmarks and the photo-realistic video frames. Extensive qualitative
and quantitative experiments have shown the superiority of the proposed method
to the previous works. Codes will be made publicly available.