MyPortrait: Morphable Prior-Guided Personalized Portrait Generation
Bo Ding Zhenfeng Fan Shuang Yang Shihong Xia
Generating realistic talking faces is an interesting and long-standing topic
in the field of computer vision. Although significant progress has been made,
it is still challenging to generate high-quality dynamic faces with
personalized details. This is mainly due to the inability of the general model
to represent personalized details and the generalization problem to unseen
controllable parameters. In this work, we propose Myportrait, a simple,
general, and flexible framework for neural portrait generation. We incorporate
personalized prior in a monocular video and morphable prior in 3D face
morphable space for generating personalized details under novel controllable
parameters. Our proposed framework supports both video-driven and audio-driven
face animation given a monocular video of a single person. Distinguished by
whether the test data is sent to training or not, our method provides a
real-time online version and a high-quality offline version. Comprehensive
experiments in various metrics demonstrate the superior performance of our
method over the state-of-the-art methods. The code will be publicly available.