Bring Your Own Character: A Holistic Solution for Automatic Facial
  Animation Generation of Customized Characters
Zechen Bai Peng Chen Xiaolan Peng Lu Liu Hui Chen Mike Zheng Shou Feng Tian
Animating virtual characters has always been a fundamental research problem
in virtual reality (VR). Facial animations play a crucial role as they
effectively convey emotions and attitudes of virtual humans. However, creating
such facial animations can be challenging, as current methods often involve
utilization of expensive motion capture devices or significant investments of
time and effort from human animators in tuning animation parameters. In this
paper, we propose a holistic solution to automatically animate virtual human
faces. In our solution, a deep learning model was first trained to retarget the
facial expression from input face images to virtual human faces by estimating
the blendshape coefficients. This method offers the flexibility of generating
animations with characters of different appearances and blendshape topologies.
Second, a practical toolkit was developed using Unity 3D, making it compatible
with the most popular VR applications. The toolkit accepts both image and video
as input to animate the target virtual human faces and enables users to
manipulate the animation results. Furthermore, inspired by the spirit of
Human-in-the-loop (HITL), we leveraged user feedback to further improve the
performance of the model and toolkit, thereby increasing the customization
properties to suit user preferences. The whole solution, for which we will make
the code public, has the potential to accelerate the generation of facial
animations for use in VR applications.