LeGO: Leveraging a Surface Deformation Network for Animatable Stylized
  Face Generation with One Example
Soyeon Yoon Kwan Yun Kwanggyoon Seo Sihun Cha Jung Eun Yoo Junyong Noh
Recent advances in 3D face stylization have made significant strides in few
to zero-shot settings. However, the degree of stylization achieved by existing
methods is often not sufficient for practical applications because they are
mostly based on statistical 3D Morphable Models (3DMM) with limited variations.
To this end, we propose a method that can produce a highly stylized 3D face
model with desired topology. Our methods train a surface deformation network
with 3DMM and translate its domain to the target style using a paired exemplar.
The network achieves stylization of the 3D face mesh by mimicking the style of
the target using a differentiable renderer and directional CLIP losses.
Additionally, during the inference process, we utilize a Mesh Agnostic Encoder
(MAGE) that takes deformation target, a mesh of diverse topologies as input to
the stylization process and encodes its shape into our latent space. The
resulting stylized face model can be animated by commonly used 3DMM blend
shapes. A set of quantitative and qualitative evaluations demonstrate that our
method can produce highly stylized face meshes according to a given style and
output them in a desired topology. We also demonstrate example applications of
our method including image-based stylized avatar generation, linear
interpolation of geometric styles, and facial animation of stylized avatars.