CosAvatar: Consistent and Animatable Portrait Video Tuning with Text
  Prompt
Haiyao Xiao Chenglai Zhong Xuan Gao Yudong Guo Juyong Zhang
Recently, text-guided digital portrait editing has attracted more and more
attentions. However, existing methods still struggle to maintain consistency
across time, expression, and view or require specific data prerequisites. To
solve these challenging problems, we propose CosAvatar, a high-quality and
user-friendly framework for portrait tuning. With only monocular video and text
instructions as input, we can produce animatable portraits with both temporal
and 3D consistency. Different from methods that directly edit in the 2D domain,
we employ a dynamic NeRF-based 3D portrait representation to model both the
head and torso. We alternate between editing the video frames' dataset and
updating the underlying 3D portrait until the edited frames reach 3D
consistency. Additionally, we integrate the semantic portrait priors to enhance
the edited results, allowing precise modifications in specified semantic areas.
Extensive results demonstrate that our proposed method can not only accurately
edit portrait styles or local attributes based on text instructions but also
support expressive animation driven by a source video.