AvatarStudio: High-fidelity and Animatable 3D Avatar Creation from Text
Jianfeng Zhang Xuanmeng Zhang Huichao Zhang Jun Hao Liew Chenxu Zhang Yi Yang Jiashi Feng
We study the problem of creating high-fidelity and animatable 3D avatars from
only textual descriptions. Existing text-to-avatar methods are either limited
to static avatars which cannot be animated or struggle to generate animatable
avatars with promising quality and precise pose control. To address these
limitations, we propose AvatarStudio, a coarse-to-fine generative model that
generates explicit textured 3D meshes for animatable human avatars.
Specifically, AvatarStudio begins with a low-resolution NeRF-based
representation for coarse generation, followed by incorporating SMPL-guided
articulation into the explicit mesh representation to support avatar animation
and high resolution rendering. To ensure view consistency and pose
controllability of the resulting avatars, we introduce a 2D diffusion model
conditioned on DensePose for Score Distillation Sampling supervision. By
effectively leveraging the synergy between the articulated mesh representation
and the DensePose-conditional diffusion model, AvatarStudio can create
high-quality avatars from text that are ready for animation, significantly
outperforming previous methods. Moreover, it is competent for many
applications, e.g., multimodal avatar animations and style-guided avatar
creation. For more results, please refer to our project page:
http://jeff95.me/projects/avatarstudio.html