MagicAvatar: Multimodal Avatar Generation and Animation
Jianfeng Zhang Hanshu Yan Zhongcong Xu Jiashi Feng Jun Hao Liew
This report presents MagicAvatar, a framework for multimodal video generation
and animation of human avatars. Unlike most existing methods that generate
avatar-centric videos directly from multimodal inputs (e.g., text prompts),
MagicAvatar explicitly disentangles avatar video generation into two stages:
(1) multimodal-to-motion and (2) motion-to-video generation. The first stage
translates the multimodal inputs into motion/ control signals (e.g., human
pose, depth, DensePose); while the second stage generates avatar-centric video
guided by these motion signals. Additionally, MagicAvatar supports avatar
animation by simply providing a few images of the target person. This
capability enables the animation of the provided human identity according to
the specific motion derived from the first stage. We demonstrate the
flexibility of MagicAvatar through various applications, including text-guided
and video-guided avatar generation, as well as multimodal avatar animation.