DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation
Qiaosong Qi Le Zhuo Aixi Zhang Yue Liao Fei Fang Si Liu Shuicheng Yan
When hearing music, it is natural for people to dance to its rhythm.
Automatic dance generation, however, is a challenging task due to the physical
constraints of human motion and rhythmic alignment with target music.
Conventional autoregressive methods introduce compounding errors during
sampling and struggle to capture the long-term structure of dance sequences. To
address these limitations, we present a novel cascaded motion diffusion model,
DiffDance, designed for high-resolution, long-form dance generation. This model
comprises a music-to-dance diffusion model and a sequence super-resolution
diffusion model. To bridge the gap between music and motion for conditional
generation, DiffDance employs a pretrained audio representation learning model
to extract music embeddings and further align its embedding space to motion via
contrastive loss. During training our cascaded diffusion model, we also
incorporate multiple geometric losses to constrain the model outputs to be
physically plausible and add a dynamic loss weight that adaptively changes over
diffusion timesteps to facilitate sample diversity. Through comprehensive
experiments performed on the benchmark dataset AIST++, we demonstrate that
DiffDance is capable of generating realistic dance sequences that align
effectively with the input music. These results are comparable to those
achieved by state-of-the-art autoregressive methods.