MIDGET: Music Conditioned 3D Dance Generation
Jinwu Wang Wei Mao Miaomiao Liu
In this paper, we introduce a MusIc conditioned 3D Dance GEneraTion model,
named MIDGET based on Dance motion Vector Quantised Variational AutoEncoder
(VQ-VAE) model and Motion Generative Pre-Training (GPT) model to generate
vibrant and highquality dances that match the music rhythm. To tackle
challenges in the field, we introduce three new components: 1) a pre-trained
memory codebook based on the Motion VQ-VAE model to store different human pose
codes, 2) employing Motion GPT model to generate pose codes with music and
motion Encoders, 3) a simple framework for music feature extraction. We compare
with existing state-of-the-art models and perform ablation experiments on
AIST++, the largest publicly available music-dance dataset. Experiments
demonstrate that our proposed framework achieves state-of-the-art performance
on motion quality and its alignment with the music.