Dance Your Latents: Consistent Dance Generation through Spatial-temporal
  Subspace Attention Guided by Motion Flow
Haipeng Fang Zhihao Sun Ziyao Huang Fan Tang Juan Cao Sheng Tang
The advancement of generative AI has extended to the realm of Human Dance
Generation, demonstrating superior generative capacities. However, current
methods still exhibit deficiencies in achieving spatiotemporal consistency,
resulting in artifacts like ghosting, flickering, and incoherent motions. In
this paper, we present Dance-Your-Latents, a framework that makes latents dance
coherently following motion flow to generate consistent dance videos. Firstly,
considering that each constituent element moves within a confined space, we
introduce spatial-temporal subspace-attention blocks that decompose the global
space into a combination of regular subspaces and efficiently model the
spatiotemporal consistency within these subspaces. This module enables each
patch pay attention to adjacent areas, mitigating the excessive dispersion of
long-range attention. Furthermore, observing that body part's movement is
guided by pose control, we design motion flow guided subspace align & restore.
This method enables the attention to be computed on the irregular subspace
along the motion flow. Experimental results in TikTok dataset demonstrate that
our approach significantly enhances spatiotemporal consistency of the generated
videos.