MambaTalk: Efficient Holistic Gesture Synthesis with Selective State
  Space Models
Zunnan Xu Yukang Lin Haonan Han Sicheng Yang Ronghui Li Yachao Zhang Xiu Li
Gesture synthesis is a vital realm of human-computer interaction, with
wide-ranging applications across various fields like film, robotics, and
virtual reality. Recent advancements have utilized the diffusion model and
attention mechanisms to improve gesture synthesis. However, due to the high
computational complexity of these techniques, generating long and diverse
sequences with low latency remains a challenge. We explore the potential of
state space models (SSMs) to address the challenge, implementing a two-stage
modeling strategy with discrete motion priors to enhance the quality of
gestures. Leveraging the foundational Mamba block, we introduce MambaTalk,
enhancing gesture diversity and rhythm through multimodal integration.
Extensive experiments demonstrate that our method matches or exceeds the
performance of state-of-the-art models.