Scalable Motion Style Transfer with Constrained Diffusion Generation
Wenjie Yin Yi Yu Hang Yin Danica Kragic Mårten Björkman
Current training of motion style transfer systems relies on consistency
losses across style domains to preserve contents, hindering its scalable
application to a large number of domains and private data. Recent image
transfer works show the potential of independent training on each domain by
leveraging implicit bridging between diffusion models, with the content
preservation, however, limited to simple data patterns. We address this by
imposing biased sampling in backward diffusion while maintaining the domain
independence in the training stage. We construct the bias from the source
domain keyframes and apply them as the gradient of content constraints,
yielding a framework with keyframe manifold constraint gradients (KMCGs). Our
validation demonstrates the success of training separate models to transfer
between as many as ten dance motion styles. Comprehensive experiments find a
significant improvement in preserving motion contents in comparison to baseline
and ablative diffusion-based style transfer models. In addition, we perform a
human study for a subjective assessment of the quality of generated dance
motions. The results validate the competitiveness of KMCGs.