in2IN: Leveraging individual Information to Generate Human INteractions
Pablo Ruiz Ponce German Barquero Cristina Palmero Sergio Escalera Jose Garcia-Rodriguez
Generating human-human motion interactions conditioned on textual
descriptions is a very useful application in many areas such as robotics,
gaming, animation, and the metaverse. Alongside this utility also comes a great
difficulty in modeling the highly dimensional inter-personal dynamics. In
addition, properly capturing the intra-personal diversity of interactions has a
lot of challenges. Current methods generate interactions with limited diversity
of intra-person dynamics due to the limitations of the available datasets and
conditioning strategies. For this, we introduce in2IN, a novel diffusion model
for human-human motion generation which is conditioned not only on the textual
description of the overall interaction but also on the individual descriptions
of the actions performed by each person involved in the interaction. To train
this model, we use a large language model to extend the InterHuman dataset with
individual descriptions. As a result, in2IN achieves state-of-the-art
performance in the InterHuman dataset. Furthermore, in order to increase the
intra-personal diversity on the existing interaction datasets, we propose
DualMDM, a model composition technique that combines the motions generated with
in2IN and the motions generated by a single-person motion prior pre-trained on
HumanML3D. As a result, DualMDM generates motions with higher individual
diversity and improves control over the intra-person dynamics while maintaining
inter-personal coherence.