Dance Generation by Sound Symbolic Words
Miki Okamura Naruya Kondo Tatsuki Fushimi Maki Sakamoto Yoichi Ochiai
This study introduces a novel approach to generate dance motions using
onomatopoeia as input, with the aim of enhancing creativity and diversity in
dance generation. Unlike text and music, onomatopoeia conveys rhythm and
meaning through abstract word expressions without constraints on expression and
without need for specialized knowledge. We adapt the AI Choreographer framework
and employ the Sakamoto system, a feature extraction method for onomatopoeia
focusing on phonemes and syllables. Additionally, we present a new dataset of
40 onomatopoeia-dance motion pairs collected through a user survey. Our results
demonstrate that the proposed method enables more intuitive dance generation
and can create dance motions using sound-symbolic words from a variety of
languages, including those without onomatopoeia. This highlights the potential
for diverse dance creation across different languages and cultures, accessible
to a wider audience. Qualitative samples from our model can be found at:
https://sites.google.com/view/onomatopoeia-dance/home/.