Choreographing the Digital Canvas: A Machine Learning Approach to
  Artistic Performance
Siyuan Peng Kate Ladenheim Snehesh Shrestha Cornelia Fermüller
This paper introduces the concept of a design tool for artistic performances
based on attribute descriptions. To do so, we used a specific performance of
falling actions. The platform integrates a novel machine-learning (ML) model
with an interactive interface to generate and visualize artistic movements. Our
approach's core is a cyclic Attribute-Conditioned Variational Autoencoder
(AC-VAE) model developed to address the challenge of capturing and generating
realistic 3D human body motions from motion capture (MoCap) data. We created a
unique dataset focused on the dynamics of falling movements, characterized by a
new ontology that divides motion into three distinct phases: Impact, Glitch,
and Fall. The ML model's innovation lies in its ability to learn these phases
separately. It is achieved by applying comprehensive data augmentation
techniques and an initial pose loss function to generate natural and plausible
motion. Our web-based interface provides an intuitive platform for artists to
engage with this technology, offering fine-grained control over motion
attributes and interactive visualization tools, including a 360-degree view and
a dynamic timeline for playback manipulation. Our research paves the way for a
future where technology amplifies the creative potential of human expression,
making sophisticated motion generation accessible to a wider artistic
community.