the sound catcher


This project was developed in collaboration with research student Max Worgan. He used a variety of computer vision techniques to analyse the shadow puppet performance and generate an algorithmic musical accompaniment. Surveillance techniques for detecting, recognising and tracking objects were re-appropriated to map the visual information of the performance into a musical domain. Using a custom built computer vision system that identified the movement of the puppets, the structure of the piece and the overall activity, he produced real-time, generative music that reflected the puppetry.

The generative music score Max developed was inspired by gamelan, a form of musical ensemble from Indonesia. Chosen because of its historical association with shadow puppetry, its semi deterministic form and structure lends itself readily to algorithmic interpretation. Striking a balance between creating a pleasing piece of music and one that represented that action on stage was the main aesthetic challenge of the project.
