Tele-Kinesthetic Interaction: Using Hand Muscles to Interact With A Tangible 3D Object

In this tele-kinesthetic interaction environment, users remotely interact with a physical object by moving and straining their hands. The system approximates the hand motions by synthesizing EMG signals.

Enhanced Life
People naturally learn how to move and strain their muscles to properly manipulate objects. This project demonstrates what might happen if we successfully couple the kinesthetic sense with tangible displays as future human-computer-interaction technologies. It combines MyKinSynthesizer and SPIDARmotion. While each of these technologies is unique, their integration is exceptionally rare. No other approach integrates motion and force-sensing systems with a 3D tangible-motion display.

To demonstrate a tele-kinesthetic interaction environment.

The core technical innovations are in these areas:

1. MyKinSynthesizer, a technology that approximates a user's hand motion and force through a musculo-skeletal model synthesized from EMG signals.

2. SPIDARmotion, an innovative device that displays the motion of a 3D object within a cubic space.

3. Integration of the two technologies to realize a tele-kinesthetic interaction environment.

This project implies that people can tele-control 3D objects more properly using their body movement and force. It is a starting point for research in tele-kinesthetic interaction, in which people naturally use their kinesthetic skills to interact with physical objects. As this research develops, it will reveal applications in media art, entertainment, education, and scientific simulation.

Yasuharu Koike
Tokyo Institute of Technology
Koike (at)

Kyoungsik Kim
Yasuharu Koike
Ryo Kojima
Duk Shin
Katsuhiro Tsukamoto
Masato Watanabe
Asa Yasukochi
Tokyo Institute of Technology

Kumiyo Nakakoji
Yasuhiro Yamamoto
University of Tokyo