Mobile phone projectors have become a convenience and novelty for users. They're great for a quick brainstorm or to show your friends what new game you're playing. However, interacting with the projected images hasn't been a possibility before now.

How did the Idea Hatch?

ShadowPuppetsAT&T Labs summer intern Lisa Cowan was working on her dissertation about different ways people use Pico projectors on their mobile phones. In considering a summer project, researcher Kevin Li and Cowan realized that the real problem with Pico projector mobile phones was that only one person could provide input to the system through the phone's screen. The team set out to find a way to provide input through the projection. Typical arbitrary gestures wouldn't work because people see different things depending on where they're standing and their perspective. While considering a solution the team realized they were inadvertently casting shadows on the display. Shadows are perceived the same by everyone — no matter where they are standing. They decided to take advantage of that property of projected images and allow users to provide input to the system using their shadows.

Their research set about to study potential shadow interactions and create a prototype for this technology. First, they studied what hand shadow users will expect causes various effects. The second study looked at how users perceive hand shadows, examining what effects they think different hand shadows will cause. Finally, a functional prototype was designed based on qualitative results from this study.

About the Project

ShadowPuppets is a system that supports collocated interaction with mobile projector phones, allowing users to cast hand shadows as to provide input to mobile projector phones. Current projector phones only support input via the handset's screen, not through the projection. With ShadowPuppets the user and bystanders can cast shadows on the projection and manipulate the screen — without needing additional equipment. With this combination, not only can you share a map with friends, but anyone can help show the way using their shadows.

The Future

ShadowPuppets represents a new way of interacting with projector phones. So far research has only looked at a gesture set for providing input into a single phone. One area for future research is to look at how the interaction changes when multiple collocated users each have projector phones.

In addition, collaboration and productivity will increase as small screens that were previously meant for one set of inputs make room for an entire group's interactions. Potential applications include:

  • Interactive Presentations. Participation during presentations can be increased when others in the room can manipulate the screen.
  • Brainstorms. Impromptu brainstorms either with clients or a group of friends can easily become a collaborative process when everyone is involved.
  • The Classroom. Whether teaching or interacting with students, the ability to gesture using your shadows would be an advantage in the classroom.
  • Gaming. By allowing others to play with their shadows, the previously small screens, now offer a great advantage for multi-player gaming.

About the Researcher

Kevin LiKevin Li is a Researcher at AT&T Labs, working on new and innovative ways of interacting with mobile devices. Li is also an Adjunct Professor in the Department of Computer Science at Columbia University. His research encompasses mobile computing, novel interaction techniques and Ubiquitous Computing and Human Computer Interaction. Li received his PhD in Computer Engineering from University of California, San Diego and his BS in Electrical Engineering and Computer Science from University of California, Berkeley.

Innovation Space Blog