Many Kinect motion controller hacks have teased a Minority Report interface, but all of them have only shown that it could be possible with a bit more work. Until now: not only is it possible, but it's been done. The Robot Locomotion Group and Learning Intelligent Systems teams at MIT have developed a system that uses the Kinect drivers for Linux to detect all ten of your fingers, along with your palms, so that you can use them to interact with a display.

"This is a graphical interface inspired by the movie 'Minority Report,' reads the video's description. "It uses the Kinect sensor from Microsoft, and the recently released libfreenect driver for interfacing with the Kinect in linux. The graphical interface and the hand detection software were written at MIT to interface with the open source robotics package 'ROS', developed by Willow Garage ( The hand detection software showcases the abilities of the Point Cloud Library (PCL), a part of ROS that MIT has been helping to optimize. The hand detection software is able to distinguish hands and fingers in a cloud of more than 60,000 points at 30 frames per second, allowing natural, real time interaction."

Next up we have a prototype of an Air Guitar in action, courtesy of Chris Oshea. The prototype was written in C++ and uses the openFrameworks and openCV for image processing, the ofxKinect addon and the libfreenect driver on Mac, as well as help from the openframeworks and openkinect communities.


Of course, there are still limitations, but Oshea made a point to give a big thank you to Microsoft for bringing the technology to the mass market. That's exactly why the Kinect is so ground-breaking: not only is it cheap, but it's also widely available.