Augmented Reality Sixth Sense System

Pranav Mistry at MIT has developed the Sixth-Sense Device which allows the user to take the digital world and essentially paint the physical world or physical objects with information from the digital world.

His device integrates information with everyday objects. The Sixth-Sense device bridges the gap between digital devices that we now use and how we interact with the physical world. It integrates digital information with reality by making the objects in the physical world our computer.

The Sixth-Sense device is composed of a camera, a pocket projector and a mirror. The projector and camera are linked to a mobile computing device in the pocket of the user. Physical objects are used as interfaces that the projector projects onto. The camera tracks user hand gestures. Software processes the video from the camera and tracks the sensors or markers on the fingertips of the user. The movement of the user's fingertips are interpreted as gestures. The hardware necessary to build the device is only about $350. The software is to be released and made available as Open Source. There are many applications demonstrated by Mistry:

The Project

The project is to develop the prototype device and, using the open source software, develop ideas for how to extend the capability to other applications. Of particular interest would be ways to integrate the Sixth-Sense device with business intelligence applications like business process or decision-making models or supply chain models.