His device integrates information with everyday objects. The Sixth-Sense device bridges the gap between digital devices that we now use and how we interact with the physical world. It integrates digital information with reality by making the objects in the physical world our computer.
The Sixth-Sense device is composed of a camera, a pocket projector and a mirror. The projector and camera are linked to a mobile computing device in the pocket of the user. Physical objects are used as interfaces that the projector projects onto. The camera tracks user hand gestures. Software processes the video from the camera and tracks the sensors or markers on the fingertips of the user. The movement of the user's fingertips are interpreted as gestures. The hardware necessary to build the device is only about $350. The software is to be released and made available as Open Source. There are many applications demonstrated by Mistry: Sixth Sense