Kinect + Javascript: Control webpages with gestures (MIT Media Lab)

vimeo.com

19 points·aaronzinman··5 comments

We have the Microsoft Kinect talking to Javascript with some Computer Vision recognizing basic gestures.

You can directly read out the Kinect data, or use our high-level gesture events. On any web page!

See the demo video at http://vimeo.com/17180651

The source is available at http://github.com/doug/depthjs

Nice, although I'm not sure that this method of navigation will catch on. You can also do this with stereo vision, although at a higher computational cost. For example, here's an overhead view of a point cloud from a stereo camera (Minoru webcam):

http://www.youtube.com/watch?v=kQPIm2Kw5kc

One possible useful gesture based form of navigation might be to use a hand wave in order to change viewports.

Thanks! Our current gesture set is more focused on showing what's possible so far and inspiring some new ideas. Navigating the web is only one application of the framework we built - we envision all sorts of applications that run in the browser, from games to specific utilities for specific sites. The great part is that now web developers who specialize in Javascript can work with the Kinect without having to learn any special languages or code. We're hoping this allows a new set of interactions beyond what we first developed.

So can this be done with just a regular camera without depth perception?

Right now its Kinect only--but it's called DepthJS versus KinectJS because we'll eventually infer depth using web cams.

a) this is wicked cool b) in person, these guys are rockstars, esp aaron

Reply on news.ycombinator.com