Minority Report Interface Is Real, Hitting Mainstream Soon

Oh, I don't know. When I'm using my computer I often think "Man, I wish I were standing up and making huge arm movements right now so that my arms would get tired within twenty minutes"

I'm tempted to say "At least it would be good for games", but frankly the Wii has failed to revolutionize gaming as much as folks thought it would -- after waving your arms around for a bit the novelty wears off and you go back to wanting to mash buttons.

It actually kinda made sense as a system meant to be used quickly, but only for brief periods of time, by completely able-bodied people.

For any extended period time, or for anyone with even minor physical disabilities, it's wretched.

I do think G-Speak has potential applications in dealing specifically with complex three-dimensional data. It's not going to replace what we use every day on the desktop, because most of what we do on the desktop works in two dimensions, but for actually manipulating visual interfaces that are, of necessity, presented in three dimensions, there's nothing like actually moving in 3D.

If common desktop applications do, at some point, evolve to meaningfully take advantage of 3D visual interfaces, it could trickle down to a system that assumes a more natural posture (try resting your arms on your desk and gesturing above your keyboard: I think that could work). We've had hardware-accelerated 3D on the desktop for a decade, though, and have yet to see anything substantial come of it for traditionally two-dimensional UIs apart from sometimes-helpful transitional animations. Things could change, but I think applying gestures and multi-touch to 2D (a la 10/GUI, naturally) shows the most promising way forward.

The LuminAR project (also at MIT Media Lab) [1] is much more down to earth. It projects onto a 2d surface and then you interact with the surface via gestures with your bare hands.

[1] http://news.ycombinator.com/item?id=1820748

I especially like how moving a video from one monitor to another requires moving a bulky plastic (glass?) plate from one terminal to another, a plate sufficiently large to hold an entire computer, touchscreen, and information storage for a hundred videos.

You don't necessarily have to wave your arms around. I don't know how the g-speak works, but if it uses the orientation of the hand to do raycasting (as it should), you can leave your hand comfortable by your side and not be bothered by having to raise your arms at shoulder level.

The real pain comes when you try to apply 3D interfaces like g-speak to problems that are not inherently 3D. You get awkward interaction and a lot of wasted time. Real applications of 3DUIs include medical operation planning, virtual reality, scientific data visualization and architecture, just to name a few. Also, in most cases, gestural interfaces are not the most efficient way to do it.

Actually there are more ways to use the system than what you may traditionally think of when using a computer.

For example, Boeing has patents related to controlling a swarm of UAV's via a g-speak based system. There are more applications where that came from but this was the only example that is public.

Its collaborative features are a huge draw. Would you ever use photoshop via gestures, probably not, but you would use it for command and control systems with more than one concurrent operator.

I recommend that you go to oblongs website and watch some of the demo videos. A computers interface can be anything from manipulating a physical object on a table to "wearing funky gloves and waving your arms around". Its not limited to just a keyboard, mouse or touch pad.

So, i had the opportunity to use gspeak for the better part of a year at risd and what's probably not explicit from looking at demo videos is that you actually don't need to move around that much. The system is remarkably good at tracking motion and (more importantly) projecting your finger/gestures onto a much larger surface.

Stop and think about that last point. If you're five or ten feet from a wall in your office and say you place a vector that runs perpendicular to your palm. It takes a surprisingly small amount of hand movement to get that vector to hit any portion of the wall.

By the time i had gotten used to the system, i could use it usually by performing completely minimal gestures so much so that people didn't even realize i was engaging it explicitly. This is similar to how you can see people playing wii tennis on their couch without standing up. [1]

If you read that sculley interview[2] you hear him talk about commoditizing the performance to create consumer products that react the way you expect them to. For me, gspeak convinced me that this was basically the case with a gestural interface, the hard part was figuring out something to do with it. I suspect this is a hard issue for many people because the majority of the complaints i see are basically "so what, i can't see myself using it" and fair enough, answering that question is really hard. But the underlying problem of how to provide that interface isn't your problem anymore, just like you don't have to figure out how to track touches or mouse movement anymore, you just build on top of that as a given. that seems like a net win to me.

A large part of gspeak for end users and designers ends up being the development of a descriptive and non-conflicting gestural language for the task at hand. The folks at usc have a good example of what i mean [3].

Say what you will about wild gesticulating or how lame it seems, the system (as any other) is only as cool as what you do with it.

[1]: http://www.penny-arcade.com/comic/2006/11/13/ [2]: http://www.cultofmac.com/john-sculley-on-steve-jobs-the-full... [3]: http://www.vimeo.com/8083372

You can code for 30 hours straight without much fatigue?

Um, yes, and this might have been the better link to submit: http://singularityhub.com/2010/10/22/new-video-of-mits-minor...