Gmail Motion April Fools' gag inevitably turned into reality using Kinect | Hacker News Reader

Gmail Motion April Fools' gag inevitably turned into reality using Kinect

engadget.com

147 pointsstevederico30 comments

The sad thing is, I saw Google Motion appear in my Gmail toolstrip, but I didn't even think about April fools, I just thought 'oh, great, another half-baked wannabe social media experiment masquerading as a feature from Google (the email and search company)' and didn't bother clicking it.

Watching the "joke" made me realize something - there's a chance that, in the future, a lot of communication with computers will be done using sign language.

After all, most people (rightly) assume that pretty soon, voice recognition will work well enough that you'll be able to dictate everything to your computer, making keyboards partially obsolete (especially for the majority of users).

But everyone also correctly thinks that there is one big problem with voice-driven computers - they won't work well with many people in the room at the same time. Imagine a cubicle environment in which everyone needs to talk to their computers.

Motion-detection, coupled with sign language, is a pretty logical next step.

I cannot imagine why would I choose to use voice recognition even if I am alone with computer.

Yep... some sort of neural control is much more likely.

In Netflix, today: "Xbox, pause." (handle phone call). "Xbox, play."

For simple commands, it's pretty neat.

"Xbox, order pizza and find a movie that doesn't suck" is probably a few years off :-)

If only there was a small keyboard with a limited set of buttons that could be pressed to command consumer electronics devices. I would call it ... a "remote control".

On a random remote I just picked up, I see buttons labeled: Menu, Top Menu, Configure, Category, Setup, Option, two Enter buttons, two Mute buttons, several banks of numeric entry 0-9, two channel changers, two volume changers, three different power buttons, two mutes, and four ways to change inputs.

I would much rather do this:

"Computer, show me some God damned teevee."

"Yes, boss."

I think it's likely we'll get decent voice-and-intent recognition before the remote control makers are brought, well, under control. Right now your average remote is like flying a fapping 747.

[I'm aware of the Apple remote. It seems cool, but I haven't used one.]

Remote controls and home automation are a space that many a geek has tried to conquer with good code and proper UI design principles. Give it some time; it'll happen. Voice control will have its place, but I'm mostly convinced that, for people with one good hand, well-designed physical interfaces will always be better for most things.

Better yet, you use a home automation system, and when the phone rings, the Xbox pauses automatically. If phones had more flexibility in their Bluetooth stacks, and/or app developers for consoles added network APIs, an app on your cellphone could simply send a packet to the Xbox to tell it that a phone call has arrived and it should pause, mute, etc. appropriately. Heck, why not just take the phone call through the Kinect's echo-cancelled microphones via Bluetooth?

"and when the phone rings, the Xbox pauses automatically."

That sounds terrible.

I don't know how common my thoughts are on this compared to other people, but if I were to voice-interact with my technology, I'd prefer that it was like interacting with Hal or Jeeves. Memorizing a list of commands and syntax to say out loud would be simple enough, but the thought of such strict syntax turns me off. Spoken language has always been fluid, fun, and spontaneous in my experience, and I hate the thought of it becoming like concrete.

I realize others think differently, but I wonder whether I'm truly alone or not on this.

Clickable link to the project website: http://projects.ict.usc.edu/mxr/faast/

This makes me want to get a kinect to play around with.

I knew Gmail Motion was an April Fools gag --- that's what the link sending me there said, but...

It's actually a good idea --- for certain users. In particular, there are many people who use sign language to communicate. For those who sign, what could be more natural than controlling Gmail via sign? And why stop at Gmail?

Further reading: http://cad.ca/en/issues/telecommunications.asp

I'm taking a multimodal user interfaces course, and for our final project my partner and I discussed a gesture based interface for GMail on March 31. 10 hours later I saw GMail Motion. Granted, we weren't thinking quite the gesture set Google had in mind, but similar idea.

Real time hand sign recognition had a published solution in 2005 ( http://portal.acm.org/citation.cfm?id=1107692 "an automatic Australian sign language (Auslan) recognition system, which tracks multiple target objects (the face and hands) throughout an image sequence and extracts features for the recognition of sign phrases. ") after about 10 to 15 years of prior work (eg: Hand movement classification using an adaptive fuzzy expert system -https://www.socrates.uwa.edu.au/Pub/PubDetailView.aspx?Publi... ) - 1996).

It's interesting that this work flowed out of research and development in sheep shearing robotics ( http://school.mech.uwa.edu.au/~jamest/shearmagic/autoshear.h... ) from the late 1970s / early 1980s.

All the tools needed to build this are now built into OpenCV (and likely other similar libs, but OpenCV is what I know) so it's really a matter of a day or two to implement this now. I personally find this amazing. The major difference, I think, between today and 2005 is that today this is doable on your average netbook as someone's weekend project.

Is there a recommended OpenCV book or website for the experienced hacker whose career focus has not been computer vision who wants to assemble some of those tools into a larger product?

Why yes there is. The OpenCV book, from Oreilly. It's excellent for getting to know the library more deeply.

I remember how xenophobic and arrogant the Google organization was in the early days when it was a hot company. Well they are definitely still arrogant, wasting time on a video like this when the technology to actually do it is not really that difficult.

Has anyone combined the Kinect with iOS yet? That plus the HDMI out from the new iPad to a big screen tv would equal epicness on an unprecedented scale. In fact, I'd actually pay more for that than I did for the iPad itself. Get to work, Apple.

I would bet any amount of money that there are iPad prototypes in R&D that use 2 or 4 front-facing cameras.

I would bet $10 that there aren't.

I would bet $100 neither of you have any way to know. :)

Kinect doesn't work using 2 or 4 front-facing camera and analysing the differences; it projects a IR pattern of dots and analyses its distortion. They say it's more robust and requires much less CPU power.

All joking aside, for some users with disabilities this sort of gesture based interface could be really useful.

My prediction is that gesture interfaces are going to be used for some basic things, but that most interactions with computers will remain fairly conventional. If you do a time and motion study, the way people currently interact via keyboards, mice and touch screens is pretty efficient. Even with augmented reality, I expect that users will still be tapping on virtual keyboards.

Other than someone with serious finger/hand issues, I'm having a hard time imagining a disability that would prohibit someone from using a mouse/keyboard as input but would leave them able to use full-body gestures.

It is not necessarily a bad idea, and it would not have to be restrained to people with disabilities. The way Google presented the idea was fun but it did make some good sense: it would make us, couch potatoes "have to" move.

That was really fast. I feel pretty incompetent (which in all likelihood, may not be far from true, despite what my client/managers say).

Good job!

Just as @comex anticipated....

@comex: "I give it 12 hours until someone makes Gmail Motion for real using the Kinect."

I wish ThinkGeek would make the Apple Store Playmobil set :(

Reply on news.ycombinator.com