Hacker News Reader

Comment by robg | Hacker News Reader

As an embodied cognition guy, I disagree here pretty strongly. Gestures are a powerful form of communication - consider deaf signers, scuba divers, even in time sensitive operations like military and sports. Gestures likely have a deeper evolutionary history than spoken language too. They are intuitive. Babies mimic long before they speak or manipulate.

View on HN

That's an interesting perspective that I hadn't thought of before.

The usual approach to a 3D gestural interface is the kind of thing that's shown in the video for Leap—writing in the air, using mimed actions in space to represent manipulation of objects on a 2D screen, et cetera.

Gestures as an abstraction, like sign language or even the everyday hand gestures we use like flipping the bird, the "A-OK" sign, and such make a lot more sense. If you move away from the idea of using hand waving as a stand-in for direct manipulation of objects, and look at gestures as a form of communication, it's a whole different ball game.

Thanks for that. I still look at the demo video for Leap with fear and loathing, but using that same hardware for a communicative gesture system like you suggest is exciting indeed. Now I'm going to be distracted all day thinking about ways to incorporate hand signs into a UI.

Upvoted both of you for civility :-)

The big question for me is whether gestures began as a proto-language. Consider mimicry and the use of tools in social learning. That's animal cognition too.

> Gestures are a powerful form of communication - consider deaf signers, scuba divers, even in time sensitive operations like military and sports.

In all of these examples, gestures are being used to interact with another sentient being. We use gestures to talk to people, not to tools.

I think gestures are great, but I don't want to have a conversation with my computer, I want to use it. I want to feel like I'm a craftsman, and not a manager of the work I create on it. (For that same reason, I'm not enthusiastic about voice recognition either.)

That being said, I probably would be excited to use gestures (and voice) for social software that was intrinsically about interacting with other people. Think multiplayer games or video chat.

Yes for communication with humans, but for interacting with a computer? Gestures are imprecise, physically tiring and non-intuitive for the very precise and specific actions we have to do on a PC every day.

Those are based on static symbols and are not used for precise, realtime control of a stateful 2-D system.

Pardon me for a shameless plug here, but I had jump in as this is an interesting discussion where we have at Flutter have long thought about it. We came up with a simple gesture to play/pause a song (itunes/spotify) by asking thousands of humans what gesture they would use if they did not have mouse/keyboard/voice. Almost everyone came up with the one we picked... even for next set of gesture, we're looking at static human gestures that are micro and intuitive (and doesn't make you fatigued).

When we released the first app, we got countless emails asking for hand-swipe as a next song gesture. We had felt a long ago that even though it seems natural it is completely impractical in certain context (i.e. coffee shop) and doing hand swipe 20 times starts to wear you down. It was important to try this number of times as same gesture can because to go to next slide or next photo or next album.

Hence, we ended-up picking thumbs-right and thumbs-left as metaphor: flutterapp.com/next

We will also hit our 4th Million gesture this month. That's 4 million times someone has either played or paused a song.

Please try our app and send feedback - would love to get your thoughts on this!

This is one of the best thoughtful discussion I've seen on this subject! thanks all for stimulating thoughts. I just thought I had something to share...

Fred Brooks agrees. He broadly outlines a design system using gestures in The Design of Design. (I know that PHK already exceeded the quota for Fred Brooks references in a week; I guess, just get over it.)

Reply on news.ycombinator.com