Leap Motion Has Launched
blog.leapmotion.com
blog.leapmotion.com
1. There are way too many gestures and motions available. The SDK needs to provide a solid framework for fewer gestures/motions and the company needs to make sure that consumers readily understand them. Unless there is a whole lot of standardization of the platform it will become hard for developers/other companies to produce a consistent and coherent experience.
2. I do not like the strategy of lets-throw-it-in-the-wild-and-see-what-other-people-come-up-with; the tech has loads of cool factor that you will get people to make interesting stuff with it. However, will that translate into useable and effective computing software? From my point of view, I have not really seen a single application on Leap which does something 'better' than a touch based or mouse-based interface can do. Better meaning more intuitively and/or more productively
3. I think one of the biggest challenges for any gesture controlled device is the shoulder/forearm/wrist/palm pain that is inherently going to be the result of prolonged gesture-driven interface use. Maybe applications have to use gestures sparingly?
On a broader note, I still think humans have an inherent desire to touch/feel interface elements. Being able to directly touch a screen and manipulate it means I can easily manipulate visual behavior through an actual/physical connection. Interfaces which disconnect this physical medium seem more alien to me at least. Maybe I am less evolved as a neanderthal but I don't think we are ready for a gesture driven interface.
It should be pretty doable to hook it up to a neural network and train it to recognize some of your own gestures.
I was already looking forward to training up a drumroll gesture for 'deploy to production.'
[1] http://ftm.ircam.fr/index.php/Gesture_Follower [2] http://cycling74.com/products/max/ [3] https://www.youtube.com/watch?v=2dSU8MeJlz4
For example, there are some startups that use Kinect to let surgeon interact with a screen showing patient data during an operation. Some researchers have also proposed the use of gesture recognition in the context of a worker having to follow precise instructions for some complex task. Leap motion + Google Glass could be an interesting combo for interactive training.
I'm sure there are other areas. But I agree with your comment and I fail to see the mass market use case. But I'd be happy to be proven wrong :-)
I make mobile applications for kids and I made a prototype of one of our games for some kids to work with, but even after a tutorial it was hard for kids to pick it up.
I think there is something very human about being able to physically interact with things. And it is extremely hard to break that barrier.
"Specialized Application" means an Application which is: (i) sold, licensed, leased, or otherwise disposed of for a list price of more than US$500 or local equivalent, or more than US$240 per year or local equivalent if on a subscription, lease or similar basis; (ii) sold, licensed, leased or otherwise disposed of as part of, or for use with, another application, system, machine or device (other than a personal computer), having a list price of more than US$500 or local equivalent, or more than US$240 per year or local equivalent if on a subscription, lease or similar basis; (iii) designed for use, or that is primarily used, with or for control, whether direct or indirect, of industrial, commercial, military or medical equipment.
I'll pass.
When it's higher than your elbow, your shoulder must be engaged and power all your movements. There are also many apps that map directly from x/y/z space. These can be hard to navigate. All your joints move in arcs, not straight lines.
You get to use the Leap however you want, so figure out what's comfortable for yourself.
Leap is quite sensitive. You could look for tiny flicks of the finger, for instance to create a virtual DataHand. These kinds of interfaces should be much gentler on the body than apps which require large sweeping motions, punches, etc.
Leap has a long way to go in defining a use case for the technology; something which it does better than existing interfaces. Otherwise it will be a really cool niche product used by tech enthusiasts, not something a heavily VC-backed form is targetting.
Also, there is a reason DataHands never really took off and it is the same reason Leap might not. Changing human behavior is ridiculously hard, and pouring money in a dev ecosystem (how Leap is doing it) is not the way to do it.
I'm planning on creating my own laptop startup and eventually try to integrate Leap(or something similar). In many cases it may not apply to general usage, but there are many use cases where it would be preferred.
1) Cases where user interacts with a GUI where user is visually engaged, imagine I am on Youtube, personally because stupid instability of HTML5 and Flash on Linux, I hate using it, especially when I am full screening. That's something that can be replaced with a gesture. So try doing a pinch gesture(basically zoom out) and the browser can interrupt that as put video in full screen.
2) Rather than using your whole arm what about just one finger. Lets say you want to scroll down and both hands are on the keyboard, rather than moving one hand back to the mouse you can just flick one finger. This gesture could scroll pages, it could flip through tabs and you wouldn't even have to take your hands of the keyboard.
3) Now there are a lot of edge cases I would say where using a keyboard/trackpad is uncomfortable. When I am lying down using a trackpad is very awkward and strains my wrist. This is certainly the case where I would want to flick some stuff around, maybe even talk to the computer.
4) My parents connect their laptop via HDMI cable to the TV and that's all they use the laptop for, basically to watch youtube and Netflix(if Leap motion had the range), it's something I would want to control almost as an imaginary but intuitive remote.
And this is kind of just the start. I think with the increasing number of input devices available interfaces are going to have to be clever to hide that complexity from the user and make things intuitive.
But in terms of changing behaviour, they are pretty much a qwerty keyboard. In use, they're not ever-so different at all. I would be very surprised if that was a significant contributing reason to why they 'never really took off'.
I work at Oblong Industries. We're the folks who did the interfaces in Minority Report. We've been building gestural and spatial stuff for a long time.
Regarding (1) - standardization is really important. I'm old enough to remember the early days of mouse-driven interfaces. It took a long time for the standard window manipulation semantics, scrollbars, buttons, and drop-down menus to develop. The original Macintosh team did a lot of quite elegant heavy lifting in that regard. It's very much worth reading the stories on Andy Hertzfeld's site and in the related book. http://www.folklore.org/index.py ; http://www.amazon.com/Revolution-The-Valley-Insanely-Great/d...
If you'd like to experiment with a toolkit that provides a standard framework for building gestural, spatial, multi-screen, multi-device applications, please feel free check out our Greenhouse SDK. Greenhouse supports the Leap Motion controller, the Kinect, and a bunch of other forms of input. http://greenhouse.oblong.com/
Regarding (3) - we've got a lot of experience now with ergonomics of spatial interfaces. (We've been selling glove-based gestural systems to early adopter bigco customers since 2005.) It turns out that accuracy of the underlying sensing hardware is incredibly important, and then on top of that design of the gesture language is really important, too. Basically, if you have good enough sensing that small motions are precisely tracked, and then if you use that accuracy to track hand movements and poses that are "natural" (glossing over what "natural" means, for the moment), people can comfortably and happily use gestural interfaces all day, every day.
Think about it this way: most people "talk with their hands" all the time. (Some people wave their hands around more than others. But spend some time paying attention to people moving their hands while they talk; it's really interesting to watch.) Our high-end, glove-based systems track finger positions to 0.01 mm at 100hz. The tracking volume is big, so your motions aren't constrained. You can walk around and you can go right up to a screen or stand back from it. We recognize hand poses ("one finger point," "two finger point," etc.) "Talking" to the computer with your hands feels a lot like what you do when you use your hands while making a point in a discussion with another person. You don't get tired and you don't feel any strain or overuse pain.
The consumer-priced sensors like the Leap Motion and the Kinect don't yet provide this combination of tracking precision, tracking volume, and robustness to occlusion. But they're getting closer! And when a $50 sensor (or a few $0.50 sensors) give you the same accuracy and precision we have today with Oblong's optically tracked gloves, our collective expectations about interfaces and user experiences are going to radically change. (Or, more accurately, as happened with the invention of the GUI we use today, our expectations will change slowly over fifteen years, starting roughly now, with some moments of punctuated equilibrium analogous to the release of the Mac in 1984 and Windows 95 in, um, 1995.)
The only arrangement I have had real success with is far away from the screen, i.e. mounted in a guitar.
My kids have been mildly amused by some of the things I've written and by the apps available in the beta, but e.g. with Cut the Rope, a lot of fun swiping on touchscreen iOS devices, they couldn't get it to work waving about in empty space with the Leap Motion. "Dad, this is too hard."
I've only played with it for an hour or so in total, but so far I'm returning it. Maybe it's a training issue (on my part; the device doesn't learn, as far as I can tell), but it just doesn't work very well for me.
The demo apps suggest a severe lack of precision when I'm using it, and complain that the room is too bright. Coincidentally, it's a stormy, overcast day, and my office lamp burned out this morning. This is as dark as it'll ever be in here during the day.
I'll definitely give it some more time (you have 30 days from when it ships to return it), but I'm not that interested in learning how to move my hands in a Leap-specific way, or in permanently darkening my office.
Could it be that it's aimed too much towards the screen? I mean, that's the most obvious remaining light source.
On the bright side (no pun intended), I actually didn't have any lighting issues though, regardless of how dim the room was. Maybe you just have a defective unit? Giving them a call might be worth a shot...
Edited to say: "Man, I use the word 'I' enough in that post."
I did call them today and they couldn't even find the order, even though it was still fine couple weeks ago. (This is an international order btw)
Anyways, the product still looks great and the customer support has been fast to reply so I have no complaints about that either. I was just quite eager to start developing some projects for it.
Additionally, developer feedback on the product is not that positive.
Here, for example, is demo video of a three-space navigation app built on these principles that uses the Kinect: http://vimeo.com/65937620
And a simpler demo using the Leap: http://vimeo.com/66196803
There's no muscle fatigue or stress using these kinds of applications. You get muscle fatigue (gorilla arm, forearm pain, hand pain) when your ergonomic setup -- which includes both hardware and software -- forces your body to conform to motion patterns that are rigidly repetitive, forces you to position yourself in ways that are awkward relative to your joint kinematics, or forces you to over-correct inaccuracy or unpredictability in the interface.
The Leap Motion Controller works with computers running Mac OS X 10.7 or 10.8, or Windows 7 or 8. It requires a minimum Intel Core™ i3 or AMD Phenom™ II processor, 2 GB RAM and a USB 2.0 port.
When I ordered it, back in September 2012, there was no indication that it wouldn't be compatible with my OS. I wonder what there is in 10.7 that they require.
I was able to install and run it on my Ubuntu 12.04 system. It had a dependency issue because I had previously upgraded to lts-quantal - the install did not recognize the OpenGL package because it was (from memory) "libgl1-mesa-glx-lts-quantal" instead of the expected "libgl1-mesa-glx", but I did a --force-depends and it installed and ran fine.
I elaborate a bit more on the subject in an older blog post: http://blog.mikebourgeous.com/2012/10/02/the-ideal-arm-platf...
You don't have to take my word for it. You can search the web for RPi USB or Raspberry PI USB problems, or even do your own tests.
Try running dd if=/usb/storage/some/file of=/dev/null and see just how slow it goes. The fact that the Ethernet interface is connected over USB is also a problem, because it hogs what precious little bandwidth the RPi has. Try running a bandwidth test from the RPi to another computer on your LAN; you probably won't come close to 100mbit/s.
No doubt the RPi is good at a lot of things, but USB performance and network bandwidth are not among those things.
The feedback is tight as long as you can continuously see your input, and motioning lower could help with tiredness.
Might be a good idea not to assume that people know what your product is when you write such a release page...
You sit down at a virtual PC, and control it by waving your hands over a virtual Leap Motion. Which is very meta, but perhaps not very interesting.
My web browser keeps scrolling up and down very rapidly.