10 finger multi-touch UI designed to replace the mouse
10gui.com
10gui.com
It also makes no sense to me to go to the trouble of allowing a user to use all ten fingers, and then restricting application interaction to only two or three fingers. I work in rich, deep applications with tons of controls and buttons, if I'm going to go to the trouble of learning to use ten fingers to navigate a computer, you better allow me to make use of all of them withing the individual programs that I'm using.
Prediction: I suspect that Apple is taking Exposé in exactly this direction.
Doesn't it help to have a documented piece of prior art like the Linux demo in this video when it comes to patent defence and implementation on other platforms?
Of course, R. Clayton Miller may have patented too. Or he could see it in the same vein as the safety match and just record it to give it away.
I also believe that as we engage more and more of the body in a control interface, for it to be natural it is going to need to mimic real-world interaction so that we can rely on our lifetime of experience interacting with physical objects.
The better solution in my opinion would contain:
-2 mirrored screens so that the one under your hands can be glanced at for reference as well.
-A UI approach that allows you to interact with objects and windows in the same ways you would with real objects. (Grabbing the edges moves the window, pushing down on it 'sticks' it in place, etc.)
He has a congenital deformity of the arms, but he can type (and is one of those clacky-IBM keyboard fetishists). With his feet, he can use mice and controllers. But multi-touch systems are beyond his ability - while he can pivot and shift to type remarkably quickly, he can't get more than one hand on something at the same time. Worse, each hand is effectively only one digit for him.
And then, there are the folks who don't even have his degree of dexterity.
I can't help thinking that it might be a good thing for general interface designers to consider the implications of disabilities for users of these designs, instead of shrugging that off onto providers of interfaces specifically for the disabled.
It becomes a problem when the machine requires all of my dexterity just to accomplish a given task. I dislike all Apple products specifically because they require mouse operations to accomplish things where Linux and MS interfaces tend to have keyboard shortcuts.
And there's always alt-f4.
In mac os x, - how to show the desktop with a keyboard shortcut ?
- how to maximize/minimize/restore a window with a keyboard shortcut (on windows you can press Alt+spacebar, and then x/n/r) ?
- how to efficiently copy/move files in finder with a keyboard shortcut ?
- how is better thought the fact that when you are in finder and then you press enter, you are going to rename a file? because in order to open/execute on a file you must press command+O...
And finally, the operation you asked can be performed on windows by pressing ctrl+shift+esc, and if you are in the process tab, spelling the first letters of the procname, then press delete and confirm.
F11
"- how to maximize/minimize/restore a window with a keyboard shortcut (on windows you can press Alt+spacebar, and then x/n/r) ?"
There is no such thing as maximize in OS X. Minimize is command + m. I'm not sure how to restore with the keyboard.
"- how to efficiently copy/move files in finder with a keyboard shortcut ?"
Command + c is copy and command + v is paste. Command + x is cut, but there are rules governing when one can cut that I don't understand.
You should be careful of "windows bias" (or whatever, I just made up the term). Just because we do something one way in windows, does not mean its better, just what we are accustomed to.
-- Selecting an arbitrary menu item Arrowing through a menu is key for me. I'm writing from memory but last I recall working on a mac, the menus were strictly mouse only.
-- What's the equivalent of the start button, giving a list of available I can select with the keyboard? I remember not being able to navigate the launch bar with the keyboard, only the mouse. And at that time, I Googled for a while for help on this.
alt-tab == ⌘-tab
Restore == Ctl-F3, arrow key to window in Dock, return
Arbitrary menu item, depends:
Application Menu (left-handside text menus) == Ctl-F2, type enough unique letters to identify the menu (or use arrow keys), return, type enough unique letters to identify the item... repeat till you get to the item you want. If this annoys you, you can implement your own keyboard shortcuts in any native application.
Status Menu (right-handside icon menus) == Ctl-F8, arrows, return.
Equivalent of the start button? Not quite sure what you mean - activate Spotlight ⌘-[Space], type app name perhaps?
I assume you mean Dock when you say launch which is activated by Ctl-F3.
The only one I don't know how to do is fine-grain resizing of a window. If you use OS X though, you'll realise the UI paradigm isn't focused on fine-grained window control though.
Plus I actually suspect that a many application don't implement the keys you've mentioned since I was definitely at-sea in my efforts to use any keys.
But thanks, I'll have more ammunition the next time I'm stuck Osx-land.
Don't take this the wrong way, but Windows skills don't translate to other platforms. OS X will feel different, weird and limited if you're not familiar with it as it doesn't do things the way you're used to.
To be honest, I rarely use most of these shortcut keys because I define my own.
A basic example is that it's two keys to start a VPN session on my machine as that's something I do frequently.
A more complicated one is as at the moment I'm doing work on a site that includes screenshots of itself, I have a shortcut key defined that takes a series of screen grabs from different pages in Safari at the appropriate size, converts them to PNG and sticks them in the appropriate folder. Huge time saver.
In short, it seems to me OS X can do anything with a keyboard shortcut that you'd like it to.
As this can all be done with one hand, does XP therefore have better thought-out keyboard shortcuts than OSX? ;)
Keep expectations high and the community will self-correct.
Other than the arrowing down bit, it's about the same.
For that matter, I'm dubious that an interface that needs ten fingers to handle shifting between applications constitutes any massive improvement for anyone.
The ideas are definitely interesting, but I am amazed at the clarity of the presentation.
I had a similar idea a while ago and might try to implement it this summer as a side project using Johnny Wii's tricks perhaps + Windows 7 (not sure how much of the interface I can pull off). I would be curious to see in particular how tired I am (i.e. my hands) after using such an interface for a while.
It is refreshing though, to see research that dispenses with the legacy of GUI convention we've built up over the years, instead of just piling on new features.
I was disappointed to see at the end of the 10gui video a concept keyboard with staggered keys and with backspace way over in the corner. If we can't move keyboards past the constraints imposed by mechanical typewriters, what hope do we have at changing the mouse paradigm?
[1] Typematrix is the only keyboard maker I know of that currently does this while still building a relatively standard keyboard (and something that could be integrated into a laptop).
This works surprisingly well, but I really wish the spacebar was split in half. And I've always felt shift+spacebar should = backspace.
The Kinesis keyboard with backspace, ctrl, alt, enter, home etc using my thumb AND the iGesture pad from Fingerworks ( http://www.fingerworks.com/ ) which is a large "mouse pad" where you can use gestures with 5 fingers (and it works for all applications).
The brilliant part is that the iGesture pad fits perfectly on the top of the Kinesis keyboard (the left and right part of the keyboard is separated for ergonomic reasons).
I've had the iGesture for a couple of years now - it will be a sad day when it dies as there are no replacements that I know of....maybe Apple will release a similar "mousepad" in the future ?
I don't know anyone that uses Windows.
I once wrote a python program to trigger callback functions when specific keys on my MIDI keyboard are pressed. The MIDI protocol was surprisingly simple to figure out (for simple things like "button-x was pressed at 25% pressure" anyway). It had a learning mode where you assign a key by pressing it and an action mode where pressing the assigned keys would trigger functions to be called.
I had intended to use it as a shortcut panel to run a shell scripts, but never actually finished it.. might do so sometime soon :-)
Now all I need is to begin using a keyboard-centric web browser. Sooooon!
The mouse is just one aspect of the computer interaction. I can't remove my keyboard from my desk just to make room for this mouse replacement. The keyboard seems a big barrier (literally!) for getting this device on the desktop.
Funny how much of the video seems act as if the mouse was the ONLY way that users actually input information into computers in the last twenty years. But even the newest user is going to be using the keyboard and the more experienced a user gets, the more they use the keyboard. And even more, as computers become a normal part of human society, interfaces pandering to novices will become less important, not more important. The question will change, is changing now, from "how do I use this thing?" to "How can I be more productive with this thing"?
Conside that MOST of the data that users input is going to be text, which a form of discreet digital data. A mouse or manipulative multitouch interface, no matter how well designed, is useless for this because it inherently produces analog input which is then translated to a digit action - dragging a file to the trash versus just pressing the delete key.
Photo editing or moving windows are exceptional cases but UI designers are constantly developing clever ways to do them specifically because they allow all sorts of kinesthetic analogs. Yet none of this solves the problem that this isn't the interface a user wants to do most their tasks.
Apple bought them up and stopped making their products (but we got multitouch on the iPhone, an interesting trade-off).
Outside of the proposed UI changes, Fingerworks product did all of this. I still wish I had bought one when I had the chance.
Anyway, if they are serious about getting these made, I believe University of Delaware owns a fair amount of Wayne Westerman's patents, as the basic idea was the premise for his doctorate. He was a professor starting in '99, so that should make them easier to locate.
Here's some pictures:
http://www.flickr.com/photos/pauric/sets/72157616093748066/w...
I also don't see any correlation between Pauric on Flickr and C. Miller of 10/GUI, at least not by exploring Pauric's Flickr profile or Twitter account.
Perhaps you can explain why you think that those images are of 10/GUI's device? How did you find them?
Sorry about that.
10gui seems to be a very cool prototype and I guess that Apple and Microsoft are already working on things like this. I also spotted a weakness of this demo: 10gui uses the little finger to invoke contextual menus. Most of the people who cannot touch type with all ten fingers (and my guess this is about the 99% of all computer users :) will find this uncomfortable.
But I may be wrong. Perhaps when it comes down to it the interface will feel so intuitive that there will be no problems at all. We'll have to wait and see.
Although, the problem with that kind of implementation is that it doesn't have the tactile feel of a keyboard. however, iPhone users seem to have acclimated after first bitching about it.
Apparently people could get up to 70 wpm with what could be considered the prior version of this.
For what it is worth, at least when I was involved with it, most people found a keyboard a lot easier to use for rapid typing.
I wish they had considered a way to make the keyboard a part of the device. Perhaps an on-screen keyboard, some haptics? The challenge with that part of the input device is that to touch type, you need that spacial reference a keyboard gives you.
As in, place all eight fingers onto the pad and this automatically becomes home row. Simply strike above and below home row. Depending on the finger you moved, it'll know the key you're striking. I rarely have more than one finger off home row, per hand, when typing.
You heard it here first.
(I saw Ken Perlin of the nyu media lab give a presentation on that earlier in the year and it was incredible, high resolution and cheap to make)
It is amazing how intuitive touch can be. I just love the multi-touch on the touchpad that Ubuntu for example supports. I never knew my old Thinkpad T43 would be able to do it but it does.
Two of the concepts seem very heavily Palm Pre-inspired: The window management scheme (reminiscent of 'cards') and the gesture areas on each side, one of which is used to launch apps just like the Pre
Just as in the early days of the mouse and windowed GUI, a system based upon 10/GUI's principles would likely be best suited to a systems builder capable of full vertical integration."
I wonder who they're thinking of.
If I have to scrape two surfaces together constantly to use a computer, I'd rather one of them wasn't the most sensitive part of my finger.
"This is Sussudio, a great, great song, a personal favorite."
30 seconds in and they haven't shown it yet?
Oh, and all accompanied by the Most Annoying Music In The World.
Ding. Ding. Ding. We have #fail.
It's pretty cool and all, but this isn't going to change anyone's life. The best idea here is just putting a big touch pad on the desktop in front of your monitor.