Keyboard vs. The Mouse, Part 1 (1989)
asktog.com
asktog.com
> I would also like somebody—anybody—to show up with one properly-conducted test report that shows that command keys are equivalent or superior to pull-down menus somewhere, somehow.
It's been 30 years since he wrote that. Has anyone in that time been able to produce data that refutes Apple's findings?
The more items in your menu the slower it is to find things in it. I can type NY faster than I can find New York in your states menu (I picked a state near the middle to ensure you have to scroll, but can't scroll to the end which might be faster if you know).
Tog was forced to back off his claim, but this wasn't publicized as much. A well trained expert with key shortcuts used all the time is faster than someone with a mouse - but if you don't use the shortcut often, or there are too many keystrokes the mouse wins.
"Showing users things they can recognize improves usability over needing to recall items from scratch because the extra context helps users retrieve information from memory."
Perhaps if you're restricting context to one app that doesn't have too much going on it's a different story, but compared to 30 years ago there is a lot more people (especially devs, or Starcraft players) need to be doing on the computer at a given time.
Yes, all of the specific experiments Tog describes are bogus. In versions of the experiments where the keyboard user isn't crippled by weird limitations that don't apply to actual computer users, the keyboard is often superior to the mouse (not always, just often): https://danluu.com/keyboard-v-mouse/
If you saw them, you probably noted they were fully keyboard-operated, had a lot of functionality accessible under function keys, and enabled the clerks to accomplish a lot of tasks without actually looking at the screen(!). You can't beat something like a muscle-remembered "F2, enter, enter, <type name>, tab, enter, enter" with a dropdown menu. I've seen cashiers do the former faster than it'd take to select a single option from a menu.
The only way I can imagine Tog's assertion to make sense is if we're comparing a mouse menu vs. looking up a shortcut on a physical, printed cheat sheet (which is essentially a menu-equivalent), for an operation that you're using for the first time in your life. None of that resembles actual use of real software.
Many times when editing in emacs I just "do the thing" without consciously knowing what keys I pressed.
If given a keyboard with no keys labeled, I would not easily know what each key was. Just the same, I could look away from it and type the alphabet. Magic.
(Not on a touchscreen at least)
More than once I've tried to describe a feature to some other Emacs user, and had to physically put my hands on a keyboard and watch them in order to say which keys were used to invoke it.
It's exactly like how you totally forget PINs, phone numbers, or passwords once you've typed them enough. You need a keypad in front of you (or at least to pretend there's one) and to just let your fingers do what they've done so many times before in order to figure it out.
I don't do well with laptops and mac keyboards due to their nonstandard key arrangements though.
That happened to me once, the electrician had to change the electricity meter, he called me to ask for the code to enter the flat and I couldn't recall it; I had to go down and open the door.
My favorite laptop keyboard is the first gen macbook pro. Great comfortable concave keys with good throw. Some thinkpads were good too.
I mean, that was what the vi fans I met in my youth tried to deny. They claimed positioning by incremental search was better.
In a late uncle's stuff, I came across a copy of "The psychology of human computer interaction" by Stuart K. Card et al. A book from the early eighties detailing Xerox PARC UI research. They did a study comparing cursor keys (or vi keys), word and line keys (home, end, next word etc.), incremental search and mousing. They found the same as Apple later would.
> I came across an interesting example of perception vs. reality while designing a small text editor: When scrolling the text horizontally in a window we would refresh the text by redisplaying each line starting at the top. This resulted in a wave of text rippling down the screen, and many complaints that the screen refresh was too slow. The remedy was to scroll the bits already on screen and then redisplay each line from the top. The second implementation was actually slower than the first because we incurred the overhead of scrolling the bits before we even started to display the new text on the screen. However, the perception was that there was an immense increase in speed. We stuck with the second implementation because it increased the overall satisfaction of the user even though it actually decreased the throughput of the product.
The commenter is actually subtly wrong about what's going on here though (but fortunately does the right thing by focusing on user satisfaction).
Without scrolling the bits first there's a "wave" artifact. That movement will prevent most users from starting to get their bearings on the new position of the content until after the wave is done. Scrolling the bits first hides the wave and allows the user to start getting their bearings as soon as the bits are scrolled. Newly revealed content does take longer to appear, but if scrolling the bits is significantly faster than the redraw the difference is small and the user is able to process the new content more quickly overall by already having their visual bearings.
While the keyboard users in this case feels as though they have gained two seconds over the mouse users, the opposite is really the case. Because while the keyboard users have been engaged in a process so fascinating that they have experienced amnesia, the mouse users have been so disengaged that they have been able to continue thinking about the task they are trying to accomplish. They have not had to set their task aside to think about or remember abstract symbols.
Hence, users achieve a significant productivity increase with the mouse in spite of their subjective experience."
this is not research, this is jingoism. i've heard this study cited so many times but never read it ... i'm astonished it is so bad.
2 seconds to decide what command key sequence to press? Sure, if you contort your test so much (only novel command sequences) that the results apply only to your contrived situation.
The first is that keyboard input is much easier to reliably repeat. Keyboard macros are much less simple with a mouse and even when keyboard macros aren't an option, I often find it easier when making repetitive inputs to convert the inputs to key presses and repeat them.
Furthermore, I find mouse usage interrupts my flow. Using the mouse is a very different mode of input than keyboarding. I have the disadvantage that my dexterity is worse than many people's and so when I have to switch to the mouse, I have to be very careful to click the right thing. I don't usually make the same mistakes with keyboard shortcuts that I use regularly. Even if the mouse is technically faster, that interruption is very unpleasant.
> I have the disadvantage that my dexterity is worse than many people's
This, though, is a very valid point that is too rarely taken into consideration by software makers.
(Automator has some "record" functionality but it's so naïve as to be useless. It records the mouse by where on the screen you move it. If you take several seconds to decide which menu item to select, it pauses during playback, too. If you bump the mouse during playback, it will select the wrong menu item.)
There seems to be at least one person at Apple who understands the importance, because macOS does support Emacs-style control-U repeats for text editing.
This article has more than a faint whiff of "new hotness" vs "old crap". They have rushed to the new thing and hate the old thing. Like rushing from Traditional Relational DBs to NoSQL. Whenever you get a new tool, that doesn't always make the old tool useless.
I think this article also came well before the proliferation of functionality. There are hundreds of functions in a simple text editor nowadays. I can't imagine accessing them all via keyboard alone, but I also can't imagine losing my keyboard shortcuts while seriously editing text.
Programs also added context menus, and more recently search capability for commands and settings.
I think we've finally learned some lessons about not throwing old tools away, but I think we need to keep learning that lesson as well.
- While I largely agree with this, I think one thing this era of computing missed (and Windows still does) is the power of querying for items - I'd love to see a comparison between mousing over, keyboard shortcuts, and querying (e.g. macos's cmd+shift+/, plugins like https://shortcatapp.com) for items between users that are deeply familiar with an application versus occasional users of an application and new users.
It might even be more efficient if it worked like Vimium in the browser, labelling all items with a single letter which you can press to select it.
Also vimac for osx
This is so helpful for commands that I use once or twice a month.
I could always learn their actual shortcut but it does not feel worth it.
When I want to capture a layout hierarchy, I just hit cmd + shft +a and type "lay".
I don't really care about doing this rare action a couple of seconds faster by learning/setting an actual shortcut for this.
From the article:
> Keyboarders only think they are faster—I think its because of the sound of the keys ...
This might be true, as a generalization, but it should be obvious that it doesn't apply to everyone. That's a bit like me pontificating on how a basketball free-throw should executed. Listening to me or studying my free-throw will reveal nothing interesting or important about performing a free-throw. Talk to a professional player, study the players that hit over 90% of their free-throws, but don't try to draw conclusions from non-players just making thing up.
I can't play basketball worth a crap, but I have used computers and their interfaces for over 50 years. All this time I have been programming, and I've programmed on more different systems than I can remember (really).
I've used every mainstream word-processor and every mainstream text-editor, often for years at a time. I have owned at least 100 keyboards and 100 mice and trackpads over the years. These are my personal observations.
* People are surprisingly good at learning repetitive fine motor skills. The average typing speed of regular people is surprising and would be, IMHO, counter-intuitive if not for our ubiquitous exposure to good typists and piano players.
* One doesn't need to be a great typist. My big fingers seem to be made for picking up rocks, not for typing, so I "only" type around 50-60 wpm despite half a century at the keyboard. However, even at half this speed my thinking speed is what limits most of my coding speed and writing speed. I don't do data-entry or transcription where more speed would be really valuable.
* Fast typists are fast because they make less mistakes than the rest of us[1].
* Qwerty is superior to Dvorak, Workman, Coleman, and other layouts for me because I have to use other people's keyboards often. I don't work entirely by myself. If I did I might be drawn to these improved layouts, but even then a small percentage increase in typing speed wouldn't be a big benefit for me (see above). Furthermore, my hands don't seem to be vulnerable to Emacs pinky syndrome or repetitive use issues where these layouts do seem to be beneficial.
* The strange historical accident that gave us staggered keyboard keys[2] is unfortunate. I have experimented with ortholinear keyboards, I have two TEK keyboards[3] and two Kinesis Advantage[4] keyboards. These keyboards have the keys arranged in columns. I really like the Advantage layout, but I make errors when jumping back and forth between my laptop keyboard and my more extreme ergonomic keyboards. However, the main advantage of these alternative layouts for me is not their ergonomics (see above) but the better utilization of the thumbs for accessing additional "shift" keys layers (control, alt, meta, hyper, command) and special vim style leader keys. I have tried the other ortholinear keyboards as well[5].
* Major keyboard manufacturers have ergo-style keyboards, but they suffer from poor physical keys (like my Microsoft Sculpt Ergonomic Keyboard[6]) or poor utilization of the thumbs (the Microsoft keyboard has split shift keys, which is good, but they can't be programmed to transmit different keystrokes, which is bad).
* The mouse is really good at controlling analog style controls on the screen.
* The mouse is really bad for drawing. For this a tablet is slightly better--I own six. I also have also owned a couple of expensive Wacom tablet/display combinations[6] these fail to be useful to me because they are (1) poor displays, (2) expensive, (3) awkwardly big, (4) require software drivers and hardware interfaces that don't work with my newer equipment--usb-c thunderbolt for example.
* The iPad Pro with Apple Pencil if the best really practical, useful stylus for drawing note taking that I have found. It also is quite easy to use as a tablet with a Mac.
* The one-button mouse was great for my Mom, but I am more productive with more buttons, and for gaming I regularly use a gaming mouse that has 12 thumb buttons. This works great, but it does take a few hundred hours of use to become proficient. (My daughter, an avid gamer, turned me on to the Razor mouse[7]).
* The Apple trackpads are the best trackpads. Trackpads seem particularly well suited for content consumption, like browsing the internet and reading email. They are much less suited for operating other analog style screen based interfaces such as draging icons (folders, selected files, etc.) around the screen. For these later tasks, a mouse is superior.
* The Macbook Touch Bar is pretty, but for professional users, it is not worth giving up function keys to have a touch bar. The lack of an escape key makes it difficult to use vim. I can remap the escape to caps-lock or use a control-key combination instead, but then I fumble around when I'm trying to use a different system. The butterfly keys suck--I haven't decided when/if I'm going to buy a new Apple MacBook Pro with the newer keys because they still don't have function keys. Furthermore, my desktop Apple systems don't have the same keyboards and they don't have the Touch Bar.
* In the early days of GUI interfaces, Apple was right to put the menu at the top of the screen. It served two purposes, it prevented confusion as to which app menu on a cluttered screen was the correct one to interact with and it was easy to get the mouse to the menu--just shove the mouse up and to the left. It eventually hit the upper corner of the screen where you could visually pick up the cursor and guide it to the appropriate menu item. With separate windows on MS one had to guide the cursor visually all the way to the menu. However, now with the large/wide/multiple screens on my desktop systems it means dragging the mouse a long way from where I was using it to interact with a menu.
* I'm not very happy with the wide variety of keybindings that exist for Emacs. However, a properly configured Emacs system has the absolute best system there is for managing software that has one-thousand different functions that a user may want to execute. Nothing I've seen compares to a well configured Emacs for making hundreds of commands available at one's fingertips. Using a mouse instead would be laughable. Emacs requires an investment in time that will not be worth the effort for most people. Like practicing free-throws, there is no point for a regular person to master Emacs. However, if you strive to be a professional developer (or basketball player) practice makes perfect and it is worth your effort.
* Configuring Emacs is important, however it is completely programmable so there is no limit to how much time this will take. Stand on the shoulders of the giants that have come before us and use someone else's configuration. Spacemacs and Doom are great configurations there are many others[9,10,11]
* Vim is great too and is better than Emacs at some things (but I'm currently being faithful to Emacs)
* Keypunch machines suck as IDEs. Just ahead of keypunch machines on the hierarchy are line editors on paper teletype machines. The next step up are editors like TECO. All of these are archaic ways to program that I've been saddled with in the past.
* Real developers should adopt tools as powerful as they can learn to control: Emacs, neovim, and JetBrains suite of tools, all take study, pick one because they are totally worth it. There are other editors, but don't wimp out and pick one less powerful than you can handle. Almost every developer should also learn git, another tool that has tremendous advantages and is worth the effort. Every academic that works with mathematics, data, or code should learn LaTeX--this will take time but again it's worth it for serious work.
* Xah Lee has thought about keyboard, mice, and menus a lot. I like his thinking about keyboards; you should check out his website if this subject interests you. [8]
[1] https://userinterfaces.aalto.fi/136Mkeystrokes/
[2] https://www.forbes.com/sites/quora/2019/01/10/why-was-the-qw...
[3] https://trulyergonomic.com/ergonomic-keyboards/
[4] https://kinesis-ergo.com/shop/advantage2/
[6] https://www.wacom.com/en-us
[7] https://www.razer.com/gaming-mice/razer-naga-trinity
[8] http://xahlee.info/index.html
I take it they didn't measure people typing a page of text using only a mouse and on-screen keyboard.
With the right interface, I wonder if we couldn't learn to enter text really fast with a mouse, even without prediction. Of course there's little point in exploring that design space since we're so fast with (and have invested so much in) keyboard typing.
Either way, I think designing a keyboard only interface to play Doom efficiently would be much harder!
If users prefer to use the keyboard, they're not as likely to enjoy (and use) your "mouse first" software.
https://9p.io/wiki/plan9/Mouse_vs._keyboard/index.html
A take-home message from there: "In the experience of many Plan 9 users, using the mouse in Plan 9 for an extended period of time and then going back to using vi in Unix highlights the amount of time you spend watching the screen as you cursor around with hjkl. Having broken out of the hypnosis that Tog describes, I just get frustrated beyond belief. Yes, I am watching the cursor move so I should be occupied, but all I can think is ``damnit, if I could just click where I want to go I'd be there by now.'' "
See also high-brow three button mousing in Acme, Plan9's text editor:
I'm a long time Plan9 fanboy, but I find myself more leaning towards line editors (Pla9's sam in terminal mode), partly to avoid watching the cursor all the time. In a line editor, you have to carry the structure in your head, and it is liberating and calming in a strange way.
Ken Thompson, prominent sam user, has been attributed saying "I don't want to see the state of the file when I'm editing." I really like this; then again, I am not a real coder, just a hobbyist and writer of words. So I have no idea how realistic it would be to use console sam or ed for daily professional coding. The way line editors resemble a typewriter can be an excellent cure to the so-called writer's blocks, though.
(See the interview containing Ken Thompson's quote: http://web.archive.org/web/20080103071208/http://www.dcs.qmu...)
vi is four decades old. It's not the pinnacle of keyboard-driven interface design, vast improvements have been made since then.
For example, I use vim-sneak, which lets you jump to anywhere on your screen in 3 keystrokes flat 99% of the time (and the 1% of the time where you miss, well, 99% of those times, just one more keystroke is all it takes to get to the right spot): https://github.com/justinmk/vim-sneak
I also use relative line numbers, so I can select or delete most blocks of text in 3 or 4 keystrokes, for example d4j deletes the current line and the next 4 lines (5 lines total), or d23k deletes the current line and the previous 23 lines (24 lines total): https://jeffkreeftmeijer.com/vim-number/
Obviously number of keystrokes isn't the only factor that determines operation speed. This is actually exemplified by my second example, commands like "d23k" actually work out-of-the-box even in original vi, the problem is that counting 23 lines by eye is way slower than selecting 24 lines with a mouse by eye, which is solved by relative line numbers.
Interesting. I wonder did anyone conducted a recent research.
Keyboards stayed the same over all these years while mice progressed a lot, modern ones have many thousands of DPI and deliver up to 1KHz of samples. This makes mice work much better in comparison.
Modern displays have much higher resolutions and sizes, a 4k 27” display, even scaled to 150% DPI, contains much more stuff than 9” CRT of 40 years old Apple computers. This makes mice work much worse in comparison.
I wonder which of these 2 trends wins.
The screen, on the other hand, is much smaller and therefore faster to navigate by mouse. The top menu bar, in particular, doesn't feel like a chore to reach for like it does in modern macOS.
No multimonitor on Mac plus either. Nothing about a menu on one screen and the thing it controls on another makes much sense :(
My experience is mostly with Macs, but I guess other OSes have all the similar things.
First, physical size matters. A button on a 4k screen will probably be a similar physical size as a button on a 9" CRT (or even bigger), so targeting is usually not an issue (unless there are glitches in scaling or you don't go for Retina's 2x scaling).
Second, mouse movement is calculated based on speed. If you drag the mouse quickly, the pointer will travel a greater distance for the same mouse movement than a slow drag. This helps move the mouse quickly and then adjust position.
Modern mice are much more precise than old mice. The high resoltion screen makes things more readable. All in all My guess is that the situation is actually better for mouse users than, say, 15-20 years ago.
IME, mouse acceleration is one of the bigger problems for people who rarely interact with computers. Not sure why it's turned on by default so often.
I'm definitely in the keyboard camp and I know for a fact this is not the case for me.
When I am keying in code what I tend to do is just read the text as it is produced.
I'm not consciously thinking about which key needs pressing, and I know this only because on rare occasion I actually find that the text I reading is not the text I was expecting.
On those occasion I stop typing and at that point I then do need to spend a few seconds consciously trying to figure out what went wrong.
1989 I didn't really use the mouse that much it really was amigas / ataris / macs and those interfaces were really built for mice.
Also you need to make a distinction between applications you use all day every day and applications you use sometimes.
A lot of games achieve a lot through mouse/keyboard cooperation. As do people in various designers photoshop/blender/unity etc
For coding, a lot is achieved through things like VIM bindings.
Then we have things like browsers, which are really mouse first UIs that you can drive through keyboard and enhance with plugins like Vimium.
Also add into the mix trackpads. Useless for somethings, awesome for other things ( especially through multitouch gesturing )
So, all these scenarios it would be interesting to have a thorough analysis of perceived speed, actual speed, etc.
Sorry but between doing a high level, interesting activity that takes a few more microseconds and doing a mind numbing activity that makes me want to die inside, and also takes a few less microseconds, I choose the former.
We should not base everything in efficiency. Eating only soylent green is efficient but also awfully boring.
Not much besides that as far as I can tell.
In some ways I struggle to find anything quite as flexible. Emacs comes close but it’s way more complex.
Quark Express, those we the days!
That was the first software that made me realize how powerful computers could be and how they would change the world.
Macromedia Director was the second.
Solid implementation beats lofty design maybe?