How 'Minority Report' Trapped Us in a World of Bad Interfaces
theawl.com
theawl.com
Of course, Pictures Under Glass interfaces have advantages too. Touching a screen has become a familiar way to interact with technology: a more tactile interface might be harder to learn. To make a screen-based interface, you only need to draw a 2D picture and respond to 2D touches, not design a 3D object which can be manipulated in space. You can buy touch panels off the shelf instead of having to manufacture expensive custom 3D parts.
But these are advantages of easiness and cheapness, not quality. If designers and engineers can put in the time to make a physical interface and users can put in the time to learn it, you can improve quite a bit on Pictures Under Glass. Think about how terrible it would be to replace a car's steering wheel with a touch screen, for example.
So that's really what the article is saying: we should think of touch screens as the cheap, easy option, not the exciting, futuristic one. Our futuristic starship control panel should be a custom-manufactured physical interface instead of a cheap glass panel.
Big assumption that you're using a mouse (especially considering the ubiquitousness of laptops). Laptops provide a ready-made surface for fully portable use and I've found getting using a touchpad for professional and personal use[0] to be more than sufficient and that the availability of easy to use gestures is very nice.
If you haven't tried gestures on OS X with Apple's touch hardware, I'd highly recommend attempting to use them regularly. I got a fancy Razer Blade 2016 and using the touchpad is significantly worse than an Apple laptop, and OS X has much better native touch support (3rd party software might make Windows better).
I'd say touch interfaces are great compliments to keyboards for text editing workflows (more or less all I do professionally), you don't even need to move your hands off the keyboard to get a mouse (hooray thumbs!).
I would liken getting used to laptop keyboard/touchpad as getting used to a minimalist *nix install. It's difficult at first, but once you get used to it you have a great skillset that lets you do anything you want on almost any computer. Not that you should never nice high-level and user friendly programs, but being able to drop down a level of abstraction and still get the job done is great for understanding how systems work.
[0]: most of my video games suck without a mouse, but that's the only wall my personal use runs into
I think hands are great when there is a massive multitude of possible next actions, like the knitting example, or artistic applications. Otherwise, if we can logically eliminate options of what the user could be doing (this object can only go in so many slots, etc), it's easy to just assist the user with that (think snap-to-grid) and then a precision input device is less necessary.
To be critical, this is a biological feature much more than it is a bug. Our bodies don't have very good feedback mechanisms for the small accumulative damage from fine motor control, which leads to things like RSI and Carpal Tunnel with very few in-between indicators.
In a way, moving to a more gestural interface that feels physically tiring is a possible means to better overall ergonomics because people actually start to get real feedback (tiredness, soreness) telling them to take a break every now and then when they are physically straining themselves.
That's fine if it's a variation on some frequently used command but I find myself forgetting the triggers and skills I have set on Amazon or the names of playlists I might want it to play.
Again, speaking for speech to text takes some practice, so it doesn't quite make sense to compare a lifetime of typing against your first shot at speech recognition.
People can also write shorthand with a pen at 300+ wpm.
A big part of the problem with regular typing is that our phonetic alphabet is not optimized for writing efficiency. Sometimes a single syllable takes 5+ discrete characters to type.
Standard QWERTY-style keyboards are also designed to use single finger motions in serial fashion, and requires various awkward combinations of finger motions and plenty of repositioning between keypresses; the design is designed for learnability rather than efficiency, and wastes much of the possible bandwidth of the hands.
If someone with deep knowledge of human hand physiology / neuroanatomy were to design a keyboard optimized for efficiency (at the expense of requiring years to properly master), we could probably push peak performance to 400–500 wpm.
At some point thinking speed is the main bottleneck though. Regular QWERTY keyboards are fast enough to keep up well enough with most writing tasks.
The world record for speaking is about 600 WPM. The world record for stenographers is about 360 WPM.
A typical speed for speech is 150 WPM, for someone who is practiced but by no means an expert. A typical (but somewhat fast) speed for typing is 80 WPM, for someone who is practiced but not an expert.
So if you're going to compare the fastest speeds or the typical speeds, speaking comes out ahead either way. It's almost as if our ancestors had evolutionary pressure to develop speech, but lacked the same kind of evolutionary pressure for typing.
As for auctioneers, most of what they say is repetition and filler. The fast talking is a gimmicky technique aimed at improving sales, not a practical tool for imparting information at maximum possible speed.
As I said before, the keyboard is not remotely optimized for the capabilities of human hands, and English orthography is not remotely optimized for information density as an encoding the spoken language.
If someone speaking had to say a full syllable for every letter, speaking speed would be much slower.
> almost as if our ancestors had evolutionary pressure to develop speech, but lacked the same kind of evolutionary pressure for typing.
More precisely, speech was developed over millennia by millions of people mutating it slowly over time, whereas modern typing hardware was designed by a single person, based on the mechanical characteristics of a particular prototype typewriter in the late 19th century (before any concept of “touch typing” even existed, and typists were forced to adapt to this mostly fixed hardware design as best they could.
http://fivethirtyeight.com/datalab/the-fastest-rapper-in-the...
Per language.
Python, you have to say return space space space space every damn line. Return space space space space space space space space while inside a function. It would nearly drive a man to switch to tabs.
Navigation, "go 2 characters to the right" isn't pleasant.
To top it all of picture the fun you can have in vim using speech to text. Dear lord.
Now imagine an open office full of people all doing this at the same time. That my friend, is hell.
Interesting convergence would be VR mixed with TTS processing. So, you use VR and presence/hand tracking to physically interact and words to refine and/or program the visual blocks.
We type only because we can't talk to computers (although it starts changing). Voice control would easily beat typing.
Keyboards are also much better than speech for making very precise inputs. You’ll be hard-pressed to invent a speech interface which outdoes vim (for instance) for efficiency at general text editing tasks, even if you have a general AI interpreting the speech commands. Even just for straight typing computer code, mathematical formulae, musical notes, poetry with precise constraints on punctuation & spelling & layout, or e.g. controlling a video game or CAD program, a keyboard is much more effective than speech. A keyboard input can also have real-time response to partial input, like autocomplete, etc.
Perhaps most importantly, typing on a keyboard doesn’t distract/leak the content of your typing to everyone in earshot.
I disagree. Modern aviation has been moving away from physical controls and towards generic screens. The last gen of fighter aircraft (The F/A-18, later block F-16s, the F-22) have a collection of generic screens with context sensitive buttons on the side for control.
The F-35 takes this a step further and has a single large screen (which is also apparently a touch screen).
In something as complex as an aircraft (or a starship), there are hundreds (or thousands) of different controls. Look at the cockpit of an older aircraft (e.g. a Boeing 747) as an example. Most of the controls in a cockpit are used in certain situations, you don't need to control the landing gear or the brakes when you're at 30,000 feet.
By using generic displays, the relevant controls for the current situation can be shown. Yes, it is more difficult to operate the controls by feel, but this is less of a worry in aviation, where you're usually several thousand feet from anything else, than it is in a car, for example.
So of course, it's all about the expected situation, you wouldn't want a little touch screen that you swipe for the safety on an assault rifle, for instance, because you need to be able to do it by feel in various situations. But a swipe unlock is perfectly acceptable for a phone.
My hope would be that the interfaces of the future would be more accessible to the disabled, not less.
In any case, why do you think that using gripping as a UI action implies that you'd need a strong grip to use it?
Actually, I think this extra thought could help everyone by treating disabilities as just special cases of the general idea that different people prefer different input/output devices and having apps (mostly) stop processing low-level input (keystrokes, pixel hit testing) and switch to handling more abstract "primitives".
The OS would have a set of more abstract primitives: enter text, zoom in, zoom out, rotate, 3d move, 3d rotate, select item, select 2d region, select 3d region, deselect, grab, cancel, undo, etc. You, the user, would need a set of I/O devices that provided one or more implementations of each primitive in the checklist.
One user would send an "undo" to an app with a click of the tongue, another with a hand gesture, another by pressing the ESC key, but the app just gets an "undo" event from the OS.
For 2d region select, for example, one would use a joystick, another uses voice, another uses eye movement. The app doesn't process pixel hit testing to do its own region select, it lets the OS tell it what region of the 2d surface was selected. The app makers don't make the app "accessible" themselves, they just make it respond to the checklist of higher-level primitives.
You could provide at least the core functionality of most apps by this means after which an app could possibly add some lower-level processing in the form of "shortcuts".
When I'm forced to use a watch interface away from my keyboard, I am in a sense "disabled". When driving in my car with eyes on the road, I am "disabled". I'm not a gamer, so relative to a gamer with mad joystick skillz, I am "disabled". Well, fine, just change the approach to software interaction to a more abstract type of interaction that treats us all as being variously abled.
I don't think it's fair to make non-disabled individuals use a less productive interface, just as it's not fair to prevent disabled individuals from using the interface of their choice.
I grip my mouse to use its side buttons to navigate forward/backward in a web browser. Human hands are generally good at holding things, and flat touchscreens/trackpads aren't very holdable. Power tools, steering wheels, gear shifts, volume knobs -- all examples of things that are gripped.
I disagree completely. The line of reasoning is "what actions can we perform that have the highest bandwidth, and lowest cognitive load?" And various forms of manual manipulation, like gripping, are an excellent avenue of exploration when answering that question.
Generally if you want to design for the disabled, you probably are designing for the disabled, rather than using them as fodder for an argument.
No single interface will ever suit all potential users, so any accessible design has to keep this in mind.
Honestly, common sense says the guy is talking about interfaces for the general population, not the disabled. If most people can grip then make interfaces that grip. Don't force everyone to read braille just because a few people are blind.
For the cats.
(I like cats. The problem then was that I had way too much stuff in various cabinets and the cat liked to make a grand tour through them, throwing a lot of stuff to the ground in the proces...)
Taking advantage of major human ability != making it an exclusive requirement to use said interface.
Also, suggestion != claim.
What edge cases? What design? What are you even talking about?
A person merely said "Hey guys, have you thought of this". GP got on their case as if they were against the disabled or something.
I realize that they let the director film the actor without something blocking the camera but they just seem like a lazy way to be "futuristic."
https://s-media-cache-ak0.pinimg.com/736x/bd/17/68/bd1768e3a...
And why are they in so many sci-fi graphical UIs?
The assertion that the gesture interface is some kind of virulent ear-worm, dismisses what may be the most interesting thing, the fact that it has captured the imagination of so many. Why?
Meanwhile, out here in the real world, I'm a real computer programmer and my real interface is even more boring looking than a standard Windows install. Nobody's going to be mining my UI for Hollywood ideas anytime soon, despite the fact that I, on occasion, actually hack.
It's the same reason cars explode when they go over cliffs, bullets spark when they hit things, and all those other tropes.
I was amazed how intuitive the controller UI was. Some of the controls are on the back of your hand, but it was incredible how natural it was to pick up and use comfortably. I've never been all that fond of the Minority Report floating screen UIs, but other 3D UIs can feel very natural in the right context.
So basically: "Good design is invisible"
I'll explain that I'm actually thinking of a movie for the UI, and I want it to be as exactly like that as possible; I'm sure they get this request really often, and it won't be anything new to them; and I'll explain that I even brought some scenes from the movie to show them exactly what I wanted. And then I'll be grudgingly persuaded to pop in the relevant DVD...
And the scenes will be from a pirate movie, with an old-timey European pirate/bandit/mercenary chief writing with a quill on parchment while wearing a coat with ruffles and lace.
Only quills will work as pointing devices. To save your work, you sprinkle some sand on the tablet and blow it off. The tablet comes with a breathalyzer attachment, which checks for enough rum in the air to prove that you're a legit pirate but not so much rum that you're signing contracts while drunk. Voice input must be in Ebonics, which is pretty much what the Cavaliers spoke in the English Civil War. Voice output is in a thick Devonshire burr (the classic pirate accent) that makes you think that the app's probably half drunk.
Now, if only someone's eccentric enough to do it...
I'm also toying with the idea of getting some reenactor-grade pirate clothing: long-wearing like nobody's business, and relatively easy to care for. I've even found places that sell those leather cavalry boots with turned-down tops! For an extravagant price, of course...
For flying, touch screens are a great idea, since you have a lot of controls, and attention to where exactly you're going is less critical, since you're usually >1000 ft from the nearest solid object. Also, aircraft usually have autopilot (and often a copilot).
I guess with Tesla's autopilot, it might make the touch screen a bit more usable? (at least while driving on a highway I guess).
[ 3 minutes of googling later... ]
Of course it's been done: https://mitxela.com/projects/etch_mouse
http://www.detroitnews.com/story/obituaries/2016/06/20/obit-...
Cause you know, having the shift level position indicate what gear its in, just isn't "cool"...
I work at Oblong and recently gave a short (unofficial) talk which covers some past and current activities.
https://creativemornings.com/talks/brandon-harvey/1
My favorite sentence from the piece is "there’s a huge gap between what looks good on film and what is natural to use". Yep!
The ancient Greeks used rooms of a house to organize information. Just think we could of ended up with that instead.
Later I elaborated on the idea in a Unity prototype called MediaGraph [2], which lets you arrange your music in an editable gesture navigable map.
[1] iPhone app iLoci by Don Hopkins @ Mobile Dev Camp: https://www.youtube.com/watch?v=03ddG3jWF98
A talk about iLoci, an iPhone app and server based on the Method of Loci for constructing a Memory Palace, by Don Hopkins, presented at Mobile Dev Camp in Amsterdam.
[2] Method of Loci: https://en.wikipedia.org/wiki/Method_of_loci
[3] MediaGraph Music Navigation with Pie Menus Prototype developed for Will Wright's Stupid Fun Club: https://www.youtube.com/watch?v=2KfeHNIXYUc
This is a demo of a user interface research prototype that I developed for Will Wright at the Stupid Fun Club. It includes pie menus, an editable map of music interconnected with roads, and cellular automata.
It uses one kind of nested hierarchical pie menu to build and edit another kind of geographic networked pie menu.
I am actually looking forward to being able to type with one hand again.
you mean key-bindings?A full keyboard already has ~100 keys.