A Conversation About Fantasy User Interfaces
subtraction.com
subtraction.com
However, there will always be tension between that narrative-first approach and the real world where narratives don't exist. A real user, trying to use a real interface, just needs it to work. There's no audience, no dramatic tension, no character arc, etc. It just serves their needs in the moment or it doesn't.
In a story, the fantasy UI really is a pixel-deep metaphor substituting for the technological capabilities available to the characters. "Access denied" isn't really an interface, it's a visual way to communicate a complex plot point, just like all of the other crafted visuals in the movie. Nobody ever has to puzzle over a bad interface for the same reason nobody ever has to go to the bathroom.
But real UIs are more than one pixel deep. They're the Rosetta stone that translates between human thought and machine logic.
So when Minority Report makes up a gesture-based interface it's because they have to show the audience something plausible that they've never seen before. However, when a designer makes a real gesture-based interface it's because there's something their users can't do with the interface they're used to.
You're right when the designer is good. A bad designer might make a real gesture-based interface just because he saw Minority Report and thought that it was cool.
My favorite movie of all time is called Stranger Than Paradise by a director named Jim Jarmusch, in part because of its original treatment of these issues. The film is peppered with subtle little moments where, for example, a kitchen cabinet doesn't close correctly, or someone doesn't hear the person they're talking to correctly and asks them to repeat themselves.
Bo Burnham's standup special has some meta jokes like this. I won't spoil them in case you haven't seen it yet.
One of the reasons Glass failed was because it was about as in-your-face as possible. Resistive screens also failed in the same sense because having to pull a stylus out to use your phone is aggressively technological.
In contrast, touch is a really casual interface, and though the touch UIs that I've used all had warts it was at least possible for anyone to grasp the basic mechanics of the interface. In my mind a wearable device that is obviously a wearable device isn't going to be as successful as clothing that monitors your heart rate or SpO2 because you have to consciously interact with the device. If you fill your gym bag with smart clothes then you just wear them like you would wear gym clothes.
A smartphone is a complete failure from that perspective: looking down and pecking at a screen? No way. Now, holding and looking at something is a natural enough movement, but chronically craning your neck down with such focus and duration? I doubt it. A smartwatch is a bit more natural, but to bring up subjective aesthetics again, I'm not too comfortable with the concept of adults wearing digital watches.
Google Glass was a step in the right direction in my opinion--you can interact with a UI while looking straight ahead--but as you said, the camera and other aspects weirded people out (there may be a future in contact lenses with similar tech?) Voice interfaces like Alexa are definitely in line with my view.
That's why so many people are trying to get voice interaction right. It's the most natural way to split our attention.
A lot of this IoT nonsense is trying to discover a few corner cases where computers can do sensible things without a human needing to specifically focus on them. Like automatically reordering cereal, or unlocking your phone when you get close, or adjusting the temperature when you leave the house.
Resistive screens worked fine with fingertips, too. In fact they worked perfectly well with gloves, pens, etc., unlike modern capacitive screens. They failed because they were less sensitive and less accurate.
My favourite feature - changing colors.
Yes, Tilt Brush.