Where Will Virtual Reality Take Us?
newyorker.com
newyorker.com
Only a tiny minority of people have lived on a frontier, that’s sort of the inherent to the definition. Most people now live in a very constructed environment (urban areas). In the past, most people lived somewhere in between; towns, suburbs, etc.
Many of us find our mysteries in artificial areas: parks, little shops, back yards. There’s plenty of beauty and complexity in nature of course. But in human spaces we see just as much complexity and mystery; somebody builds something, it decays, gets fixed up, added to, gets some graffiti. We’ve all got little worlds in our heads that motivate us to do things, there’s plenty of complexity in there, especially once our actions start bouncing off each other.
I think most of the problem with VR so far has been, well… first, the tech just isn’t there yet, it kind of sucks still. But also we’re mostly getting silo’d VR experiences from single authors or companies. Games. There’s no mystery there because it was all intentionally placed by the designer. Eventually we’ll get better persistent dynamic multi-user spaces I bet.
What this whole AR/VR obsession is all about is concentrated tech capital jealousy afraid of anyone even appearing to be slightly more technologically advanced. It is all ego.
Why? To either claim.
Also, is it possible to accidentally be disingenuous? I’m happy to be wrong, but I guarantee that I’m not intentionally being disingenuous.
> I would take a mediocre physical couch over a virtual one any day. At least, my buttocks will have some cushion with a real couch.
I don’t really follow. I certainly haven’t suggested replicating a couch in VR. I think it would be just as bad as trying to replicate a couch on a monitor.
Apple has been the king of telling consumers what they want for years. If they can't sell VR, I don't think anyone can sell VR. This feels like the make or break moment for VR, where we'll know what people want to use it for/if it's capable of living in the mainstream, or if it's forever going to live as a niche product, as it has been since the VR renaissance started when the Occulus Rift came onto the scene.
FOV is still bad. Weight is still too high. Resolution I'd say is adequate for gaming (but see FOV above).
Apple are selling the same problems in a very expensive package that they're desperately trying to pretend isn't a VR platform.
I think this is a better approach than trying to get people to do spreadsheets in the metaverse.
> If they can't sell VR, I don't think anyone can sell VR
Imagine having as many monitors as you need. Anywhere. Rearrange them whenever and however, even on airplanes! It all weighs under 1 pound (0.45 kg).
How much would you pay? I'd pay at least $600.
> where we'll know what people want to use it for/if it's capable of living in the mainstream
A cheaper, better multi-monitor replacement would be very useful. It might even be fun in some cases, but it's still mostly a productivity tool.
Unfortunately, short-term incentives seem to limit funding to projects with potential for:
1. Platform lock-in (Apple Vision, Oculus)
2. Ecosystem extension (Apple Vision, Valve Index)
> or if it's forever going to live as a niche product
Is having 2+ monitors an obscure niche? An overwhelming number of offices and other workspaces disagree.
And this would make it even worse.
The problem with physical displays isn't that you can't move them around, it's that there's no reason to.
It's like a chair, sitting for too long is terrible for your body. Chairs don't literally limit you from getting up, but there's just no immediate reason to get up, do some stretching.
It would be the same except you're adding annoying glass and plastic attached to your face.
If a headset can be light enough, it can be a tool for AR documentation overlays. It's like taping a manual up next to the thing you're working on.
For example, engine components. I've seen airline mechanics consulting laptops as they work on engines. I didn't get a close enough look to be sure, but it seemed like it was a combination of manuals and running diagnostics.
> it's that there's no reason to.
Some monitors come with swivel mounts to support both horizontal and vertical modes. It's common enough that I think it's worth exploring.
> except you're adding annoying glass and plastic attached to your face.
This is absolutely true. Getting headsets down toward the same weight range as regular glasses is probably a major hurdle for use outside industrial or piloting applications and into the office.
because putting weight on your head does that. Your muscles and joints are not going to like it no matter how you try to be comfortable.
> thats not a Very enticing proposition
Consider the following use cases:
* A surgical team consulting a 3D CT scan as they remove the tumor threatening to kill you or someone you love
* A firefighter who can overlay a building's plans over thick smoke
* A rescue helicopter pilot who can see LIDAR and IR data as needed
All of these are literally life-saving. However, office uses such as replacing multi-monitor desks are far more pleasant to encounter.
AR definitely has real industrial uses but 90% of the examples people can think of are not better or are worse than current methods.
The problem is "virtual" monitors in VR aren't high enough FOV or resolution to be as useful as real monitors.
I intentionally chose heroic examples because GP's viewpoint is understandable. Not everyone is as excited by window management and good UI as I am.
OP's article touched on attention management through UI a little bit, but I think that concern isn't given enough attention. Using the most heroic scenarios possible helps drive the point home without getting grim, but I will now. To be blunt, it's a historical fact that bad UI kills people [1][2].
> AR definitely has real industrial uses but 90% of the examples people can think of are not better or are worse than current methods.
I agree completely. My main concern is pinning manuals or reference sheets next to my work, even if paper fallback is still crucial. That's pretty much the use case for multi-monitor anyway.
The surgical example is the same thing: CT scans are used to tell the surgeons where it's ok to cut. However, there's an extra benefit here: reducing the need for a physical 3D print of the skull reduces infection risk. It's one less thing to moved into and out of the OR.
1. https://en.wikipedia.org/wiki/Therac-25
2. https://arstechnica.com/information-technology/2017/11/uss-m...
Super tangential but this documentary is about pr0n going mainstream
https://www.imdb.com/title/tt0418753/
In it they point out in the 70s how the movie the documentary is about was a huge public topic of discussion. They show Phil Donahue actually telling people to go see it (or at least I remember that in my memory. If it wasn't him it was some other household name). Eventually the prudes push back and effectively get it banned from lots of places but the VCR comes out and the prudes lose.
VR pr0n is pretty amazing. It's not perfect because there's only so many things that make sense first person. But, when someone gets up close and breaths on you your brain tells you you felt something even though you know that's impossible. If you saw it and didn't find it compelling I'd suggest you ask around for some better examples.
Then there's apps like Virt-A-Mate that take to the next level by being real 3D (Game tech) instead of recorded video. They're not perfect. They have bugs. They run too slow sometimes. But, unlike video, you can change any character to anything you want. That's only going to get better. Apple might ban it but eventually it will be a reason lots of people want VR/AR
Friends at Meta already tell me their metrics say the #1 use of VR is pr0n. (no different than the internet itself.
Nowadays the youth call it corn or p*rn or just (insert corn emoji here).
> because one can see all imperfections
be qualified "for now".
Once upon a time 2D digital porn d/loaded scan line by scan line over several minutes in low resolution and reduced colour maps.
However that may (or may not) push the use of models that live IRL and look pretty decent from most angles being used as the foundation for VR modelling.
The point being, one way or another, 3D VR will likely lose the weirdness, through viewer acclimation or product delivery advancement.
I'm curious how he feels about Half Life: Alyx.
What makes that game so compelling is the degree of interactivity it affords. You can pick up anything. If it's glass, you can break it. It allows you to interact with a tangible world to a degree that no other consumer VR software does.
It's one of the most well-reputed games of all time, and it's only available in VR. As such, modders have figured out how to make it playable without a VR headset. The reviews of non-VR players are that it's actually a pretty boring game when you're not in a simulated environment. If you're not taking the time to open the drawers and break the bottles and duck behind the barrels in a gunfight, it's mostly just walking through tunnels waiting for the occasional ambush.
Its not relevant in my day to day life.
And as long as i'm limited to what VR can do today, it will never replace reality.
I want to move in vr, i want to touch, move around freely without hitting the floor, walls or roof.
You’ll figure it out, but take the time you have now to discuss with your spouse what your approach is going to be.
Good luck!
I'm disappointed there are no images in this article. I'd love to see this. I love how it celebrates something by giving it a more playful character.
Is imagination not infinite?
Someone is still on this journey.
Yeah, that's Jaron Lanier.
Apple's headset isn't VR at all. It's stuff overlaid on the real world. Rather banal stuff, usually screens. That's augmented reality.
The hardware ought to be able to support most of the stuff Magic Leap faked a few years ago. Since you see the world through cameras, they can draw dark, which optical overlay systems cannot do. Someone will probably do some games that way. Pokemon Go would work.
Someone really should do Hyperreality, the famous concept video of AR hell.[1]
> There are many reasons why V.R. and gaming don’t quite work, and I suspect that one is that gamers like to be bigger than the game, not engulfed by it. You want to feel big, not small, when you play. (“Star Wars” might have got this right with holographic chess.)
Don't see that. VR about going some place more interesting than where you are.
As for tabletop AR combat chess, Jeri Ellsworth made that work quite well with Tilt Five.[2][3][4] Good idea, never really caught on.
[1] https://www.youtube.com/watch?v=YJg02ivYzSs
Yes, it is VR, no matter how much Apple wants to avoid saying that.
Reading all the reviews, the thing most people agree it is good for is immersive media consumption, which is definitely VR.
You cant go 10 comments in a post about it without people saying they want to use it on an airplane to block out their surroundings, which is again VR.
Even 90% of the functionality people claim is AR is really about having a virtual computer monitor that isn't interacting with the real world, which is more mixed reality than augmented reality.
I think highly of Jaron and his unique point of view, but this is invalid.
This is one of the throughlines of Interstate 60. It's an absurd existential comedy from the writer of Back to the Future with an insane cast for an indie film. It was one of my favorite movies in college; I wonder how it holds up.
-----
From the movie:
> He said the frontier was a safety valve for civilization, a place for people to go to keep from goin' mad. So, whenever there were folks who couldn't fit in with the way things were, nuts, and malcontents, and extremists, they'd pack up and head for the frontier. That's how America got started - all the crackpots and troublemakers in Europe packed up and went to a frontier which became the thirteen colonies. When some people couldn't fit in with that, they moved farther west, which is why all the nuts eventually ended up in California. Turner died in 1932, so he wasn't around long enough to see what would happen to the world when we ran out of frontier. Some people say we have the frontier of the mind, and they go off and explore the wonderful world of alcohol and drugs, but that's no frontier. It's just another way for us to fool ourselves. And we've created this phony frontier with computers, which allows people to, you know, think they've escaped. A frontier with access fees?
On this line contrasting Apple's approach from Magic Leap's approach:
> Camera-based mixed reality is vastly easier to accomplish than the optical version, but it is concerning. Early research by a Stanford-led team has found evidence of cognitive challenges. Your hands are never quite in the right relationship with your eyes, for instance. Given what is going on with deepfakes out on the 2-D Internet, we also need to start worrying about deception and abuse, because reality can be so easily altered as it’s virtualized.
The two thoughts, cognitive challenges and deception, are unrelated. And Lanier spends no time saying why the optical version will be free of deepfakes. Instead he uses this piece of emotionally charged language to justify why the camera approach is inferior:
> Today, in much of the online world, algorithms calculate “feeds”—a term suggesting that people are cattle, and not free-range ones.
Again seemingly uninterested with engaging substantively with why the camera approach is worse than the optical one other than by making us feel, well, moral.
Then there's:
> Today, the V.R. community seems to want to make headsets smaller and smaller, until they disappear. But in the old days I’d build one-of-a-kind V.R. headsets into big masks from different cultures, sometimes adding lightning bolts and feathers. I wanted the headsets to be vibrant, exciting objects that enriched the real world, too.
It's unclear how much this is some shift in values vs simply a shift in aesthetic culture. The culture of the '80s and early-'90s was loud bikes, loud cars, big heavy computing, lots of cables. This isn't the aesthetic culture of today.
What confuses me the most is:
> There are fresh, urgent reasons to reaffirm the value of experience. It is impossible to judge technology without a sense of its purpose—and its only plausible purpose is to benefit people, or perhaps animals, or the over-all ecosystem of the planet. In any case, if we pursue technologies that make it hard to delineate the beneficiaries—for instance, by blending brains into robotics not to cure a disease but just because it seems cool—then we make the very idea of technology absurd.
Even though near the beginning Lanier says:
> V.R. can be a way of exploring the nature of consciousness, relationships, bodies, and, perception. In other words, it can be art. V.R. is most fun when approached that way.
So which is it? Must technology be anchored to a purpose or can it simply be art? The idea that technologies are purely utilitarian is surely not something people still believe right? Not after synthesizers, modded cars, modded motorcycles, personalized beatboxes, and the like?
Then there's:
> But the truth is that living in V.R. makes no sense. Life within a construction is life without a frontier. It is closed, calculated, and pointless. Reality, real reality, the mysterious physical stuff, is open, unknown, and beyond us; we must not lose it.
The average person in the developed world lives in a meticulously designed world. They ride some form of manmade transit or hop into some form of human constructed personal vehicle and go along infrastructure meticulously planned and designed to go to a destination whose very form has been prescribed by pages of municipal code and undergirded by large sums of public monies. They have access to sanitation systems, emergency services, and more. The average person does not spend time on some open frontier pondering their "real reality".
In general the article seems... like a weird, romantic, pastoral polemic steeped in the sort of romanticism common to older folks waxing about how the physical world is "real" while the virtual world is "fake."
So I think you can read him as simply vouching for implementing certain technologies in a certain way that would make him happy, and frankly you don't need much rigour for that. He's really just saying that he wants some prototypical human experience to remain the most valuable, and if you also want that, here's how you'd want to use VR. As for why that kind of experience is or should be valued, you won't get that from him beyond a bit of emotional handwaving, because it's just not what he's setting out to do.
I always roll my eyes when I see this or things that share a similar sentiment. I have dreams every night that aren't "real", but produce real emotions, feelings, and sensations. Whenever the dream machine comes out, I'll be hanging out in my cool af castle and all you philosophers can keep kneecapping yourselves with "what's really real".
Dreams are another beast entirely, you're fabricating and perceiving the contents at once, like you say, with all the senses.
VR takes advantage of our vision being so prevalent among our senses, but the fact that only it and sound are involved is what makes it fake.
One of the things I will never stop finding weird, is the tendency to group together things that aren't really related.
I wonder if I do it too? If I did, I expect I wouldn't notice…
> I wonder if I do it too? If I did, I expect I wouldn't notice…
A very good point! You might not. But grouping things together that don’t immediately seem related can be one of the most fruitful acts of all…
‘Mathematics is the art of giving the same name to different things’ - Henri Poincaré.
Really? I can see that for GenAI and Crypto (to my mild annoyance because I was doing GenAI before such people heard of Bitcoin), but not AR/VR.
> I think you’re just objecting on the basis that AI is obviously already immensely useful while other hyped technologies (in their current state) aren’t.
While I do indeed think AI is valuable, I also look at AR/VR and think "this is just another games console — some are handheld, some connect to your TV, this one is on your face". VR just… it just doesn't seem any more hyped than any of the other games consoles over the years that were also "revolutionary" and "amazing photorealistic graphics" etc., and mostly by different people?
Actually, you’re right. I haven’t seen AR/VR hyped in quite the same way as the other two. I’m not actually sure why… because it seems just as ripe for hyperbolic claims as any other ‘futuristic tech’. Possibly because the idea’s been around for much longer and has had a chance to burn out before it came close to being practically possible.
Of course, ‘AI’ of the kind popular now is not an entirely new idea either (as you point out), but to the grifters on Twitter it is.
Pro tip: WebGPU provides a common API to all hardware acceleration going forward. From the din of bitcoin miners parked next to hydroelectric plants to the vents in the new Apple Vision Pro to the render farm spitting out hyper-surrealistic midjourney banality to your handset rendering this text; it's all happening via a Web-focused API. You may be asking yourself why Apple would release a half-baked $3500 HMD. This is why. They can balk all they want, but the app store model is eroding and they're looking for cover.