Karl Guttag on Apple Vision Pro (Part 1)
kguttag.com
kguttag.com
As a Meta Quest 2 owner, and Android user, I'm a bit of an Apple-skeptic when it comes to their pricing on some products; but I have to acknowledge that while the Vision Pro seems mad-expensive on first reveal, it doesn't seem to be poor value going by the previews and specs, which seems to match the article's conclusions.
That is where the Quest Pro seems to have failed - at least at its initial pricing it was very difficult to see the added value matching the price premium.
Apple - at several times the price of the Quest Pro here - has me rooting for it as a considerably more serious product at a relatively-justified serious price in what is an uncertain undertaking.
I doubt we'll see such freedom on the Apple headset as it looks more of a "VR iPad" than a "VR iMac". If I know Apple well, sideloading apps will be a huge no-no and you'll only be able to run what Apple approves on it, so those hoping for immersive VR porn or other wild fantasies, might be left hanging.
As always, I'm waiting to see the final implementation before rejoicing prematurely.
Appel can always intentionally maliciously screw with the implementation of this law in so many ways, in order to make sideloading an unattractive proposition for the majority, but that is still compliant with the law, like they did with the self repair program, or like the implementation of many GDPR cookie banners use dark pattern to annoy you into just giving up and accepting tracking.
A law is one thing. It's the implementation that makes or breaks it.
Meta left sideloading relatively open without a law having to drag them kicking and screaming into it, which is much better IMHO from a hacker perspective, as it means our vision & goals for the product usage are slightly more aligned.
I agree with your implication this is not an appropriate price format for gaming, but I would have thought the Vision Pro is aimed at more of that latter market; though presumably future less premium models will aim at the gaming and more natural consumer markets, or gaming will emerge as a secondary use for the more expensive models like this.
EDIT: See cma's comment below which adds more info: apps that use their own render still support foveated rendering too.
All of this is: Apple taking further control of the experience.
This seems prudent, because as personal as smartphones are, a headset like the Vision Pro dials that up tenfold. Considering what third parties have done with just the information surfaced in mobile operating systems, or heck even the web, I shudder to think of what they'd do with gaze data and a high fidelity color 3D map of your surroundings.
In this particular case I think perhaps a better approach would be to allow apps to bypass App Store restrictions so long as their source code is public and binaries match that code. This would naturally deter those with shady intentions, allow FOSS projects to thrive, allow both manual and automated third party vetting of apps, and help users better know what they're getting into.
(source: "Discover Metal for immersive apps - WWDC23 - Videos - Apple Developer " https://developer.apple.com/videos/play/wwdc2023/10089/)
If they try to hide it by restricting access to reading the foveation texture, you can be sure anti privacy apps like Immersive Facebook will try tricks like recovering it from GPU timestamp queries (render two transparent triangles covering the screen in separate draw calls, find which took the longest, subdivide and repeat).
A restriction for which I am thankful.
>Vision Pro is destined to keep being a toy fully steered by Apple.
I trust Apple a lot more than I trust the "let's figure out how to fingerprint users based on how they scroll and other metrics and use it to sell ads" crowd.
>You can't access eye tracking,
A restriction for which I am thankful.
>so say good bye to foveated rendering.
Vision Pro does foveated rendering.
The Vision Pro does not provide foveated rendering if you do not go through the Apple renderer. There's some extremely vague wording regarding Metal supporting it, without any API documentation out.
Apple isn't telling me that I can't be trusted. Apple is telling me that it's a stupid idea to trust third-party developers with unfettered access to my data, and they're not even a little bit wrong. This has been happening at least since the "Foursquare is uploading and storing every user's contacts" fiasco well over a decade ago.
> Now can the adults use their devices?
Of course. Feel free to bring to market your wonderful headset that allows VC-funded startups to create a never-ending torrent of privacy hellscape diarrhea apps you can enjoy until they implode.
>The Vision Pro does not provide foveated rendering if you do not go through the Apple renderer. There's some extremely vague wording regarding Metal supporting it, without any API documentation out.
That's because the API has yet to be released to developers. Perhaps you can wait until it is to get your mouth all frothy, sunshine.
You do realise you're describing 99% of the iOS app store here right?
In fact, while the Android app store is curated in an even worse manner, I find that Android does offer a lot more high-quality ad-free open source software on platforms like F-Droid. Because the iOS developer subscription makes it hard for FOSS developers to do their work for free so things become more monetised.
I do. And I also realize how much worse it'd be if Apple didn't gate user data behind OS-level user consent dialogs.
It's third party devs doing that, not Apple. They've proven themselves unworthy of trust more times than can be counted. Where there is room for abuse, abuse will happen. We might be numb to it, but it happens all the time on desktop operating systems. Examples include Adobe putting files in corners of the system it has no business touching, practically every modern AAA game installing rootkits and scanning our filesystems, and until just a few years ago Dropbox installing a kernel extension on macOS.
The only devs that deserve any trust are FOSS devs but there isn't good FOSS software for every use case.
Is there any headset that does allow this currently? Doesnt seem like any kind of differentiating factor
The Vision Pro is pretty much the only one that doesn't, because Apple thinks of their users as children.
I think the bigger factor is that Apple is trying to push this as more of a workplace productivity machine than the iPad, so third party developers will hopefully be more amenable to porting those types of apps than they were to iPadOS.
I'm sorry but that's exactly the problem, I want the app natively (well, as native as you can get with Electron), executing code that I have on my file system. I don't want to execute remotely through a browser.
I don’t work for Google but I often SSH to my desktop to run my code. Why heat up my lap or drain my battery?
> Apple either doesn’t seem to be able to provide it, or didn’t prioritize it.
Sure, that's up to them. I can still want what I want however.
Productivity !== coding. Coding is a subset of productivity, but most jobs do not require coding.
I get that you want this to be something else, but a lot of people are incredibly productive on iPads today, and adding the ability to mirror your MacBooks screen means that this platform absolutely has a future in productivity.
HN has a lot of this attitude — “I can’t code directly on it so this is a toy” — and it’s always a laugh for me. There’s … other people? Who want to do other things?
Good thing Apple isn't marketing it as a replacement for a MacBook then.
Hardware company entices consumers to buy more hardware. Film at 11.
>If it were a Surface-like product instead, with full macOS, I'd have bought an iPad in a heartbeat.
Ah yes. The Surface, the device that showcases the appeal of running an OS designed for mouse/keyboard input with a thin veneer of touch support spackled on top. It's amazing Apple isn't sprinting to release a product along those lines.
Sure, they can do whatever they want, of course. Similarly, I can also call it a shame.
> The Surface, the device that showcases the appeal of running an OS designed for mouse/keyboard input with a thin veneer of touch support spackled on top. It's amazing Apple isn't sprinting to release a product along those lines.
You're right, it works great as a portable computer that I can run anything I want on. I'm sure Apple could make an even better version if, I don't know, they created a keyboard attachment for one of their devices and made that device's OS more mouse and keyboard friendly, while still retaining the ability to run whatever software the user wants. Ah, one can dream.
Most others just have to put up with Windows whether they like it or not (working in IT for a company with 120.000 PCs and 600 Macs). And yes most companies are like us.
"The same chip inside" doesn't mean that much though. This one pushes a LOT more pixels at really high framerates, has to handle realtime input from a ton of cameras and do live algorithms on their data etc. I think the whole realtime usecase on a more open OS like the Mac? Very hard to make that work.
So I see where they're coming from, sure. But productivity wise, iOS apps are not great. Needing a Mac to connect to isn't either considering the price of each.
Apple still thinks people can use an iPad for all their work though my experience at work is very different (and I managed a large fleet of mobiles)
You thought they meant actual apps that professional people use, like VSCode?
Guttag's section about the lenses not being pancake and being refractive only I think is wrong, Apple says: "combined with custom catadioptric lenses," which implies reflection is is used like with pancake.
(source: https://www.apple.com/newsroom/2023/06/introducing-apple-vis... )
> It should be noted that the AVP displays about 3.3 times the pixels, has more and higher resolution cameras, and supports much higher resolution passthrough. Thus the AVP is moving massively more data which also consumes power.
An earlier Sony presentation on their 4K microdisplays from last year I believe showed they had some form of foveated scan out on the microdisplay itself, letting the display controller move less data.
(source: https://www.youtube.com/watch?v=IVUSUdzsNgY )
>Apple discussed (linked in WWDC 2023 video) how they implemented features in watchOS to encourage people to go outside and stop looking at screens, as it may be a cause of myopia. I’m not the only one to catch this seeming contradiction in messaging.
Isn't this less of an issue with displays at a farther distance (whether physically or optically)?
> I was doubtful based on what was rumored that Apple would address VAC. Like many others, Apple appears to have ignored the well-known and well-documented human mechanical and visual problem with VR/MR.
There is less vergence accommodation conflict as you age. You slowly lose elasticity of your natural lens as you age and need reading glasses to see up close and the effect isn't as big: you're at a wrong farther focus when your eyes are converging anyway. It might actually be an improvement in vision for people over 45-50.
I thought it was very interesting and unique, I have no idea what the optical design could look like. Maybe something like the perkins-elmer solid cat design, shrunk down for a long focal length but compact pancake lens.
The contradiction being introducing a new product with a screen? Guess what, watchOS will still suggest you go outside, whether you use a MacBook or a Vision Pro.
This analysis is not particularly sharp.
The vergence issue he is very hot about, but like anything in this space the question is “is this good enough for a lot of people” not “did they solve this to my personal satisfaction (having not used the device)”
This is fascinating to me since I am getting vision corrective surgery soon, where could I learn more about this?
Ok. I'm fine with being told "you aren't the target audience" but... name 5 things I would do with it if I bought one.
Why would I put it on my face/what would I gain from it?
Gaming? Productivity? What can it do that a screen + monitor + mouse + keyboard / iPad / iPhone can not?
A wearable that acts as if i have floating monitors anywhere sounds like a massive win, in a device that appears to be far more portable than actually bringing the aforementioned accoutrements with me everywhere i go.
From the perspective of “it can give you an experience as if you have multiple large displays” and costs the same as ~2 Apple displays, plus comes with compute built in.. it’s actually a steal.
I wouldn't feel comfortable wearing this in public around strangers whatsoever.
Similarly, i wouldn't wear it at a coffee shop. But i could see it coming into style at co-working spaces. It feels like a no brainer in hotel rooms and airbnb's.
Vision Pro presents theoretically unlimited size and number of screens to do categorically the same kind of things you'd do on an iPad/iPhone/Mac. As someone who likes to work with multiple monitors, at the least I see this enabling me to take my work on the go (coffee shop, library, plane) without significantly compromising my screen real estate.
Give me one thing that those goggles let me do or do better right now.
I think it's a lot more compelling than that, personally, but you didn't set the bar very high.
Given that an iPad Pro with max specs is about 2/3 of the announced price of the goggles, I think the goggles are in fact at an advantage.
2d screens can't really do 3d, and definitely can't put things in the room around you.
People have already dreamed up all sorts of game ideas for passthrough AR, as well as CAD and 3d design (think autocad, blender etc.).
If you're using it as a stand in for a 2d display it may not be worth it -- it's just bigger in your field vision.
The question in my mind is if I want the thing strapped to my face for an 8 hour work day. If so then (while still too expensive for my blood) it's not crazy; it just needs a couple of years in the oven, if not then it's not a couple of years it's a decade or more.
Assuming that's true, I imagine this would have similar effects.
In fact if those optics weren't there you wouldn't see anything except a few coloured blobs.
The amount of brightness your eyes are ingesting is also much lower than walking around on a sunny day without sunglasses. Remember our eyes work logarithmically so the outside could be tens, hundreds of times as bright.
I've never heard of this actually causing harm though, except some questions were raised about kids whose eyes are still developing. It would be similar to looking through a microscope or binoculars for hours, which is not unheard of in human history (e.g. researchers, lookouts on ships)
Even as someone steeped in this space, the fact of the 3-element AVP was news to me. That is a pretty big difference from other designs and tells me they are reaching for anything they can to squeeze optimizations out.
AR is really really hard.
I don't know about the ideal use cases, but it certainly appears that the SDK and dev environment are going to be the best and most accessible. So if a great use case does appear, it will probably be on the AVP.
Fundamentally though I see AR as how we give our AI personal assistants our eyes ears and real life action space evaluator.
Everything from highlighting objects you’re searching for to giving instant information about anything you look at that you want to know about.
It will also give you perfect recall
Hey siri where did I leave that toilet brush two weeks ago? How many pages of [Book] did I read on average last week? What was that pattern that [person] wearing when we were at the farmers market last week?
It was known for at least 1.5 years https://www.macrumors.com/2022/01/05/kuo-apple-headset-panca...
I remember also seeing some shipment records from Apple ordering these lenses last year.
1: https://en.m.wikipedia.org/wiki/Vergence-accommodation_confl...
2: https://www.aaojournal.org/article/s0161-6420(16)00025-7/ful...
https://en.wikipedia.org/wiki/Vergence-accommodation_conflic...
The thing to realize is there's two different kinds of focusing: the first is the one everyone thinks of which is how your two eyes work together. They point together when focusing close, and spread apart when focusing far. But separately each eye also can focus individually. Try covering one eye and holding up a finger at arms' length for your other eye to focus on. Then switch to focusing on the wall with that eye, and back. You'll feel your eye muscle making the adjustment.
In the real world these two kinds of focuses normally operate in sync. If you want to look far, you'll normally both point your eyes further apart and adjust the individual eye focus for distance, and when you want to look close, you'll point your eyes kind of together and adjust each eye for near focus.
But in VR, your "focal distance" is fixed at the generally comfortable ~2m I mentioned. However, because it's a 3D world you can look near and far, and in that case you'll change how your eyes point but not the other kind of focus.
So I'm not sure to what extent that causes problems, if any, but it's different from staring at something a few centimeters away.
Introduction:
Lack of technical analysis and understanding of VR/AR issues in AVP reviews. Neglecting issues of variable focusing and Vengeance Accommodation Conflict (VAC). Meta to present a paper on VR headset with retinal resolution varifocal display. Apple and Meta's inability to solve inherent human visual system limitations.
Advantages of AVP over MQP:
MQP considered poorly executed and a bridge to nowhere. MQP's passthrough mode criticized for low resolution and distortion. AVP has higher-resolution cameras, more depth-sensing cameras/sensors, and improved processing for passthrough. AVP's Micro-OLED provides better black levels/contrast than MQP's LCD with mini-LED local dimming. AVP's micro-lens array enhances light collection efficiency.
Price Perspective:
Comments on AVP's price being too high lack historical perspective. Apple and Meta need to prove the viability of a highly useful MR passthrough headset. AVP's price is subject to reduction with increased volume.
Resolution for "Business Applications":
MQP has lower pixel density, resulting in less readable text and slower reading speed. AVP's higher pixel count provides around 40 pixels per degree (ppd) for improved text readability. Long-term use of virtual desktops lacks evidence of being good for humans.
Passthrough Capabilities:
MQP's passthrough mode criticized for its poor execution. AVP's passthrough, although not perfect, is significantly better than MQP's. The accuracy of AVP's 3D views of the real world is unknown.
AVP Micro-OLED vs. MQP's LCD with Mini-LED Local Dimming:
AVP's Micro-OLED offers better black levels and contrast. MQP's local dimming feature has limitations and issues. Apple's Micro-OLED CMOS backplane is assembled by Sony, known for Micro-OLED expertise.
Pancake (MQP) versus Aspherical Refractive Optics (AVP):
AVP incorporates custom catadioptric lenses for sharpness and clarity. Apple's acquisition of Limbak, an optics design company, suggests the use of pancake or similar optics. MQP uses 3-element aspherical optics. Pancake optics are less efficient with Micro-OLED displays, requiring higher OLED power output.
Conclusion:
AVP reviews lack technical analysis and overlook critical issues. AVP offers advantages over MQP in passthrough, resolution, and display technology. The price of AVP needs to be viewed in the context of technological advancement. AVP's Micro-OLED and catadioptric lenses contribute to improved performance. Long-term suitability of AVP for business applications and human visual comfort remains uncertain.
Neither really do product reviews as their main gig either.
Apple loves keeping the "magic" on their products as long as they can. Once they are in consumers hands they can't stop teardowns and detailed comparisons but right now they won't want some wise guy ascertaining the FoV, PPD, distortion figures etc.
They want their marketing to be viewed as a class of their own, without direct specs to compare to their competitors. Once you bring it down to technical fact territory, that "magic" they're going for is lost.
Right now they're only showing it to their most loyal followers that are bound to go "wow" but don't pick up too many technical details.