Wow, is Apple’s Vision Pro loaded with pixels
spectrum.ieee.org
spectrum.ieee.org
The Vision Pro has 23 million pixels total, or 11.5 million per eye, or 3,391 x 3,391 if square.
The Quest 2, by comparison, is 1,920 x 1,832 per eye, or 7 million pixels total across both.
So the Vision Pro has 3 times the total pixels of the Quest 2, but just 1.81 times the number of pixels along a single dimension (sqrt(23)/sqrt(7)). And the article quotes people saying that the field of view is comparable to other headsets.
The Quest 2 has a PPD of 18.88 horizontal and 18.69 vertical. The article also mentions the Quest Pro as having 22 PPD.
But this article is claiming that by their calculations, the Vision Pro has "50 to 70 PPD", which is supposedly in the ballpark of the PPD of the fovea which it says is around 60.
If the Quest 2 is 18.8, and the Vision Pro has 1.81 as many pixels in a dimension, then the PPD of the Vision Pro should be around 34. Nowhere near "50 to 70".
Did the authors forget that PPD is calculated along one dimension only, rather than two?
I mean, the resolution is a great improvement, but it's not "retina-level" yet. It's halfway there.
I think it would have to be something like immersive AR glasses on the order of glasses for simple corrective lenses to have the level of comfort necessary to not only want to use it, but use it long enough to actually be seeing a productivity increase.
You'll be fatigue from having a screen on your face within an hr.
Seems like that would be the case otherwise it would cause insane stress guess apple isn't dumb.
What’s hard to fathom is just good eye tracking can be on these systems and how many hacks such tracking enables. They can for example spend extra rendering time on the pixels in the center of your vision and update this fast enough that you don’t notice the difference. Similarly using adaptive focal control the screen can always be in focus no matter what depth you focus on.
I guess that's easier for vr than ar and may tend to exaggerate any issues in the depth mapping.
In terms of VR it depends on how well you can pull off the blur based on eye tracking vs how much control you have over focal lengths. A perfect version of either is probably equivalent, but doing both reasonably well is probably easier than taking just one close to perfection.
Direct social interactions seem really... creepy. It somehow appears rude, like looking at your phone while speaking or pointing a smartphone in someone's direction while potentially filming them.
But if this device enables WFH in the garden/nature where it's too bright for laptops or installing a desktop would be impractical, now that sounds great!
This is better than a Covid mask in that even Version 1 lets people see “behind the mask”, kind of. By version 14 it’ll be so completely natural that it won’t occur to anybody on HN to even comment on this aspect of it.
I would like an AR device that acts as a monitor for my Macbook or phone, isn't so heavy that it pinches my nose or strains my neck, isn't pixely, has a wide field of view with low-to-no distortion, and overall doesn't feel like a McDonald's Happy Meal toy.
You would have to make your virtual monitors pretty big in order to have a lot of comfortably readable content and that sounds like an ergonomics nightmare to be moving your neck around all day looking all around your virtual screens.
> “The resolution of the fovea, the highest resolution portion of the eye, is considered to be 60 pixels per degree. And if you have a display like 60 pixels per degree, probably like 99.9 percent of people wouldn’t perceive the pixels”
As a comparison, Quest 2 has a resolution of 20 PPD and sitting in front of a 27" 4K monitor on a desk leads to about 70 PPD.
From what I've seen the common advice for monitors is that 5k is the ideal resolution for a 27" screen and 4k is a little bit less sharp if you're looking closely.
But you also have lenses that are stretching out the screen to cover your periphery where it can be much less sharp so it's not exactly comparable.
Even if it turns out that 60 PPD is slightly too low, and some people perceive pixels, it doesn't take a very high rate of improvement to reach 90-100 PPD in under a decade, and at that point we as humanity will always be able to enjoy basically perfect no-perceived-pixels VR and AR, which is absolutely amazing.
I would certainly like to have a Double robot streaming 8K video to my eyes from other countries. If someone could add smell sensors and smell reproduction, it would almost be like being there. And something like Starlink might eventually make it possible to do the 8K streaming from absolutely anywhere, regardless of local infrastructure, although limiting it to certain wifi-enabled zones would work meanwhile.
Now that I think about it, I actually have a business idea to take that even further, but unfortunately it won't make sense for another 10-20 years until this technology becomes more widely used.
everyone that worked on these for however long it took just reached for their virtual pitch forks and are looking your way
With any luck whatever 3D picture/video format is open and includes roving geolocation, compass direction, inclination, smells, humidity, wind speed/direction, light sources, humidity and more. Saving more metadata now should encourage technology to take advantage of it later.
The complete lack of cellular/satellite announcement is not too surprising as Apple computers and CarPlay also suffer the same lack. Apple pinches pennies in the weirdest places.
There is an opportunity for widespread Vision Pro adoption even at this price point so perhaps get started on the business idea. As a display alone there is high success potential as long as Apple does not feel the need to knee-cap the future of Vision Pro by abandoning the shared display market to others. It has nearly the resolution of a Studio display ($1599 at 14.5 million px vs $3500 at 11.5 million px/eye) and can emulate multiple monitors at once. Apple will want to control the experience, but personally I am hoping they quickly jump to WiFi 7 and become the central home hub for multiple PCs, consoles, and headless/clamshell Macs with multiple desktops each.
Desktop PCs and Phones are hardly social to start with so I think people complaining about isolation with AR are overstating their case. I expect most will just take this off like putting down the phone for conversation. Apple has an opportunity to create unique environments, increase personal privacy for all screen based devices, and champion compatibility if they cut back on ego a bit. Here is to hoping they open source some sort of easy pairing/streaming/VR protocol that limits frames/resolution based on attention/distance and the console makers adopt it with Find My in every controller and everyone makes a bundle doing it. One can dream.
Eventual Thunderbolt ports for a cellular dongle/fast media backup/head mounted lamp/charging controllers while playing/etc would also be nice if they don't add flashlights and cellular options to future models would be my main current request.
The first AirPods were great, yet the original Apple Watch and iPhone were bare bones so I will keep my expectations low. Here is to hoping Apple grabs every opportunity as quickly as possible to increase adoption as the competition will be fierce. As long as they are not too patient with the long game I think even five years should have a good market with a lighter plastic lens no front screen non-pro version for the casual.
Apple had a years-long lawsuit with Qualcomm which, to be extremely reductive, had as its main goal “stop charging us a licensing fee that’s a percentage of the final sale price of the device, you should charge a fixed price per chip”.
Apple essentially gave up when they realized Intel was years away from making good enough modems, bought Intel’s modem group, which has been chugging away since then making still-not-good-enough modems.
> The [Virtual Boy] display consists of two 2-bit (four shade) monochrome red screens of 384×224 pixels
That would be 127K pixels for the Virtual Boy.
By analogy, if you resize a 1025x1025 image to 1024x1024, it's usually going to look bad.
import math
distance_to_screen_inches = 24
pixels_per_inch = 254
ppd = 2 * distance_to_screen_inches * math.tan(math.radians(0.5)) * pixels_per_inch
print(ppd)Crazy stuff.
It's a 1.0, and it's state of the art. We won't have retina-level AR/VR for a few more years yet, and a few more years after that for it to become affordable in consumer devices.
Anyone here work on that stuff and know how it is done in practice? Is it just things like AA and post-processing that are cranked up in the foveated region, or is it also using simpler meshes and whatnot? The former I can easily believe, the latter sounds really hardcore
(brains "drops frames" while the eyes move, so "popping" during eye movement probably won't be perceptible)
The quest does this but without eye tracking, whatever you’re facing toward is higher detail than stuff toward the edges.
Rendering full resolution at high frame rates for 23 million pixels would be too much for an M2 gpu, but most of your retina can’t see that fine detail so you don’t need to.
The one rendering related thing that I'd guess can be offloaded to that is warping the passthrough camera frames into whatever form they need for correct optical alignment.
It reminds me of the "Neural SubNet" from Deus Ex.
So more VR than AR? Not walking around with magic fashionable glasses that tell you things? I can imagine interesting aspects of AR but I'm not sure this gets to the starting line. (Other than maybe a developer platform)
And I agree - laptop on a plane wrecks necks.
On my first trip to most antique Corinth, someone noticed the fine Casio timepiece from my last (Reagan-era -- ca. AUC 2750s, I think -- or do I reckon now from the founding of Christ?) trip. When the watch was seen to "move", I explained the function of the band by analogy to a water clock (a clay vessel with a small aperture near the bottom to let water drain) and compared the water to the (non-visible) battery inside the Casio. I even went into elaborate explanation about the special symbols for numbers which mark the hour and how they are distinct from the symbols for making sounds for speech (e.g. "sure, '1' is rather like 'alpha'...").
Sadly, it was a very clever Greek to whom I explained all this. Slightly irritating were the questions about why it was on my wrist in the first place -- something something WWI. But my heart sank during the debrief when I came to really understand the time-travel problems which were exploited to near extinction by ancient sci-fi authors.
Happily, those (extremely antique) courts found it so much more cost-effective to cut off an advocate when the water drained from a clay vessel than to invest the wealth of the city-state into the production of a -- dare I say 'Byzantine' -- contraption wrought of too much brass and requiring too many servants to operate?
I guess the real lesson here is that while everyone has been conditioned to be distrustful of the Greeks, my personal experience has been that they are pretty solid folk.
It seems insane that CEOs of the world's biggest companies don't seem to understand the very simple fact that nobody wants to wear a fucking headset all day long.
The difference here from Meta is that Apple isn’t selling a dystopian future where we all spend our time in a meta verse with a “fucking headset all day long” - they are explicitly showing it as in use for certain applications as part of - rather than replacing - reality.
Driving video games.
Driving in VR is 1000% times better than on a 2D screen, you can actually judge breaking distances, and you feel like you're actually going through the corner.
As you allude to there is no killer app for VR after a decade. Maybe the "Apple factor" might bring some more devs and some fresh ideas but at the price point its at you're still at the "the market is too small" for most companies to ivents in.
Apple might also simply release this model Vision Pro as the Vision in a few years when they release the next Vision Pro generation, something like they’ve done in other product lines.
You know, for visionaries. ;)
I feel like I'm reading cryptobros saying "we're still early".
It's not a new market, it has existed for more than 10 years, and it is a micro-niche market.
And I'm saying that as a guy that has a Quest 2.
So were smartphones before the iPhone. Sure, some people had a Blackberry. But now _everyone_ has a smartphone.
The other factor is probably the the high price made everyone realize Apple is really targeting devs and early adopters. They aren't ready to sell one of these to every person who owns an iphone...at least not yet. So regardless of how promising the technology is, they probably won't start making money from it until some point in the future.
AAPL made a new all time high this week and yesterday it set a new all time high closing price.
Selling off 3% after running up to a new ATH prior to WWDC is not a bad reaction, lol.
But worth noting, in an era of higher inflation and stock buybacks, that probably the relevant metric for investor happiness is real enterprise value (that is, discount market cap by net cash, then adjust for inflation).
By that measure they’re pretty far from the pandemic peak.
One use case for me would probably be to wear them around the house (or at the grocery) and expand the text on labels. Being over 50, my presbyopia forces me to constantly have my glasses available. If I used AVP I wouldn't need glasses, right? What about hooking it up to my Tesla so that I can drive with them on and not have to look down to see my current speed? Talk about a HUD!
Or what about expanding our natural capabilities? FLIR cameras are getting pretty reasonable; could there be an add-on for detecting heat patterns? Or what about visualizing ultrasonic waves; like in "The Dark Knight"?
I'm not intimately familiar with the precise technical details of the Quest system, but I do know that real time photogrammetry even more sophisticated than depth maps can be achieved on mobile chipsets - google LSD-SLAM.
When I can't see pixels, it sure seems to me that I've reached the limit. Antialiasing isn't blurriness once you can no longer distinguish pixels, it's just how vision works. The "blurriness" of the source matches the blurriness of the eye so any additional sharpness is a waste.
Can you expand this idea? I thought 4K became popular because it was easy to see pixels at lower resolutions.
It may be the highest resolution micro-OLED on the market, but micro-OLED [1] could do quite a bit higher than that. Well above 4000 PPI.
And I know some people are pissed with the pricing. But I would not be surprised if this micro OLED cost $300 per piece. Or $600 in total. And I believe even the $300 is a very conservative number it could be much higher.
[1] God I hate this term as if they intentionally name is to distract it from microLED. It used to be called Si OLED or Silicon OLED.
They're claimed to be the size of blood cells.
Wow, the resolution + lens solution on the Quest Pro was already really close to workable for streaming your computer screen. I imagine it’s a solved problem at 60 PPD.
It's passable, but not all day usable. It feels closer to working on a 720p projector than a virtual screen.
I suspect the turning point will be around 100 ppd.
I’m sure there are people at Apple who are terribly unhappy with the Vision Pro’s external battery pack connected by a cable and port as well as the ability to connect to power outside (I recall there was some mention of external power connectivity for usage longer than the two hours the battery pack provides). This is one of the first things Apple will get rid of by the time the third generation is released.
If we assume about a 100 degree FOV, and you are using the headset to watch a big virtual movie screen at 36 degree FOV (typical THX).
Then if it's 3391 pixels wide, that only gives you a virtual movie screen resolution of around 1220 wide, or approximately 720p.
So from a technical point of view, this isn't really that groundbreaking.
Then, just like an oled has transitors along its rows and pixels for timing and signaling, so too does the CMOS have a 'rolling shutter' effect caused by the sensor having it's pixels sampled row by row, column by column, just like an oled display draws its pixels.
Here's a few sources in case you'd like to learn more, but clearly if a CMOS sensor of a given resolution is constructable, the only thing preventing its OLED counterpart is the actual organic chemistry driving the pixels, which is a large reason OLEDs are better for fine pitch displays like this than LCDs, they can be made this small.
https://isl.stanford.edu/~abbas/ee392b/lect04.pdf https://isl.stanford.edu/~abbas/ee392b/lect05.pdf
I couldn't find perfectly similar cross sections of an OLED, and the wiring will of course vary a bit, but not enough to blow past the limitations of CMOS's precision lithography process.
https://www.researchgate.net/figure/Cross-section-of-the-pix...
And to give you an idea of how far that can go, Sony has a 61MP sensor that's roughly the same physical size as these displays, but basically 5-6 times as dense with photodiode based pixels instead of oled based ones. Maybe we can't make a 61MP display that size yet, but we can clearly make a 4k one (which is about 12MP, I think...)
In theory you could crudely rig it up with a set of Index controllers, if you can stomach adding another $580 of gear (controllers + two lighthouses) to your $3500 headset.
The lady in the presentation just used a regular game pad.
I don't know why people keep thinking that the only way to use this thing is with hands, when the presentation showed several different input devices, including Bluetooth keyboard and trackpad.
The industry already went through this phase, the very first Oculus Rift came with a regular Xbox controller and that idea was quickly abandoned. The Quest has hand tracking but it's treated as secondary to the set of controllers it comes with.
You need individual hand motion controllers at a minimum.
Look at all the other VR headsets out there. Look at the Valve Index controllers for example.
They also announced official support in the SDK for CrossOver (Wine). [0] It could be part of a strategy to support existing VR apps for Windows.
> the Vision Pro packs a pair of 1.41-inch micro-OLED displays that combine an OLED frontplane from Sony with a silicon backplane from chip foundry TSMC