Hacker News Reader

Comment by pjs_ | Hacker News Reader

I know people are very accustomed to miracles these days but I still find the fact that these guys are comfortably doing foveated rendering kind of crazy.

Anyone here work on that stuff and know how it is done in practice? Is it just things like AA and post-processing that are cranked up in the foveated region, or is it also using simpler meshes and whatnot? The former I can easily believe, the latter sounds really hardcore

View on HN

I assume the real challenge here is preventing some kind of "pop" when a pixel leaves or enters the eye's area of focus. Any kind of change in the periphery of your vision would be distracting.

Saccadic masking probably comes into play: https://en.wikipedia.org/wiki/Saccadic_masking

(brains "drops frames" while the eyes move, so "popping" during eye movement probably won't be perceptible)

It’s probably rendered at a lower resolution outside of the focused area, so you don’t have fine detail, text becomes blurry, etc.

The quest does this but without eye tracking, whatever you’re facing toward is higher detail than stuff toward the edges.

Rendering full resolution at high frame rates for 23 million pixels would be too much for an M2 gpu, but most of your retina can’t see that fine detail so you don’t need to.

The question is how much of rendering actually happens in M2 gpu, and how much is delegated to R1.

From how they described it I don't think the R1 does any rendering, it's for handling input from a dozen cameras and other sensors within 12 ms.

The one rendering related thing that I'd guess can be offloaded to that is warping the passthrough camera frames into whatever form they need for correct optical alignment.

Came here to write the same thing. How have Apple pulled this off? I remember watching one of Meta's researchers describe the extreme challenges, such as the fovea wobbling as it moves. They made it out to be unachievable any time soon, and yet here it is.

Reply on news.ycombinator.com