Hacker News Reader

Comment by doorhammer | Hacker News Reader

Something of an aside: I'm most stoked about this kind of thing because of the oculus rift. Even just playing through HL2 with my DK2 is an amazing experience.

What I'm really interested in with things like the Rift is foveated rendering, in order to get much higher quality graphics out of less computing power. Basically, using high precision fast eye-tracking to only render the portion of the screen your looking directly at, since your eye can't resolve detail to any great degree outside of a fairly small area in the center of where you're looking.

Foveated imaging in general: http://en.wikipedia.org/wiki/Foveated_imaging MS Research paper on foveated rendering: http://research.microsoft.com/pubs/176610/foveated_final15.p... MS trial on foveated rendering: http://research.microsoft.com/pubs/176610/userstudy07.pdf

View on HN

For foveation to go undetected, ideally the central region should update within 5 ms. That's unreachable even with top hardware right now.

The MS paper achieved some nice results using a 120 Hz screen and a very fast tracker, but the rift is far from that.

It's interesting, but comes with all kind of issues aside from delay (periphery degradation strategy). I've been trying to adapt shadow mappin to foveation and latency is killing it. Especially since shadows are high contrast information, quality popping and self shadow flicks are so distracting.

Yeah. I'm not involved in any kind of dev work with that sort of thing, so you'd probably know better than I would, but it definitely seems like it's a problem that's very much in the research phase.

I can imagine the popping would be that much more painful in a VR environment. Game stutters that my brain largely tunes out on a normal monitor become world rocking cataclysmic events in the rift.

So, do path-based rendering (which gets the nice, slow pop-in of noise, right?) and bias the sampling to favor the current point of focus.

You naturally end up with fewer samples in places that don't matter, and faster fill-in where they do.

Would dual GPUs help the problem?

You could have one low-resolution GPU (such as an onboard Intel HD) and another focusing solely on rendering the foevated patch.

5ms per 320x240 section might not be implausible?

The problem isn't solely in rendering fast enough--it's end-to-end latency. Keep in mind that if your eye is being tracked by a 120Hz camera, that's 1/120 =~ 8.3ms latency just from inter-frame time alone. Add to that all sorts of communication bus overhead, image processing the eye tracking imagery to figure out eye direction, potential buffering in the display adapter, 5ms is not really possible, yet.

That said, I don't think there's theoretical limitations that prevent extremely low latency systems from happening, it's just not something we've kept in mind when designing modern consumer hardware.

Would you be able to throw hardware at it? You could render in high-detail not just the area that your eye can see well, but also the area around it which it could be pointing at if it moved during your measurement latency. (Can your eye look anywhere in your image within 20ms? I honestly have no idea.)

Yikes, that makes sense. Thank you for spelling it out a bit! :)

Yeah, I'm not an expert in this area, but my casual reading seems to make it out to be similar to the problems you have with a lot of VR stuff, where the hardware just hasn't needed to be ultra-low-latency before now, but that there aren't necessarily hard limits that would prevent it.

If they ever get it ironed out, it seems like the kind of thing that could offer pretty huge advantages for gaming, if just one person is looking at the screen (I have no idea if anyone's considering multiple sets of eyes to track or if that introduces particularly novel problems that wouldn't be covered in solving the single set of eyes)

Reply on news.ycombinator.com