Hacker News Reader

Comment by nightcracker | Hacker News Reader

The authors' argument fundamentally boils down to 'people want cutting-edge graphics and that always involves squeezing the last bit of performance from hardware with complex tricks'.

And while true for now, I think the title is plain wrong in the longer term. There will come a time in which hardware is sufficiently powerful that one no longer needs these tricks to create top graphics. And physically based raytracing absolutely will simplify all of rendering to a couple core components when that time comes.

View on HN

Optic is incredibly complicated, because matter is incredibly complicated.

Ray tracing is good for rendering surfaces, reflection and refraction, but what we see is not limited to these things.

Volumetric stuff is all over the place: fire and other plasma, smoke and other aerosols, other forms of mixed-state matter. Also we have lots of liquid water on our planet, when it interacts with solids things and/or air, the visual complexity of the scene simply explodes. Look at boiling water, or streams of water, or shoreline. And to complicate things even more, some visuals depend on wave nature of the light: soap bubble, rainbow, or DVD disk.

Here’s some 15 years old tech.demo: https://www.youtube.com/watch?v=HsWh66MvqBg

Stuff like the brick wall with these shadows and reflections is awesome fit for RT. The final scene also good. However, close up scenes of the ground with water pouring all over the stuff, or that glass window with drippling water at 1:30-1:45 — RT won’t help a bit. And the rain / water / wet surfaces is about half of their shaders: https://developer.amd.com/wordpress/media/2012/10/ToyShop-Eu...

How long does it take to render a frame from a Disney's newest CGI movie release? 12 hours? So yeah, once the hardware improvements bring down that 12 hours to 16 milliseconds, then we won't need the hacks...

If you take The Mandalorian, which is arguably Disney's hottest ongoing project, then it happens almost entirely in realtime inside the Unreal engine by using their led dot dome screen.

https://www.starwars.com/news/the-mandalorian-stagecraft-fea...

The requirements are different from animated movies or games, though.

Backgrounds are, by definition, not what the viewer is going to focus on. Oftentimes they'll even be out of focus and therefore slightly blurred.

Don't get me wrong, Disney's interactive set technique is revolutionary and shows how far Unreal has come, but I still expect future Pixar movies to take hours of rendering for every single frame (because by now that's basically Pixar's selling point).

There's no way the final rendering is realtime though, right? Post production work is reduced, but not totally eliminated. But the advancements they made with that dome are amazing

From what I understand from the documentary they put out then the the background they render in real time is indeed the final picture.Of course mistakes might still need to be fixed later but the rendering is intended to be final.

> There's no way the final rendering is realtime though, right?

No I think they literally do photograph the LED screen on the stage with the actors and that is the final composite. Just like with traditional back- and front-projection.

I guess it’s graded etc, but so is a completely natural scene.

Disney's latest CGI films are rendered on a much higher resolution and with multiple denoising passes and other settings set to max to hide even the smallest instances of grain. One would get a similar result with much less time and the differences would not be noticeable for 99% of the people, if they were to turn down some settings a bit.

> rendered on a much higher resolution and with multiple denoising passes and other settings set to max to hide even the smallest instances of grain

Is that to let them create something like a (say) 16K archival master that covers all the formats they might want for the foreseeable future without an expensive re-render?

Rendering at a higher resolution than the format target and downsampling to the target provides anti-aliasing. Gives them the "CGI" smoothness.

While that gap between film and games has been large for a long time, it definitely is closing now with the latest generation of film renderers based data on ray tracing hardware, and a large push in the film industry to incorporate real-time pre-vis workflows and reduce iteration times.

It’s very common for film CG frames to take a few hours but not usually 12 hours. You need to be able to start rendering jobs at the end of the day and have them ready for dailies on the morning. Jobs that take more than 4 hours risk clogging the queue and taking 2 days per iteration rather than 1, so people are motivated to keep it reasonable.

Another important difference between games and film is antialiasing techniques. Much of the difference in render time between them is in using fancier texture sampling and using lots more samples than games are willing to use or even need for decent quality.

Actually people don’t expect realistic simularion. They expect better. I love the original Lion King, but the movie was just plain boring with its realistic graphics.

Making good entertainment has nothing to do with limitations of the medium. It has do with creativity and talent.

The original lion King worked because it was hand drawn graphics, the story , the dance and expressions on cat faces worked because graphics was limited.

The new one was a dull imitation with no understanding of what made it successful.

you should checkout the screen rant video on this https://youtu.be/sbx97kKHUHw

Art and games have been successful even if medium is limiting,ppl still read books watch plays and enjoy playing games like FNAF even if far superior tech is available.

Sure, you are right about all of this, I just was explaining that even when we have perfect simulations, people want more.

If the limitations of medium wouldn’t matter, people and studios wouldn’t pay billions of dollars for VFX.

I personally cannot wait; the YouTube channel YMS is doing a 3+ hour review. If it's close in quality to their Kimba video[0] we're in for a treat.

0. https://www.youtube.com/watch?v=G5B1mIfQuo4

I though Lion King was ok.

The new Beauty and Beast movie however was a whole lot worse than the cartoon. Lumiere, Cogsworth, Mrs Potts, etc all felt a lot more human and emotional in the cartoon than with the “realistic” CGI. They actually looked creepy in the new movie.

> 16 milliseconds

Way too slow for VR, try 5 ms just to be sure.

Thankfully few [1] care about VR, and the fad cycle is over for now.

If they get it better and more relevant in the future we can check again!

[1] Few as in "not enough to matter much". There can still be millions of enthusiasts, but the market predictions and "disruption" didn't pan out.

> but the market predictions and "disruption" didn't pan out.

"Disruption" means that you create a less qualitative, but cheaper product that would cannibalize the margins of the product of a company whose products are high-margin (think cutting-edge CPUs of Intel). This "cheap" product will for a long time ridiculed as toy (but nevertheless sell in huge amounts because it is cheap) - until it suddenly "lifts off", becomes an inconvenient competitor, and disrupts the market.

Does this description of disruption sound like any market prediction from the last 10 years how the market for VR glasses will develop? I don't think so.

> Does this description of disruptuon sound like any market prediction from the last 10 years how the market for VR glasses will develop? I don't think so.

That's because the limiting factor on VR hardware value to the consumer isn't resolution, or scene complexity as measured in polygons, or FPS, or any of the other metrics by which desktop games and game hardware are evaluated. It is latency. Latency in estimating and rendering the user's pose, movement, and viewport changes.

We would have had widespread adoption of immersive content by now if VR experience developers would be willing to sacrifice the 'traditional' metrics in favor of a ruthless focus on latency (and ideally push the hardware vendors in that direction too), but instead everyone keeps chasing "AAA graphics in a headset" , which, given that AAA graphics are (as the OP states) an ever-rising bar, can't ever really work except on whatever the current highest-end hardware is, and any experience that breaks people's willing suspension of disbelief on everything but the highest end hardware can't end up being a commercial success, any more than a multiplayer game with noticeable lag on anything under gigabit FTTH could (and for the same reason, except that the lag for VR starts with the single-player version, and only gets worse from there).

>"Disruption" means that you create a less qualitative, but cheaper product that would cannibalize the margins of the product of a company whose products are high-margin

Not exclusively. That's disruption in the sense of "undercutting".

Disruption in the large also means:

"radical change to an existing industry or market due to technological innovation"

E.g. "things will change in how we do IT, entertainment, education etc due to VR".

No sane person thought it would be a total disruption- that's absurd.

But the fact is it brings an entirely new genre of games that can simply cannot be played on a keyboard/mouse/joystick & monitor.

However you seem like someone who enjoys pretending like VR is a little fad and doesn't offer anything new to the game space- so whatever.

>No sane person thought it would be a total disruption- that's absurd.

Whether it's absurd, tons of pundits still did tout it to high heavens, big media outlets, tech startup sites, etc. The usual BS propping up they do, like they did for stuff ranging from fuel cells to grid computing and autonomous cars at different times.

"The VR revolution", "The next big thing", "How VR will disrupt everything", "revolutionize the economy", and so on. This started around 2012 and peaked around 2016 or so.

>However you seem like someone who enjoys pretending like VR is a little fad and doesn't offer anything new to the game space- so whatever.

"Offering something new to the game space" is a spectacularly low bar compared to the oversold promise of consumer adoption and applications of VR.

And even at that, VR thus far didn't even make a real dent in the gaming space, now that you've mentioned it. So much for all the 2012-2017 hype of the tech finally being "nailed".

It's doubly funny to me, because I have memories of the first VR fad, in the 90s. At least then besides the media hype, we've also got a few camp movies out of it ("Lawnmower man" and co).

Backdrops for the Mandalorian are rendered in real time.

They're backdrops though.

They have all kind of motion, detail, etc, though, even persons in the distance.

They just don't have close ups of humans...

16ms is too slow. Gaming has long since moved past 60hz. 120hz to 360hz are the current targets, even the latest generation consoles are pushing 120hz modes.

> There will come a time in which hardware is sufficiently powerful that one no longer needs these tricks to create top graphics.

I deeply wish this were true. But it doesn't seem likely to me.

Games are real-time. And frame rate expectations are getting higher (60, 90, and even 120Hz) which means frame times are getting shorter (16ms, 11ms, 8ms). Rasterization is likely to always be faster than raytracing so the tradeoff will always be there.

Pixar movies look better and better ever year. The more compute they can throw at the problem the better pixels they can render. Their offline rendering only gets more complex and higher quality every year. It's so complex and expensive I'm genuinely afraid that small animation studios simply won't be able to compete soon.

Maybe raytracing will be "fast enough" that most gamedevs can use the default Unity/Unreal raytracer and call it a day. But imho the author is spot on that AAA will continue to deeply invest in highly complex, bespoke solutions to squeeze every last drop and provide competitive advantage.

FWIW i think it's possible. The question is whether it's within our lifetime.

There's an upper limit to what's useful - the real world. If our minds can't comprehend it, you've hit the upper limit of practical use. Once in game graphics become literally indistinguishable from real life then we'll start to plateau in terms of complexity of "tricks" and raw computing power will be able to catch up.

> Once in game graphics become literally indistinguishable from real life then we'll start to plateau in terms of complexity of "tricks" and raw computing power will be able to catch up.

Maybe. Alternatively games won't actually want to render photorealistic and will want to render with varying types of stylized graphics. Is that easier or harder? Probably a little bit of both.

We actually are at a point where we can real-time render photorealistic scenes... for certain types of objects. Primarily static environments. Photogrammetry is basically cheating, but it is highly effective! Mandalorian is famously filmed on a virtual set and it's cool as fuck.

Graphics is moving rapidly into physics. We might be soon be able to render photorealistic scene descriptions. However simulating the virtual world we still have a long, long ways to go. By simulation I mean everything from the environment (ocean, snow, etc) to characters. We most definitely can not synthesize arbitrary virtual humans.

Will we someday see a movie that _looks_ like a live action movie but is completely virtual? Oof. Maybe? But even if we could, would we want to? I'm not sure.

I'm afraid we won't see this limit getting hit in our lifetimes though. The gap is way too many orders of magnitude.

Coming from the VFX field, I think that point is pretty far off still. Even with current gen offline renderers running on renderfarms, we're left doing cost-benefit analysis on rendertime vs. quality issues. (Noise being the big one). Real-time, fully generic, photorealistic rendering is decidedly not here yet, and I seriously doubt it's around the corner, either.

Monitor resolution and frame rate inflation seem to have endpoints at 4K or 8K and 144Hz. Scene detail still has some way to go before diminishing returns and so that will consume years of hardware evolution. I agree with you in principle but I don't think we're within a decade of that yet.

VR/AR will continue past 8K to 16K and beyond. 240+ Hz will be desirable there too. We're going to need to keep squeezing the hardware for all it's worth for the foreseeable future. I doubt there will be "sufficiently powerful" hardware to drive that kind of workload without heroic optimization work within my lifetime.

Nah, the human eye has only ~5 MPix worth of sensors, most of them grayscale at that. We just need inexpensive eye tracking and foveated rendering.

https://www.cambridgeincolour.com/tutorials/cameras-vs-human...

Our retina and occipital cortex undo a lot of rendering (e.g. line detection), therefore, a theory:

It will take such a long time to get realistic VR, that we will have direct neural interfaces that are much simpler because they can skip the encoding/decoding.

\tanget And perhaps at higher semantic neurons, so we can just invoke the concept of "cool graphics" without actually having to simulate it...

But our eyes can move, so while we might not need to render and update XXk@xxxHz we still need the display to have that resolution wherever we look right?

No, because you don’t need to render at full resolution where you’re not looking. That’s the appeal of foveated rendering, you just render say, a tiny 1080p region where you’re currently looking, and render the rest also at 1080p but stretched out over your entire FOV, and with fast enough frame rate and eye tracking, plus a decent blending algorithm, it will be unnoticeable.

I'm sorry if I wasn't explicit enough. What i meant is that we still need the display to have "the 16k" amount of pixels so that we can have the superhigh ppi wherever we choose to look. We indeed would only need to render high quality in a small percentage around where the eye is looking.

My bad, I see what you mean. You're quite right, although there is some research in using MEMS to move the projected foveated region physically, which would remove the need for ultra-high density displays.

Which headsets have you used? I’ve found I don’t move my eyes as much in the oculus quest but it could just be I’m conditioned at this point.

In earlier headsets it was more narrow field of view and maybe that made it feel more obvious when my eyes were moving.

I've used the Quest 2 and i did move my eyes when playing games with it. I don't know of screens with different pixel density in different areas either if we were to assume people always look straight ahead.

I disagee about 240Hz for VR. Who is saying that? Oculus hasn't said it and they know more about practical VR than anybody.

I think after 144Hz, you hit a point of really diminishing returns. Like is it possible in blind test to feel 240Hz vs 144Hz? It might give you in game like 1% advantage for +100% the costs.

FWIW tried testufo [0] with Samsung G7, 240Hz vs 144Hz was very noticeable on image and text scrolling tests. I guess for a person who never seen 240Hz it would be hard to identify it in a blind test, but after you've seen how the images/text looks like in scrolling, it's trivially identifiable.

- [0] testufo.com

But that's the point. If in a blind test you can't tell the difference, then there is very little to no perceptible difference.

If testufo was randomized with multiple repeating patterns and random offsets, it could be a much better test. Here you can accidentally bias yourself.

240 is not evenly divisible by 144. Curious to know if 240 vs 120 Hz is more or less noticeable on a 240 Hz monitor than 240 vs 144 Hz on a 240 Hz monitor.

With games as long as you have good enough hardware to run the game at a steady frame rate you want it being divisible by some magic number does not matter. Also the old "standard" Hz for games is 60hz so going to 120 and then to 240 is the "natural" steps.

The old 144hz goal was/is nice if you want to display 24fps (movies) content but games don't work like that. They display correctly without any judder/pulldown at whatever frame rate you want (as long as you have the hardware to run it at that rate)

> 240 is not evenly divisible by 144.

Sounds like the beginning of a sales pitch for a 720 Hz display. :-)

While they are more entertainment than science, I believe LTT performed a few tests like on this topic:

- https://www.youtube.com/watch?v=tV8P6T5tTYs - https://www.youtube.com/watch?v=OX31kZbAXsA

As someone who owns a 240hz monitor, absolutely.

And here's how I can probably immediately convince you that I can as well: drag your cursor at a rapid speed across your screen back and forth (e.g. 2-3 times across the width per second). Focusing on one spot, do you see individual cursors with gaps of background in between? (I'm assuming yes.)

Now on 120 Hz the gaps will be half as big as on 60 Hz. On 240 Hz they are half as big once again, but they are nevertheless noticeable. The size of the gaps easily allows you to distinguish between 120 and 240 Hz.

Until I can drag my cursor across the screen and see nothing but a continuous moving object, we haven't hit diminishing returns on monitor refresh rate yet.

Just because there's some measurable difference doesn't mean it has significance.

> Monitor resolution and frame rate inflation seem to have endpoints at 4K or 8K and 144Hz.

laughs in light fields

I seen many indie games made with Unreal Engine or Unity that have terrible performance in rendering a simple level where a AAA game like Tomb Raider will use less resources and have 10x more details. So when the hardware gets so popular that indie developers can deliver similar graphics as AAA games today the AAA games will still do it 10x better by having engineers optimizing things.

It's not the engine's fault for having terrible performance on simple levels. Most indie devs don't bother with LODs, occlusion culling, and turning off unused features. Decades old tricks like turning off unseen/far away actors, merging mesh and material to bring down drawcalls is also alien in the minds of asset flicks. Both engine provides many monitoring tools and utilities to fine-tune performance and settings on each quality level...

Correct, sorry I was not clear and it appeared I blame the engine , the indie devs are also not to be shamed because they did not done the optimizations, they most of the time focus on the content.

My point was that even if hardware gets 100x better we still have a difference between a AAA game that can render a giant open world and a AA game that will use smaller worlds like a space ships with loading screens for each room.

One game that was recently released has shown that most games are simply unoptimized and that is Doom Eternal. It performs substantially better compared to any other game that was released in 2020 at the same resolutions even on old GPUs.

I don't think so, more rays will equal better graphics for a veeery long time. The Rendering Equation [0] is basically an infinite dimensional recursive integral, so computing it will be expensive. Today's Nvidia RTX hardware doesn't even come close to computing it properly, it can still only do bad approximations.

[0] https://en.wikipedia.org/wiki/Rendering_equation

In a different context, look at text rendering. It should be the opposite of the spectrum and stabilized for a long time, except we keep adding better rendering tricks and simplifications (not at the scale of 3d rendering but keep in mind how simple the problem looks compared to it)

I think it's the opposite.

Text rendering used to involve complicated hinting and then subpixel rendering.

These days on Retina screens you don't need hinting or subpixel rendering.

You just render it like any set of curves, no longer any tricks about it being a font, because it's hi-res enough.

What rendering tricks are you suggesting are being added rather than being taken away, for fonts?

Thats what I meant, were still simplifying things in 2020 for text rendering. Antialias and Cleartype are still used so it's not a settled technique, the environment is still changing.

Well, there are signed distance field fonts for one thing.

I honestly wouldn't be surprised if Moore's Law dies off before we get to a point where a heavily optimized graphics pipeline isn't worth it.

I think the author makes a good point at the high end, but Raytracing is going to simplify the crap out of whatever we have that's like Unity in a decade or two (hopefully not Unity).

So many issues like transparency sorting, AO & GI, depth of field, baked and real time shadows and reflections you basically don't have to deal with the complexities of as an end user.

But you'll be leaving a lot of performance on the table with a one size fits all solution.

Moore’s law has ended quite a few years ago - or do you mean it regarding GPUs because I don’t know about that.

But even though parallelism can be increased, the gains from them are limited as per Amdahl.

Moore's law is about transistor density and has not ended. See the performance ramp of recent model GPUs.

Amdahl basically does not apply to graphics tasks which are embarrassingly parallel. Raytracing will be no different.

And in fact you're getting rid of many preparation steps pre render that may be tricky to parallelize well, and replacing it with brute force.

I don't believe that's going to happen in the foreseeable future. Just like in film CGI the hardware is always going to continue to chase the best images content creators would like to create.

Hybrid render pipelines and AI enhanced RT passes is the future.

Reply on news.ycombinator.com