Hacker News Reader

Comment by zokier | Hacker News Reader

It is baffling just how well RGB rendering works despite it being such a huge hack; many real-world scenes are practically indistinguishable between rgb and spectral rendering. But still I find the physically based rendering pretty disingenuous considering the huge gap still remaining between real-world and standard pbr although I accept that it is just a domain-specific term that shouldn't be taken too literally.

Just to spark discussion, here is one example of research aiming to reduce that gap and bring more physicality: https://ssteinberg.xyz/2023/03/27/rtplt/

View on HN

Is it that baffling? After all, the core principle is that instead of representing a continuous spectra, you project the spectra to a few basis functions[1], so you can later reconstruct an approximation of the original spectra.

Our eyes have three different cone cells with different responses[2], so the fact that three basis functions[3] that roughly align with the cone cells work well as an approximation isn't that surprising to me.

That said, way back when I was doing rendering I was toying with the idea of trying to find a optimal basis functions for rendering, like if scene-specific ones could improve things.

[1]: https://www.psych.mcgill.ca/misc/fda/ex-basis-a1.html

[2]: https://en.wikipedia.org/wiki/Cone_cell

[3]: https://en.wikipedia.org/wiki/CIE_1931_color_space

For basic direct lighting its not all that weird, but for any indirect lighting or translucency etc it is pretty whacky. I guess it basically boils down to most real-world scenes having both fairly broadband light-sources and typical material spectrum being also relatively broadband, and in general having smooth spectrums. I'd imagine if real-world for some reason predominantly consisted of super spiky or notchy spectrums then the RGB rendering would struggle more. Or maybe not.

Yes, indeed. If you have narrow-band light sources, say a mercury vapor lamp[1], and some material that only reflects a different narrow-band spectrum, then you should get very little reflection while with an RGB renderer you'll get some reflection.

However as you point out, very few materials and lights are like this, hence why the approximation works well.

[1]: https://en.wikipedia.org/wiki/Mercury-vapor_lamp

yes, I suppose the 'baffling' part is partially from that theoretically there are infinitely (or nearly so) many different spectrums that produce some one specific color perception. So sort of probabilistically it could be seen as lucky that the spectrums of lights and materials align in a way that they do in real world.

I wonder what are the pathological cases here. I'm imagining some setup where an object could appear variably green, yellow, or red; this could be maybe accomplished by different mixes of different red/green lasers and maybe white light. The paint might be bigger problem :)

The eye also does a lot of work here, see color constancy[1]. BBC had a nice documentary on this back in the days, Colorful Notions[2], see the 19min mark for an experiment showing this in action.

[1]: https://en.wikipedia.org/wiki/Color_constancy

[2]: https://vimeo.com/11932120 (sadly poor quality, better rips can be found)

I think translucency is mostly whacky because it's typically represented by a single alpha channel, and a tint is represented by a color added on top. Which is not how most transparent materials look like.

I think we could do better, even in RGB by treating translucency as a 3x3 matrix instead of a single alpha scalar. Per-channel transparency could be represented by a diagonal matrix, but there is no reason that the current RGB basis would be adequate for representing transparent materials, hence the generalization to a 3x3 matrix.

Then a tint wouldn't need to be defined additively, a green tinted glass could simply not let most of red and blue through.

> I find the physically based rendering pretty disingenuous considering the huge gap still remaining between real-world and standard pbr

> ... I accept that it is just a domain-specific term that shouldn't be taken too literally.

Actually it is supposed to be taken literally. That's why it's physically based rendering and not physically correct rendering. The goal is to create a system based on real world physics to get as close as possible within the constraints of current computer hardware.

I believe this article gets it wrong:

> The primary goal of physically-based rendering (PBR) is to create a simulation that accurately reproduces the imaging process of electro-magnetic spectrum radiation incident to an observer. This simulation should be indistinguishable from reality for a similar observer.

I don't think the stated goal of PBR is to be indistinguishable from reality for a similar observer. At least, I have never seen this stated anywhere.

However, it's worth noting that PBR is a loaded term and includes both physically based lighting, and physically based shading (surface properties). If we are purely talking about lighting, that is, modelling the way light bounces in a physical way, it might be possible to get an exact match to physical reality. Light bounces in a relatively simple way.

The problem is that surfaces are not simple. They absorb, reflect, diffract, scatter, and so on. So even if the equations modelling the reaction of light are 100% physically correct, it's an entirely different story to model a complex surface like skin in a physically correct way.

This article is about light, so it's talking about physically based lighting. In which case the phrase this simulation should be indistinguishable from reality for a similar observer is more reasonable, but I think it should still be qualified for clarity.

Indeed. The book[1] on the subject states that the point is that, unlike the rendering methods that came before, the math is based on physics and physical principles. Of course one has to simplify and approximate, but the approximations should be grounded in physics and obey the underlying physical principles.

For example, in the early days people created reflection models without considering energy reciprocity for example. Well that's a physical principle that's very well established in physics[2] so any physically based renderer should only have reflection models that obey that principle.

[1]: https://www.pbrt.org/

[2]: https://en.wikipedia.org/wiki/Helmholtz_reciprocity

In a sense all (non-stylized) rendering is physics-based in the sense that all that physics is a model, and indeed the field of physics has different models of differing accuracy/precision/applicability. So even pre-PBR could be called physical in the sense that it represents some sort model of light, the rest is more of question on degree of accuracy and applicability. What I'm aiming at is that the line drawn for PBR is drawn in sand and feels somewhat arbitrary.

It's not that arbitrary to me. The earlier work was sometimes physically-motivated, and sometimes even based on physics. For example using Lambertian reflection[1] for diffuse surfaces.

But the key, in my view, to PBR is that the whole pipeline is based on physical principles and math, while the old renderers were not. So previous renderers would not bother properly normalizing the Lambertian coeffients for example, leading to violation of energy conservation.

[1]: https://en.wikipedia.org/wiki/Lambertian_reflectance

OP author here!

> I don't think the stated goal of PBR is to be indistinguishable from reality for a similar observer. At least, I have never seen this stated anywhere.

I’m, admittedly, highly biased as I work for Wētā FX, a high-end visual effects vendor. We have been developing for over a decade a spectral renderer, i.e. Manuka [1], whose design principles and philosophy are aimed at recreating how a motion picture camera, its lenses, filters, and even colour science, image the real world so that we can integrate virtual cg elements into plates as if they were part of the original photography. This hopefully gives some context to your quote. Whether we are successful at this is a different topic but this is a core business offering we are hired for.

- [1] https://www.wetafx.co.nz/research-and-tech/technology/manuka...

Hey, thanks for writing the article! My nitpicks aside, I learned a lot. I'm always interested in reading more about spectral rendering.

> I’m, admittedly, highly biased as I work for Wētā FX, a high-end visual effects vendor

That's fair. I can see it being a stated goal of PBR at Weta. However, I'm at the extreme other end of the rendering spectrum. I work with WebGL (I contribute a lot to three.js) and being indistinguishable from reality is most assuredly NOT a goal of a real time in browser rendering engine. I mean, we can dream. But it will be a long, long time before we could realize that dream so our time is better spent finding approximations that can work even on a midrange mobile device.

Reply on news.ycombinator.com