My mental model of how alpha opacity works is wrong (2020) | Hacker News Reader

My mental model of how alpha opacity works is wrong (2020)

coolbutuseless.github.io

133 pointssebg69 comments

He hasn't found the hard problem yet.

If you have two translucent objects, how do you combine the colors? This is hard to do with Z-buffer rendering. The obvious coloring operation is not commutative, and in a Z-buffer system, you don't know which face gets rendered first.

This is a huge pain. You can't just dump everything into the GPU and let it draw in parallel, which is what GPUs do fast. You usually have to depth-sort triangles, so the front triangle gets rendered last. This delays rendering while the CPU does the sort.

There's a GPU-based solution to this, but it involves some compromises in quality.[1]

[1] https://developer.nvidia.com/content/transparency-or-translu...

The "right" solution with no quality compromises is per-pixel linked lists (PPLLs) that are then sorted afterward. This is straightforwardly implementable on GPU, but it remains to be seen whether the technique will take off, given the overhead.

Pixel Shader Interlock (or Raster Ordered Views in D3D12 terminology) is effectively the same thing but without a resolve step, but the overhead is massive and basically too slow in practice.

To be honest, the game developer way of solving this is to just not use transparency. Too many things rely on the G-buffer / depth buffer these days, and transparency is usually limited to a lower quality render anyway.

Involving colors is much more complicated than that; in theory if you shine white light through stack a red filter on top of blue filter, I'd expect the intersecting area to be closer to black than purple. You can see that in for example this picture: https://www.walmart.com/ip/Sightmark-32mm-Filters/31197086

This is just the tip of the iceberg when it comes to the complexity of translucent objects. To get truly convincing results spectral rendering is needed, but of course that is prohibitively expensive for real-time use.

Yes its all very loosely approximated to real life anyway when avoiding the spectral level. Its not just the filter but also the quality of light being emitted/reflected from behind - A broad spectrum light like the sun will create more even effects while narrow spectrum lights like flourescent lamps or HPS street lamps can act variably.

Looking through a yellow filter at a yellow flourescent light - you could easily see almost all the original light (bright yellow) or none (black) depending on the spectral transmission of the filter.

Why is that any more difficult than calculating color reflection/absorption on a surface? Both presumably involve multiplication in spectrum space.

You don't have multiple of those stacked on top of each other. At least not as long as you don't dive deep into the global illumination rabbit hole. And then you've got bigger and more pressing problems to solve.

Yes, you are right that conceptually it is the same problem both with reflections and transmissions. The key difference is that in typical scenes indirect/bounce lighting tends to have comparatively subtle effect whereas translucency, especially when colored, can be easily quite apparent. Colored mirrors are pretty rare but colored transparent objects are very common.

You also need to simulate polarization (and maybe quantum mechanics) for sunglasses to work properly.

I don't know where the quantum nature of light would come in for sunglasses, but you are correct polarisation would definitely be needed.

It's claimed that polarization is due to quantum effects.

https://www.youtube.com/watch?v=zcqZHYo7ONs

…I think it actually isn't and the people explaining it have just forgotten it's supposed to be a metaphor though.

polarization is very much a quantum effect, however the specific experiment in the video does not prove the quantum wierdness because the setup does allow a hidden variable explanation.

Polarization of a single photon has to be treated in a QM framework , but there is no need for that with normal sources.

Also, polarization as a degree of freedom arises out of symmetry considerations, and that's what you learn in QFT.

The (stochastic) order-independent transparency and k-buffer line of work is definitely cool for what it was. I get the sense that RTX will finally make it so that folks are going to end up just ray tracing the transparent volumes (even if you first did a rasterization pass of the "non transparent stuff").

The hard problem in 2D rendering (where objects are drawn in a user-determined order) isn't translucent blending of unsorted polygons, but conflation artifacts. If you have two adjacent squares where the first covers the left half of a pixel and the second covers the right half, and both are drawn with alpha blending, there will be background showing through which shouldn't exist, and too little influence from the first square's color. Similarly if you drawn two identical squares on top of each other, which both cover the left half of a pixel, the first square will have some influence over the background which shouldn't happen at all.

The lazy way to deal with this is to render at a high resolution and then scale down before displaying. Is there a better way?

Collect all polygons with influence over a pixel, then process the resulting lists pixel-parallel using exact math instead of trying to abuse a few planes of Z-buffered framebuffer to accumulate per-pixel state into a few numbers (while handling polygons virtually one-after-another, using some locking tactics of sorts to serialize their access to a single pixel's framebuffer values, if even that much concurrency-correctness is done).

> There’s a GPU-based solution to this, but it involves some compromises in quality.

There’s another GPU-based solution with RTX and other ray tracing solutions that is high quality. Might be compromising efficiency in some typical cases, unless a lot of geometry and/or instancing and/or samples are used, and then in that case it may be both an efficiency and quality win.

We really don't have the ray-budget for translucency with RTX. We basically only have the ray budget for a handful of rays per final screen pixel, and good translucency requires dozens for every interacting surface. There's a reason most engines using RTX only use it backed with a deferred style implementation.

If you’re talking about games, I agree, and there are multiple reasons, but generally speaking for viz or pure rendering or other cases, the budget might be bigger than you think. I’ve implemented real time transparency + path tracing in OptiX. Depends on your quality goals and denoiser options too, but several billion rays per second is a lot.

Is it actually not commutative? If you consider them to be filters in the same way you can with the article but on each RGB channel doesn't it still work out? Or is this assuming we're also dealing with reflection in addition to transmission? Or am I missing something entirely?

transparent red on top of blue means more red and less blue, transparent blue on top of red means the opposite

So we're assuming reflection as well as transmission?

If we're just talking transmission they would both be the same, ie transmitting all red photons and 50% of the green and blue ones while the other filter transmits all blue photons and 50% of the red/green ones. Either stack has 50% red values, 50% blue values and 25% green right?

Oh is sorting a bottleneck here?

Well... if we're trying to model how a translucent pane would work in the real world, then this is correct for one particular interpretation of alpha.

However, if we're talking about how alpha blending actually works in software, then it can vary a lot.

For example, in Direct3D you would typically use the `D3DBLEND_SRCALPHA` and `D3DBLEND_INVSRCALPHA` for the source and destination blend modes respectively. In that case, the combined alpha is actually `src_alpha^2 + (1 - src_alpha) * dst_alpha` which is different again from the two calculations in the article.

There are even other ways to achieve a physically realistic result that use a different interpretation of the alpha channel. Or maybe you want to solve for something else (eg. order-independent blending)

sometimes i read things like this, and i'm astounded that anyone could have developed a mental model so incorrect and unintuitive but close enough to persist for a while.

then i remind myself that i probably have a few situations like that in my own mind and i should remain vigilant.

edit: and then i get just a little more frustrated next time someone resists a correction that i provide.

> incorrect and unintuitive

The thing that you might be missing is that "incorrect" is (usually) objective, but "unintuitive" isn't necessarily so. I can see how someone else would think additive opacity is intuitive, even if it's very easy to show that it isn't correct.

Every one of those things limited to [0; 1] combine multiplicativelly. (Mostly because they are some kind of probability or vectorial projection - what are the same thing, by the way.)

You are right that "intuitive" is a subjective property. But I'm completely lost trying to imagine what kind of life experiences can lead on to expect them to be additive.

I also thought that opacity was additive and clamped to 0,1 until right now.

Seem intuitive to me. Half + half = whole. When I make a chart in R and set the color to alpha 0.5, the result looks more or less like an opaque point if I plot 2 of them on top of each other.

Apparently that's wrong, but it's consistent with my not-very-precise observations, and I have never had a need or desire to learn more about how opacity works. One cannot be an expert in all things.

What intuition should a layperson have to suggest that obviously opacity/transmissivity is multiplicative? I don't see the value in deriding the incorrect intuition of uneducated people. In statistics at least we accept that statistics is hard specifically because it can be unintuitive.

Out of curiosity, did you have any rationalization for why clamping it made sense, or you just didn't think about it?

"How can something be more opaque than fully opaque?" was my intuition (and probably many other people's). Also RGBA numbers are generally clamped to 0,255.

Ok, I guess the repulsiveness of a clamping operation is something we learn. If one doesn't learn to fear it, it's an operation like any other.

Should have been obvious, but I never thought about it.

Why do you call it repulsive?

It's a very common operation in graphics. Even so that certain graphics processors have a flag for floating point operations which allow the results to be saturated (clamped to [0,1]) at no extra cost.

I know I prefer to avoid it and design equations so it isn't necessary. That often helps make the math cleaner and more well behaved. But repulsion sounds stronger than what I feel.

I would say there are plenty of cases where things do act additively yet we wish to clamp their values to a specific range (often valid colours).

It does appear when you are trying to optimize graphical operations and change the real physical models with approximate ones. It's also a signal that your approximate model won't behave well on repeated application, what is normally not a problem in graphics display, but a red flag on general purpose image processing.

On a physical model, it is a very strong signal that you messed something up and your model is wrong. Sometimes you get something that looks like a sigmoid but has the exponential parts very short, so it looks like a clamped linear function, but you normally need some kind of mechanism limiting the output on the linear region. Simple phenomena do not have clamping, it is a very complex operation.

Thank you for putting it so well. It seems we're on the same page and I actually agree with everything you said. That's why I avoid it when trying to model things and often see it as a warning flag when reading code written by others.

The naive model: Eventually you've blocked all the light, and adding more semi-transparent things on top can't block any more light.

> What intuition should a layperson have to suggest that obviously opacity/transmissivity is multiplicative?

My intuition is that if I put on dark sunglasses I can still see through other dark things (like a CD or welders glass) if I look at something bright enough (like the sun). If opacity/transmissivity is modeled like it works in real life, given this example, it follows that it wouldn't be additive.

That's a great point, and it illustrates the problem with the mental models of uneducated people. It's difficult to know what information should and shouldn't be part of the model! I figured it just didn't work like real life, because RGBA numbers are clamped.

> What intuition should a layperson have to suggest that obviously opacity/transmissivity is multiplicative?

Tinted windows and sunglasses come to mind. Intuitively, if you put translucent things back to back, you expect light to still pass through, albeit less of it, but always some. A noteworthy exception is when using polarized filters, which can unintuitively block out 100% of light depending on the angle between the filters.

It’s just bad intuition, sorry :)

E.g. when you mix two grays (#80808000) in RGBA space you probably don’t expect white. None of R, G, B, A are additive in a sense of color/transparency mixing. They are only additive component-wise regarding the resulting color, i.e. R+G+B is a color, but two reds aren’t 2R.

>But I'm completely lost trying to imagine what kind of life experiences can lead on to expect them to be additive.

I, on the other hand, am completely lost trying to imagine what kind of life experiences can lead one to believe the OP error requires some "unique/rare" life experience.

How about the very common life experience of

(a) not being good at math but still being a programmmer and having eventually to deal with something math-y?

(b) the life experience of someone who might now enough math to get this, but didn't give it alpha opacity much thought before, and the simple addition model was good enough for them (as they never had to do anything special with it)?

It depends on what your experience is with. If you've done any digital painting, you're probably all too familiar with low-opacity strokes seeming to never add up to full opacity, and will have concluded the opacities are not simply added together.

I immediately assumed the author might have been pretending a bit in order to motivate the article. Self deprecation sometimes makes things more relatable. I bet there’s even a name for using self deprecation initially as an excuse or motivation to make a point, but I don’t know what it is.

That said, I’m well aware that opacity is multiplicative, and yet it still caught me off guard before to realize that the commonly used lerp blending mode (SRC_ALPHA, ONE_MINUS_SRC_ALPHA) is incorrect in the alpha channel. Pre-multiplied alpha can also be unintuitive at first, even with a decent mental model.

> but I don’t know what it is

Or are you just pretending not to know to seem more relatable?

:) hehe, well now I have to admit I actually tried to look it up, ended up on the Wikipedia list of rhetorical device names, and only got halfway through the A’s before giving up.

Discussions of photographic materials usually use density instead of opacity. Density is a logarithmic value equal to -log10(transmission), where transmission is the amount of light passed through (or reflected) on a scale of 0 to 1.

So to give some examples, a material with 100% transmission would have a density of 0, a material with 10% transmission would have a density of 1.0, and a material with 1% transmission would have a density of 2.0.

Density values have some advantages, including:

- They can be combined with simple addition

- Compared to transmission or opacity values, they correspond more closely to human visual perception

This is more generally called absorbance[1] and stems from the Beer-Lambert law[2].

Been too long since I've done computer graphics, but you made me think about implementing transparency pass using absorbance instead.

[1]: https://en.wikipedia.org/wiki/Absorbance#Absorbance_of_a_mat...

[2]: https://en.wikipedia.org/wiki/Transmittance#Beer%E2%80%93Lam...

Light is additive (in a linear colorspace). But the opacity of a layer means blocking part of the light, and such light absorbers are multiplicative.

Alpha is also multiplicative, not just transmissivity. You just pick the right operation to think about it.

https://en.wikipedia.org/wiki/Alpha_compositing

If your compositing operation is A over B, then transmissivity is multiplicative for the A&B region. If your compositing mode is A in B, then opacity (alpha) is multiplicative.

This has the same structure, mathematically, as thinking about logical operations like AND and OR, which are complementary to each other. If you think of AND as multiplication, well, you can think of OR as multiplication as well, just with the values flipped around.

This is a common problem with statistics. Ask someone to answer "if Saturday has 25% of rain and Sunday another 25% what is the probability of rain during the weekend?" The intuitive, but wrong, answer is to add probabilities capping them at 100%. So, this mental model seems quite natural for humans.

If the original author is here, I find that the original paper [1] still remains one of the better discussions of what is trying to be represented with alpha blending operations.

It's all just an approximation. Don't lose sight of that!

[1] https://graphics.pixar.com/library/Compositing/paper.pdf

Alvy Ray Smith wrote a good paper which might help to set things straight for people: http://alvyray.com/Memos/MemosCG.htm#ImageCompositing

For more fun, draw a filled circle in Inkscape. Then copy and paste it in the same place, and again, and again ... and watch the edges get more and more jagged. Try to explain that to an average user.

"The pixels on the edge are supposed to be halfway between white and black to indicate being half-covered by the circle, but if you pile them up on top of each other anything with a little black adds up with its duplicates to complete black."

But if you zoom in, it looks the same.

That's because people, including me when I started programming, confuse alpha with coverage. Most people use the alpha value for coverage. Most of the time it doesn't really matter visually but there some edge cases where it does matter.

The best docs I could find in a short amount of time, are those from anti-grain[1] where the author describes it as a "second alpha channel.

[1] http://agg.sourceforge.net/antigrain.com/doc/basic_renderers...

I find this usage of “mental model” to be really strange -- it implies you’re intentionally simplifying or abstracting something. Alpha blending is well-defined mathematically, so why would you need to abstract it? It’s perfectly OK not to know how it works, or to think you know and then realize you’re mistaken; but thinking in terms of “mental model” implies it can’t be fully understood, which isn’t the case. When you know how it works, you don’t have a more refined mental model, you just know! (Or think you know...)

I think they're saying they thought they knew how it works, but were wrong.

Your mental model of mental model doesn't match mine.

I have a mental model of how an elevator works from using them many times, but it's absolutely a simplified understanding of the real mechanisms at work. I believe elevators are fully understandable but I don't need that level of detail to operate one. If it became important to know more about them (say I was in an elevator that malfunctioned) then I'd invest more in making my mental model more correct.

So to me, mental models aren't necessarily intentionally or consciously formed and they can certainly be used for quick simplified reasoning about things that may be in fact well-understood.

Yeah I realised this one day after thinking about it. This is a very elegant explanation of the correct way to think about it.

Why oh why can't we just settle on 'transparency' as the best word to use in this space?

Its more mentally simple than 'opacity' ('transparent' is a more common word than 'opaque' to hear used in the real world - as 'opaque' is the default of most objects/surfaces, having 'transparency' is a special property that needs to be described), more correct than 'translucency' (also used commonly including in this thread but in real definition means an object that passes light but not a clear image of what is behind, like a frosted window - light is not travelling in a straight line but is being bent) or 'transmissivity' which the author uses (needlessly making it more technical and opaque to the end user - and actually less specific as transmissivity could be translucency also)

We use “opacity” when referring to the alpha channel because that’s what the alpha channel measures. A high alpha value makes the pixel more opaque, a low value makes it more transparent (less opaque). Saying that a pixel with “transparency 0” is transparent and one with “transparency 255” is opaque would be far more confusing.

Yes and no. Yes opacity is what is in the alpha channel. But all im saying is that it would be easiest for all to think of as inverse of transparency, and this is how documentation should describe it, otherwise people make weird assumptions like OPs. Its stored as opacity rather than transparency for minor performant reasons not because its a different concept or better way to think of it

Relevant video about cross-fades in CSS which I definitely learned something from: https://youtu.be/PYSOnC2CrD8?list=PLNYkxOF6rcIAKIQFsNbV0JDws...

On the topic of color mixing in realtime, we talked about pigments previously this week:

Rebelle: Experimental online paint software https://news.ycombinator.com/item?id=30191520 (3 days ago, 22 comments)

Next: learn about premultiplied alpha.

https://en.wikipedia.org/wiki/Alpha_compositing#Straight_ver...

Want to break your mental model about the real world? Try overlapping polarization filters, get puzzled why the center spot stays brighter than surroundings as you add more filters.

He still hasn’t got it. Assets can have straight alpha or alpha premultiplied with a color.

Well that was a dumb mental model. I thought it would reveal something unintuitive.

Reply on news.ycombinator.com