The non-Riemannian nature of perceptual color space (2022)
pnas.org
pnas.org
The fundamental “not-geometry” of color space is the output of a very complex deep neural net.
Thinking about color space as “real” world is almost nonsensical ;-). Trying to map “real” world into brain now has a deservedly bad wrap. Modern philosophy has largely given up on Descartes, sharp subject-object dichotomies, and the brain and mind as a warped mirror of nature.
If puzzling at first then check out Humberto Maturana and F. Valera (both experts in color vision electrophysiologist and philosophers), Martin Heidegger, Richard Rorty, and Daniel Dennett have powerful arguments against representationalism of the type implicit in most neuroscience.
Valera, Rorty, Dennett are all readable. Heidegger and Matura are more cryptic unless you are a philosopher.
https://en.wikipedia.org/wiki/Francisco_Varela
Valera’s “The Embodied Mind: Cognitive Science and Human Experience”, co-authored with Evan Thompson and Eleanor Rosch, is a good intro on this and related concepts.
But for Hacker News readers by far the best intro to this topic are chapters 4 and 5 of is the CS classic by Terry Winograd and Fernando Flores that is often highlighted on HN. It is a key text in understanding both what goes on in our brains and what is and may go on in machine brains.
1986. Understanding Computers and Cognition: A New Foundation for Design (with Fernando Flores) Ablex Publ Corp
It’s ridiculous that linearly adding “red” and “blue” would make “purple” — all different wavelengths — without a nonlinear effect occurring. CA prism separates these two back out and shows us purple didn’t exist in such an experiment. But we do see “purple” in a rainbow !?!?
We see beautiful pictures on LCD screens but we only sense only 3 colors from it? An alien that could see the whole spectrum would think we were completely crazy!
Red/green/blue are one set of primaries. Lots of artists working with paint use red/yellow/blue. Inkjet printers use cyan/magenta/yellow. Which colors are "primary" depends on the medium.
I think The Origin of the Work of Art by Heidegger is fairly readable. I'd also add one of Heidegger's classmate's, Walter Benjamin, whose early writing was directly about the experience of color. There's a book about it called Walter Benjamin, The Colour of Experience by Howard Caygill. Though its a bit difficult for the layman because it interprets Benjamin through his neokantian education (at the Marburg school, with Heidegger) and assumes a certain familiarity with Kant.
Different kinds of color spaces in different parts of that system.
The additive "RGB" colorspace has to do with the fact that we have 3 different types of color cones in our eyes, which have different spectral sensitivities. Any color perception that we can register is a combination of these three signals. Hence a three-dimensional space.
https://en.wikipedia.org/wiki/CIE_1931_color_space
The RGB primaries that we choose for monitors do not match these color sensitivities, but also map to a space that is made to overlap with our perceptual space.
> The color yellow has been an issue for many decades and diverse combinations of wavelengths evoke identical conscious yellow percepts.
No. Metamerism is a general property of human color perception. For each color we can perceive there are a vast number of spectral distributions that can produce that particular color.
https://en.wikipedia.org/wiki/Metamerism_(color)
One of the many reasons a purely physical approach is both too difficult and insufficient for color management.
In addition to these physical color spaces, there are perceptual color spaces, my favorite one being CIE Lab, an opponent color space.
With CIE Lab, the L component is overall luminosity, a is a red-green difference and b is blue-yellow difference. This corresponds to color processing that occurs after the initial signal is received and closely matches how humans actually subjectively perceive color, philosophical (im)ponderables aside.
Only approximately. In the linked article you can see that the color gamut is a horseshoe shape that cannot be fully spanned by a triangle. So there are color percepts such as responses to pure wavelengths that can’t be reproduced as combinations of real primaries. You could invent imaginary primary colors to make it work, hence the XYZ color space.
In any case, the whole point of TFA is that a space is not an adequate characterization of color percepts, because there is no Riemannian geometry that matches the psychophysical data.
> Only approximately.
No. That is precise, because three color sensors is all we (well most of us) have available[1].
>> Hence a three-dimensional space.
> Only approximately.
There was no claim of precision or precise matching for the space, only of dimensionality. The point was that it is a three dimensional space, and that's because there are three basic components that span it, the three color signals coming from the three types of cones. And once you have three base vectors, they will span a 3d space (unless co-planar).
So quite the opposite of what the poster I was replying to claimed, which was that the idea of representing human color perception as a "space" is somehow "comical".
> So there are color percepts such as responses to pure wavelengths that can’t be reproduced as combinations of real primaries.
Sure, which is one of the reasons why the matching functions can have negative coefficients for some colors / colorspaces, something that is, er, more difficult with real RGB monitors. Or, as you also noted, you invent primaries that cover the space but go outside of reality. The spaces are mathematical constructs, after all. However they are not arbitrary mathematical constructs. The fact that they are 3d spaces is rooted heavily in reality. And mathematically they are actually all the same space (as distinct from specific regions in that space if you constrain the vector components, for example to range [0,1]).
> In any case, the whole point of TFA is that a space is not an adequate characterization of color percepts, because there is no Riemannian geometry that matches the psychophysical data.
Not really. The article is a bit fuzzy on some of the terminology, but if you read even the abstract closely, it is clear that they are talking about perceptually-uniform color spaces that make a claim about color differences in that space matching perceived color differences, our good old friend ΔE.
https://en.wikipedia.org/wiki/Color_difference
I don't see the paper making such a claim for the more general concept of a color space that does not claim to be perceptually uniform. CIE XYZ for example, does not claim to be perceptually uniform. And isn't.
And once again, it is well-known that "perceptually-uniform" color spaces are only approximate. Always have been. ΔE varies in different regions of the space. It also varies depending on the area of the color swatches[2]. It also varies depending on whether the color is a light source or reflected. (IIRC, CIE Luv performs better for light sources). That these distances don't add is an interesting additional wrinkle, but not surprising, given all the other oddities and also the fact that the color spaces were never intended for having the distances add.
[1] Some people have reduced function in one or more of the cone types, they are color-blind, and some apparently have four types of cones.
https://en.wikipedia.org/wiki/Tetrachromacy
[2] The way those color spaces are tested is to give people swatches with colors that, I think, they are allowed to put next to each other. If they can tell they are different, the ΔE is greater than 1.
I have quantified variation in photoreceptor mosaics in humans, primates, cats, dogs, and mice. The variation in ratios of cone types is impressive. Variation of R:G ratios in health human retina is impressive (work by C Cursio). Yes, sure, humans still call green green despite the differences, but not so true of those with opsin gene variants.
Here are three of my own studies: the first of which re-discovered that the nasal periphery of humans is tiled by large cones.
https://pubmed.ncbi.nlm.nih.gov/1829378/
I am not sure what you mean with "this space" which is supposed to be "closer to the truth". Can you elaborate? I am pretty sure I made no claims in that direction[1].
What I am saying is that modeling human color perception as a three-dimensional space is sensible, given the fact that we tend to have three color receptors with different spectral sensitivity.
Similar physiological reasons apply for the CIE Lab space. Yes, it is a mathematical construct, but it is a reasonable mathematical construct. My color theory has been a while, but my understanding is that there is some pre-processing of the raw photoreceptor inputs that means the signals arriving in the brain are differential signals like the red-green difference (a) and the yellow-blue difference (b) of CIE Lab.
And of course we get the luminance from the non-color receptors separately, though of course the signals are correlated. So how we interpret a specific (a) signal depends on what L is.
So there are good pragmatic and scientific reasons for most of the proposed color spaces, but they are all compromises and driven by different pragmatic needs. They tend to be based on CIE XYZ, which defines a "standard observer".
Is your objection this definition of a "standard observer"?
To me, the very fact that such a "standard observer" is defined and called that makes it pretty clear that this "standard observer" is not the "universally true observer", but rather a useful fiction. And again, not an arbitrary fiction, there are reasons for choosing that particular one, but I don't see any claim to universal truth.
I am still somewhat puzzled by your mention of metamerism as only applying to the color yellow.
[1] I did single out CIE Lab, but clearly marked that as a purely personal preference.
How can you possibly determine whether different stimuli evoke identical percepts? (I don't know a technical definition of ‘percept,’ but am assuming that it carries some additional data beyond something that can be directly measured from brain activity. If not, then I withdraw my question.)
Dogs see only two colors, this means compared to humans they are color-blind (or something like red-green blind).
First of all the difference between a strict and a non-strict inequality is not that big, stating two things are never exactly equal is nigh impossible to measure, especially for something subjective.
Secondly that could happen simply by reporting distances in some non-linear fashion, and who cares if the unit of distances is non-linear? If there is a metric that correctly identifies which colour is closer then we're in business, nobody ever said this had to correspond linearly to whatever measure of perceptual distance someone came up with.
Humans are not good at judging questions of the type “How big is the difference?” that form the basis for many large difference experiments.
This is not that surprising, since people generally get worse at noticing small differences as the absolute quantities get larger. Unfortunately, the model of perception uncertainty used in the paper (Thurstone’s theory of Gaussian perceptual process) does not take this effect into account.
On thinking about this issue, it seems to me that I think of large differences categorically (red vs. green, for example) while thinking of small differences as being on a continuum, even when the colors in question lie in the borderlands of the above categories.
So ΔE is actually 1 in some parts of the space, but up to 4 in others. However, that is "good enough" for the purpose for which these color spaces were created: quality standards for color ("can I buy more of the same color and it will look the same?"). If your measured and computed ΔE is below one, the difference will not be perceptible by most humans regardless.
And last I checked, new and improved "perceptually uniform" color spaces are proposed every couple of years.
(Yes "length space" (or "path space") and "geodesic space" are not exactly the same thing, but "length space but not geodesic space" doesn't exactly seem like a very likely possibility.)
I agree with others that it is not surprising and is a technicality. If you put that aside then the situation is much closer to your intuition.
I do not know if the authors are claiming that what we perceive as color similarity is not a metric at all, but personally, I would not be surprised if it were not. See my other post for how my subjective perception of color differences seems to me.
That's not a violation -- "stronger than" is the opposite of a violation!
Humans are not good at judging questions of the type “How big is the difference?” that form the basis for many large difference experiments.
This is not that surprising, since people generally get worse at noticing small differences as the absolute quantities get larger. Unfortunately, the model of perception uncertainty used in the paper (Thurstone’s theory of Gaussian perceptual process) does not take this effect into account.
Ramsay (30) suggests that the principle of diminishing returns or its opposite may have been spuriously identified by otherresearchers (daringly including his PhD advisor, Helm) because of experimental procedures (successive intervals, paired compar-isons) ill suited to the task. Indeed, open-ended and criterion-dependent tasks give less accurate measures of similarity because of individual factors and the difficulty of the task…
Instead, we use a more reliable two-alternative forced choice (2AFC) task, where the participant simply answers the following question: "Which is more different?" Specifically, we use a triad arrangement of stimuli with the reference in the middle and one test on either side. Each of 320 triads covering the neutral axis was judged by at least 250 different participants in a crowdsourced study on Amazon Mechanical Turk (MTurk) (52).*
Color has long been a convenient “entry point” into the study of human unconscious and subconscious data transformations, most notably through Wittgenstein’s musings about red squares. I look forward to this technique being extended to other spaces, even ones as abstract as “moral space” or “valence space”!
It's not perfect, but well documented and discussed, has freely-available implementations, efficient shader code, etc. and overall works pretty well.
This isn't to reduce the notability of the article, but isn't BIPM's interest in any color-perception stuff basically limited to making lightbulbs roughly equally bright? (i.e. https://en.wikipedia.org/wiki/Luminous_efficiency_function) I guess I don't see how even nonlinearity in perceptual space matters for that purpose.
So, RGB is close to how screen works, but is very un-intuitive as its algebra is additive rather than subtractive (like the one we have with paints). Then HSV is a middle ground to be more practical for humans, as it separates the color into its 3 most significant dimensions, that are value, hue, and saturation. It does so while being a fast linear transform, but it is not a really good mapping with regards to human perception. For example a given fixed value will appear as very different when given two different hues. Still, it's a nice middle ground that made it a good choice for painting and image manipulation software.
There are other spaces that are "more" perceptual, in that changing the hue of a color should not affect how its value is perceived. This can be useful in design work as a palette could be hue-shifted and always stay legible, which was not possible in the general case with HSV or RGB. Yet, these fancy color spaces are not used a lot because in most practical applications there isn't enough justifications to use them. In particular because professionals are used to workaround the space deficiencies, so there's a cost to re-learn some of the primitives. The only really practical application I have ever seen of these perceptual models is to simulate 'color blindness', which theoretically give the possibility to design charts and maps that are more accessible to people. I believe such a validation tool should be required for public communications.
The article is interesting and well written, as it gives an history of how some of these abstractions were developed, and highlight some of its shortcomings.
Sure, color space can be reformulated many other ways for applications but all just operational kludges.
In Riemannian space, the distance between points is the length of the shortest path between them. Euclidean space is a subtype of Riemannian space, where angles in every triangle sum to 180°.