Hacker News Reader

Perhaps a nitpick, but 256^3 doesn't really represent anything about human perception-- it's just how we store colors in computer memory. The human eye can perceive a far wider range of colors than any screen can (currently) represent. It's just as valid (and probably more accurate) to use floating point to store color (rather than bytes), and in many high-fidelity applications this is actually how it's done.

The bit about colors "looking different" in different contexts also isn't really an "extra axis". The example you give is just your brain (attempting to) separate out the illumination from the scene colors. The range of actual colors you can perceive isn't increased by this.

Fundamentally human color perception is three-dimensional, because humans have three classes of color receptors on their retinas. There are also rods in the retina (the cells that are color-insensitive and detect only brightness), and in principle they could inform color perception, but this does not seem to be the case-- the intensity of light only shifts things around in the original three-dimensional space.

There are also a very, very few people (women only) with genetic mutations that give them four different color receptors in their retina. However, there is no evidence that their brains can take advantage of this extra information and allow them to perceive an extra "axis" of color. It is much more likely that the "mutant" receptors are lumped in with the stimulus of one of the others, which will shift their perception of color, but not expand it.

In short: There are three axes of color perception. Optical illusions don't change that.

Optical illusions reveal that we have more than three axes of color perception. The first gif and the last photo are examples of this, and the renaissance artists had a firm appreciation of it. (See DaVinci's journals, specifically the chapters on color.)

Part of the reason I want to put together this series is to rekindle this knowledge. It seems very much "lost": your comment represents the status quo of color science, but if you were an ambitious artist in 1450, you'd run experiments and determine that much of what we take as fact is actually quite a lot more complicated in practice.

The eye, and especially our perception of colors, is so complicated that entire tomes barely scratch the surface. And Munsell's discoveries were only made in 1900, barely over a hundred years ago. There's still a lot more to discover.

To be clear: it's true that our three receptors imply three axes of color perception. But when you assemble an image as a whole, the entire portrait results in an experience quite different than any individual color.

EDIT: That the perception of color is influenced by surrounding colors is true, but that this adds a "fourth channel" is [citation needed].

One of the most important aspects of color science is that you have to be willing to believe the possibility that some strange ideas are true.

In this case, I can dispel that illusion, but only if you're open minded to it:

Consider a painting. Why choose a certain shade of yellow? To produce an effect.

The above images demonstrate that where you put that shade of yellow causes a very different effect.

Now, if you accept as an axiom that the only reason to choose a color is to produce an effect, then that means all three primary colors (red, green, blue) are different axes in your ability to cause an effect. But the fact that the arrangement of colors causes different effects means that the spatial arrangement is a fourth "lever" that you can use to change the experience. That implies it's accurate to call this phenomenon a fourth channel.

(I'd post this edit as a reply, but HN isn't having it right now.)

One interesting area of science to investigate is the frequency spectrum of natural images. Our brains are tuned to see certain frequencies more than others, e.g. blades of grass. And the reason colors produce different effects depending on where they are in relation to each other is to help us resolve different shapes in an image.

http://web.mit.edu/torralba/www/ne3302.pdf

https://www.cs.cmu.edu/~efros/courses/LBMV07/presentations/0...

The statement I take objection to is this:

> The fact that colors influence colors around them means that there are far more than 256^3 viewable colors: the spatial axis is like a fourth channel.

That the perception of color is influenced by surrounding colors is true, but that this adds a "fourth channel" is [citation needed].

I'm not sure I buy that it means there are more absolute viewable colors there, but rather there are merely different interpretations of the same colors. We do have fairly strong confirmation that our brain doesn't interpret color absolutely but can adjust based on luminescence (see: all examples of shadow illusions) and patterns (see: https://www.youtube.com/watch?v=mf5otGNbkuc), perhaps that's what is meant by a "fourth channel"?

I’ve referred to DaVinci as the father of CG, based on his formalizing the perspective projection. This bit about color adds to my thesis.

The simplest example of this is that in a watercolor class I was told to make shadows bluer than the object, not just blacker. And indeed it looks much more natural than a black shadow, and I think it's this effect of juxtaposing colors.

The sky is blue and the sun is yellow. Subtract the sun and you are left with only blue light from the sky.

...which explains why indoors, shadows /do/ tend to be colorless: http://www.camilleprzewodek.com/uploads/1/5/9/2/15923552/639...

If you have a one dimensional line, you can use a 1 dimensional value x to represent its position. If you add a second input, y, and instead say that your position on the line is x+y, the line is still one dimensional, even though y changes how the value x is perceived. (Because the point x,y can still be represented with a single value z where z=x+y)

There are three colour receptors, so colour is 3 dimensional to us. That the surrounding context shifts where that 3d point falls in the colour space doesn't create more colours, it just shifts the perceived colour around the colour space. In your strawberry example, the gray and red already exist in the 3d colour space. The context did not create these, it just shifted our perception a little off the point in the colour space that the pixel value alone would have placed it.

Similarly, if I wear sunglasses, the darkness doesn't add another dimension, it just shifts all the values a bit in a direction that makes them darker.

Note: I'm not actually arguing that there are only 3 dimensions of colour (although our three receptors would suggest it), as I don't know enough about it. I'm just pointing out a flaw in your logic.

The retina itself does various forms of processing, some of which is about eliminating noise, but some of which is involved in relatively complex things like edge detection. The nerve bundles connected to the retina do further processing. The visual cortex itself does a lot of processing, then the conscious part of the brain does even more. There's recompositing and temporal correlation, motion-compensation, prediction, edge detection, lighting/shadow compensation & color perception, threat assessment, pattern recognition, object recognition, face recognition and that's just scratching the surface.

Even transmission of color information isn't a simple RBG signal. It appears the eye transmits differential signals (such as B-Y) and other information as well.

There are multiple layers of processing that go on in our visual system. You can't reduce it to three axes. The eye is not a camera, doesn't work like a camera, and can't be simplistically thought of as if it were a camera.

All of those things about visual processing are true.

> You can't reduce it to three axes.

This is my claim: It is possible to represent all possible visual stimulus with: f(direction, time) -> (a, b, c). That is what I mean by "vision is three dimensional".

It is not possible to add another channel "d" to the (a,b,c) tuple which is not redundant; so anything you add would not be an extra axis or increase the dimensionality of the perception space.

The brain/retina may generate lots of extra "channels" from processing this (moving) three-valued image, but those extra channels can't be stimulated independently of each other; they depend entirely on those original three degrees of freedom as a function of space and time. In that sense they aren't really "axes", because they aren't degrees of freedom. That's what I mean by "there isn't an extra channel."

If you agree with that, then we don't disagree and we are just arguing semantics. :)

It’s funny that you introduce time in your equation, because it shows how your view is incomplete. In our visual system, you cannot reduce to three dimensions without creating a recursive relation. It should be more like:

Ret(a,b,c)_t = f(Ret(a,b,c)_{t-1}, current_stimulus)

Since it is a recurrence, we cannot reduce to three dimensions.

Couldn't you use this same line of thinking to argue that only one dimension is required? Our eyes may generate 3 channels, but don't they depend entirely on the frequency of light received which is one dimensional?

Light is not perceived at a single frequency. It is a spectrum. That’s why you can see non-spectral colors like pink.

OK, well, are there any colours which are not representable with two frequencies? That is still less than three dimensions.

White.

Isn't that representable with a mixture of blue and yellow light?

You can pick a blue and a yellow and a proportion at which to mix them which will produce white, but for any two primaries you pick, there are a multitude of colors you can't represent. For example, if your basis is blue-yellow as you suggest, then you can't represent red or green (and an infinity of other colors off-axis).

Fundamentally the chromaticity diagram[1] is 2D (with a third axis for intensity). You need to mix at least three points to span the space of the plane.

[1] https://dotcolordotcom.files.wordpress.com/2012/08/anatomy-o...

To be fair, the receptors are VERY time dependent too. Take a stare at the top bar for ~20 seconds and then move your eyes down just a bit. You'll see an echo image of the opposite color (colorblindness depending). This shows that even a 'little' time spent saturating the retina will effect the later images.

As other people mention, the retina does a LOT of pre-processing before any 'signal' gets sent. The very first synapse from the receptor is heavily modified by horizontal and bipolar cells in a time-dependent reactive manner. The 'images' we 'see' are nothing like what the retina 'sees'. People with cataracts will commonly have very advanced occlusions before thinking there may be a problem, due to the incredible amount of processing the retina itself does. They really cannot 'see' that they cannot 'see'.

So, trying to reduce vision to a color mapping is just so ... computational. Biology is not like that all the time. It's not a 'map' like with sound and hearing (where tonotopy is semi-conserved throughout brain processing). With vision, things are used as 'information' almost instantly, but for certain by the end of the LGN into V1. You can't really backtrack the data through the system once it hit V1, it self-modifies the memristance of the synapses too quickly to chase the action potentials again. So thinking of things as 'red' isn't what the brain wants to do, it's not matrix algebra. It's 'grandmother' and 'tiger' and 'that-look-in-her-eye'.

You can represent all colors on three axes. If you choose each primary color so that it stimulates exactly one of your three color receptors, it is possible. However, there are no wavelengths of light with that property, so in real life, it's not possible to produce a monitor that can show you all colors you can see (with only three primaries, that is).

You can see the various compromises that people have developed. sRGB has real primary colors (your monitor is probably using them right now), but there are many colors that it can't convince your eye to see. (There are also colors you can see that don't actually exist as a single wavelength of light; magenta for example.) Adobe RGB picks different primaries and you can see more colors. DCI-P3 picks still different primaries and gives you even more colors. But in the end, there aren't three points you can pick that will give you all colors.

You can pick more than three primaries to get more colors. CMYK for printing is an example of this.

If you want to mathematically represent all colors, that's easy. CIExyz does that, but the "primaries" can't even be called that because they have no relationship to human anatomy.

This Wikipedia article is a good starting off point for understanding color perception:

https://en.wikipedia.org/wiki/CIE_1931_color_space

(As for computer use, I tried a wide gamut display once and it's a disaster. 99.99% of all content is sRGB, so you might as well have your monitor using sRGB primaries. The metadata situation is too broken for browsers or the OS to know what color space an image is actually in -- displaying an sRGB image with Adobe RGB primaries makes images unreasonably vibrant. Displaying an Adobe RGB image with sRGB primaries looks ridiculously washed-out. Adding to the fun is that browsers like Chrome will convert so that things look right... but assume display colorspace if there is no metadata in the image. And all the photo and art sharing websites I checked a couple months ago actually delete that metadata, so if you compose your images in a colorspace other than sRGB, most people will see the wrong colors. It sucks. I took up black-and-white photography. Everything can display that :)

I would say I'm surprised that there are no monitors that use more than three primaries to display color... but the reality is that even wide-gamut 3 primary color is so broken that it's probably not even worth trying. You could see more colors, but nobody else would see the right colors. Why even bother, I guess.

Don't forget purple, which doesn't exist in the rainbow. It's red and blue, which average to green but we see it as purple. Which validates the three-axis idea: purple is when the red and blue axes are dominant. But it's still peculiar.

'red and blue' averages into green? Yeah, that's not really how our eyes perceive light.

We see the colors that we see because certain wavelengths of light stimulate certain cones in our iris, which makes our brains hallucinate in color, basically.

We see green when the short-wavelength-sensitive, medium-wavelength-sensitive, and long-wavelength-sensitive cones are all stimulated. A single wavelength of light around 520nm will stimulate all of these cones and satisfy the requirement for green to be seen. In fact this is usually the wavelength of light we'll find in a green laser.

But there can be multiple different wavelengths of light in combination to make the eye perceive green as well. And that's what's happening when we see purple. When we see purple (magenta), we're never, ever, seeing just one wavelength. Because as we've said, there is no such thing as a magenta wavelength. It's not in the rainbow.

So take red-wavelength light, and blue-wavelength light, with the key absence of green-wavelength light, and we'll see purple/magenta. But green-cone stimulating light must be absent or we will not see magenta.

It's pretty interesting how what we see is not necessarily what's out there.

Another interesting one is gray. There is no "gray" on the rainbow. Any time we're seeing gray, we're actually seeing a bunch of colors at once. Which is weird because gray appears colorless. But depending on the colors present that make us hallucinate that grey, there could be as few as 3 colors or even millions. We just don't see them because they're stimulating our cones in such a way as to neutralize each other. So under natural lighting, an aluminum macbook will be grey and colorless. But walk inside a room that only has red lighting (so no white lights) and that gray colorless macbook is now a colorful red.

Anyways, what we perceive has never been a literal transcription of what's actually present. The brain takes in stimuli from the outside world, but it alone ultimately decides what we see. It shows us different things under different conditions. Different lighting conditions, under the influence of drugs, colors next to other colors, light arranged in certain textures, whether we're malnourished, whether we're daydreaming with our eyes wide open, whether we've just finished staring at an intensely bright light or a colored wall and are now looking at something else, all of it influences the colors that we actually end up seeing. The brain always has the final say in what we perceive, not the outside world. We're basically hallucinating 24/7, all the while thinking what we see is what actually is.

> Lastly: https://i.imgur.com/epXYhJd.png If you cover up the top half of the image, the bottom looks gray. But the strawberries look red.

The strawberries even look reddish to me if I crop the image down to just one of the red-appearing parts of one strawberry and zoom that to a reasonable size. Looking at the RGB values, I see that the exact gray (128,128,128) parts of the strawberries are surrounded by areas that are slightly different, usually in that they have a slightly smaller red component and maybe a little more green and blue, e.g., something like (118, 129, 129).

If I zoom in enough that I don't see those very slightly less red regions, the strawberries seem gray with no hint of red.

This works even when 80% of the visible image is pure gray in the central, with edge areas having slightly reduced red or slightly increased blue or green. I wonder why it works that way, instead of the other way around, which would be that the large central gray would be seen as gray, with the edges seen as being bluish/greenish?

I wonder if it has something to do with the distribution of colors in nature? Offhand, I can't think of many natural things that are a pure gray, so maybe we favor an interpretation that has the whole image having some color over one that has a large, unnatural gray area?

Several years ago I went to a James Turrell exhibit at the LA County Museum of Art. He had this one piece called "dark spaces" that is kind of like the experiment where you go sit an a dark (i.e. lightproof) room for like 10 or 15 minutes and then you start to see this 'blob' in front of your eyes where he's leaked in a really small amount of light. Anyway if you like weird things with light he has a lot of them. Since he uses light as his medium you can't get too much of a sense of them from a photo, you kind of have to see them in person.

http://jamesturrell.com/work/type/dark-space/

http://www.lacma.org/art/exhibition/james-turrell-retrospect...

There is a book called "Vision Science" (by Stephen Palmer) that is full of interesting stuff about how the brain interprets visual information.

One thing I liked (there were lots): you probably know that your eyes move around with jerky motions called saccades. But if our eyes move in jerks, why don't we observe motion blur (like a camera would have) while our eyes are moving? It's not that our eyes aren't affected by motion blur: the answer is that our brains are wired so that when our eyes are moving, we don't really see anything at all. You just kind of discard the input you get while your eyes are jumping around.

https://en.wikipedia.org/wiki/Saccadic_masking

> there are far more than 256^3 viewable colors

This is trivially true: the gamut of sRGB does not cover all possible colours. See this diagram: https://en.wikipedia.org/wiki/SRGB#/media/File:Cie_Chart_wit.... As examples, you can't accurately represent true (spectral) violet or darker/deeper but fully saturated yellows in sRGB.

> Lastly: https://i.imgur.com/epXYhJd.png If you cover up the top half of the image, the bottom looks gray. But the strawberries look red.

To me, the strawberries look red, the plate is a desaturated cyan, and the meringue is a very desaturated yellow, whether I cover the top half of the image or not.

>Lastly: https://i.imgur.com/epXYhJd.png If you cover up the top half of the image, the bottom looks gray. But the strawberries look red.

For me covering the BOTTOM half makes the bottom look grey. Otherwise it looks slightly red.

So the brain has its own adversarial examples, it's not just GANs that can be fooled.

"Would there be any interest in a series I've been thinking of putting together about all the fantastically counterintuitive ways your eye fools you without you even noticing?"

Yes

> fantastically counterintuitive ways your eye fools you without you even noticing

It does not have to be "fooling" when if fact it's just part of the processing pipeline.

> The fact that colors influence colors around them means that there are far more than 256^3 viewable colors: the spatial axis is like a fourth channel.

No it's not: it may as well be that the function applied to the perceived colours, while takes neighbours as an input - still outputs the same 256^3 colours.

> No it's not: it may as well be that the function applied to the perceived colours, while takes neighbours as an input - still outputs the same 256^3 colours.

Kinda like those post-processing shader effects (like blur etc) that look at neighbouring pixels to output a 256^3 colour value for the center pixel. The pixel value never leaves the 256^3 range, but its value is adjusted based on what surrounds it.