How to implement search-by-color when all you have is a good coffee

Using the distance of the colors in a Euclidian space makes sense, but RGB isn't the best space to use.

You probably want something like CIELAB which is (quoting from Wikipedia) "designed so that the same amount of numerical change... corresponds to roughly the same amount of visually perceived change."

https://en.m.wikipedia.org/wiki/CIELAB_color_space

This seems like a really fun problem and I think you found the solution with the best cost/effort ratio. The first thing that sprung to mind on seeing the problem for me was to use the fact that products often come in several colour variations but use very similar images for each. If you mask out the parts of the image that are similar you can get the single 'actual' colour as it might be labeled by a human. This would also allow you to search for neutral tones like grey and black.

And seeing as you've gone to all the trouble of figuring out how to grab dominant colours from images, why not allow users to upload an image of their logo and grab the dominant colour from that? This could help avoid the problem of having noticably different shades, without making the user do anything technical like searching with an RGB value and a threshold.

You'll get better results using a perceptually uniform color space [1]

CIE76 is not ideal but would let you continue to use CUBE.

You could consider doing a first pass with CIE76 and then sort by something more accurate in a second pass.

[1] https://en.wikipedia.org/wiki/Color_difference

> I just took the euclidean distance between the given color and (0,255,0)

If you used HSL, you possibly could match on the hue while being more lax on the brightness and saturation—so a pastel-pink color could still match to a red image. Not sure if the database can use different weights for positions in a vector (though with pre-processing S and L could be collapsed to narrower ranges, so they match more freely afterwards). Just need to make sure that white and black aren't matched to random hue values.

Though, I guess if you use only the distance to fixed chosen high-saturation values then it's about the same thing.

Except it seems that with just RGB distance, a semi-saturated cyan or yellow color might be counted as closer to ‘pure green’ than dark-green. Something like (0,255,180) vs (0,70,0).

Also:

> values('(${ color })'

2020 and still no placeholders.

I'm only marginally technical, but I would like to congratulate the author for a great example of how to explain the issues, the sequence of problem solving and the solutions so clearly and entertainly. Very good writing.

This might be a dumb question, but what does coffee have to do with this task?

the palette approach is a little reductive. we've built a solution in this space, a more robust approach would be to get the dominant colour, and from that colour brightness and tone extract the accent colour and the contrast colour by considering both distance from the dominant and extent on the image, that solves the problem of having a weighted palette ignoring, say, a red stitching because it's too small

I wrote a tutorial a while back on how to do this in Postgres using the Google Cloud Vision API in case its useful for anyone..

https://medium.com/@adamgross_6978/getting-started-with-json...

I like the color pallet option you provided showing the primary colors. Adding the colors as facet options would be interesting since you can align each of the colors with the items, then maybe a numeric metric you can use to sort the results by the percent coverage of that color.

Another item to consider is also taking into account how the user might search - "red shoes", then classify the query as "color: red" & "shoes".

The next fun challenge is determining which image to display if you have a variant product (color/size combination for clothing for example). Figuring out which product image to show in the result set requires identifying the primary color in the search.

Always happy to learn about more good ways to use PostgreSQL!

Mike Bostock (the d3 creator) has a nice little demo illustrating the use of linear RGB to assign an average color to an image. [1]

[1] https://observablehq.com/@mbostock/image-average

Vibrant.js [1] also does this.

It is nice to look into the source code to see how it is done.

But in my experience it works around 80% of the time because there are so many exceptions.

[1] https://jariz.github.io/vibrant.js/

There's a classic injection attack when interpolating in the SQL query

Also as others have mentioned RGB space and human perceptible color space is different enough that distance in RGB doesn't equate to difference/similarity between colors

Impressive. I wasn't expecting that approach! The whole thing implemented as a stored procedure in PosgresSQL a data type (cube) I didn't even know existed. That was a wild ride for me.

Great article. However in my experience, using the Euclidean distance in RGB space can bring about some really weird visual 'matches'. The best color space I've found so far for this type of task is the LAB color space.

And there are also other tricks of the trade. But for starters this simple solution might be enough.

This solution also assumes the colors in the images are 'calibrated' and are consistent. This is almost never the case, but may be overlooked in for the given use case.

I don't believe coffee (good or bad) contributed to this article.

Good article, and I like the implementation.

I am sure it has some drawbacks, like distance on the rgb space may not be the best option, or that it does not ignore background color of it is not transparent, still, I like the way of thinking

What does the title refer to?

I really like this title: "Can we implement something easier than a Convolutional Neural Network?"

Nice to see people still remember problems can be solved with anything else than, ehm, "AI".

I thought this was going to be about how coffee is a consistent color so you can white balance your customers uploads by asking them to photograph coffee in their ambient lighting.

xkcd did a perceptual color survey a few years ago: https://blog.xkcd.com/2010/05/03/color-survey-results/

It's population specific to the type of people who read xkcd but it's an interesting starting point if you want higher resolution info on, eg: exactly where people draw the line between "green" and "blue".

Raw data is here: http://xkcd.com/color/colorsurvey.tar.gz

Nice write-up, thank you.

Nitpick. What's the point of writing a good guide with rich text and images, but showing end result only in video? I have embedded YouTube videos disabled by default, so it's a little embarrassing.