Widespread gamma error in image scaling algorithms (2007) | Hacker News Reader

Widespread gamma error in image scaling algorithms (2007)

4p8.com

84 pointsjmillikin23 comments

The primary reason for this problem is twofold:

1. People writing image scaling libraries are big on speed. Many, like libswscale, are intended for realtime video playback on systems without graphics overlay acceleration. They contain truckloads of shortcuts to cheat wherever possible in the name of performance.

2. A "correct" method requires one of two things:

A. Performing one LUT per input pixel to the tapfilter, plus one LUT per output pixel (see http://news.ycombinator.com/item?id=1142084). This completely eliminates any possibility of SIMD. Other LUT schemes are also possible, like LUT[pixelX][pixelY].

B. Converting the entire image to linear 16-bit colorspace internally, then converting it back. This adds a slow preprocessing (and postprocessing) step and halves the speed of image processing.

Since both have huge speed impacts, most people writing graphics libraries don't like to do them.

Oh, and standard GPU-accelerated overlays can't really do either of these.

> Since both have huge speed impacts, most people writing graphics libraries don't like to do them.

Python Image Library does gamma correct rescale by default. I did not realize this until I wrote my own implementation. I thought it would be several times slower than the built in, but it was only 3% slower. Boy was I surprised when I figured out why.

I don't know why, but gamma is one of the most confusing topics in all of image processing. Most software screws it up, and I'm not surprised at the results shown. Unfortunately while I applaud the awareness-raising, the explanation here left my head spinning, despite writing code for one of the apps he highlights as doing it right! If you're in the same boat, I'd recommend Poynton's Gamma FAQ:

http://www.poynton.com/GammaFAQ.html

the explanation here left my head spinning

Then here's a simpler explanation:

We need to blend pixel values X and Y.

Normal image-scaling algorithms perform the operation (X+Y)/2, because this is fast.

However, the correct operation is ToNonLinear((ToLinear(X)+ToLinear(Y))/2), because X and Y represent a nonlinear scale, similar to how adding a 60db and 60db sound doesn't result in a 120db sound.

The approach that helped me is thinking in photons. 50% in pixel values is only 25% of the photons that 100% in pixel values represents (approximately). Linear light measures just means converting to photons before you do any operations, and then converting them back to the encoding.

I almost wish gamma was a more obvious encoding process, like gzipping a file, rather than a simple function. That would save a lot of conceptual pain.

Although the article is dated 2007, the error it describes is still present in every implementation I've tested. There's a test page for browsers at:

http://www.4p8.com/eric.brasseur/gamma_dalai_lama.html

Are projects like Gimp and ImageMagick not yet aware of it, don't consider it a bug, just haven't gotten around to fixing it, or what? In some brief trawling through bugtrackers and mailing lists I haven't been able to find any discussion on the subject, but it's somewhat hard to search on variations of the widely used terms "gamma", "scaling", and "bug".

Part of the issue is that people have been trained by two decades of Photoshop and expect the results of the 8-bit algorithms. Gaussian blur for example looks very different in linear light, with light areas of the image blooming out over the darker. If you've spent years working with the 8 bit gamma space algorithms, that's a bug. Instead, I've ended up implementing both algorithms, usually with the more correct one labeled as 'defocus' or 'camera blur'.

Rescaling and in general resampling is a lot simpler. These days it's pretty inexcusable not to do it in linear space. Here's my own exploration of that problem from a couple of years back:

http://petewarden.typepad.com/searchbrowser/2008/08/why-font...

Because human visual response is non-linear, doing operations like gaussian blur (which is an artificial operation, unrelated to physical phenomena) in linear light space isn’t “more correct” in any absolute sense. Calling a gaussian blur “camera blur” or “defocus” (whatever the color space) is just inaccurate and confusing.

Compositing and interpolating should definitely be done in linear space though.

> Calling a gaussian blur “camera blur” or “defocus” (whatever the color space) is just inaccurate and confusing.

Nonsense. When you defocus a camera, you get an effect that's comparable with a gaussian blur performed in linear space. Blurring is a well-understood operation that's existed for a long time in the analog world, before we ever did any digital processing.

Well, “comparable” insofar as you don’t care about the details, I suppose; you could also call a square or triangle blur kernel “defocus” if you wanted.

A real lens blur doesn’t look much like a gaussian blur for someone who is familiar with both and paying attention.

For explanation, see for instance http://en.wikipedia.org/wiki/Bokeh

Yep, my 'camera defocus' also included arbitrary kernel geometry as an option! I'm not trying to be obnoxious about this, but I really did spend five years of my life coding this stuff for Apple.

>human visual response is non-linear

Human visual response to color is also nonlocal, which has an array of interesting effects.

You can make ImageMagick do quite sophisticated image interpolation, if you want.

Pretty sure GIMP’s main goal is to just do what Photoshop does.

The video game industry is aware of this problem. Any pre-process mipmaping is being done is done with gamma-corrected downsampling.

What the article does not seem to be aware of is that most Photographic workflow products, such as Aperture and Lightroom, are working in a 32-bit colorspace even if you open an 8-bit JPEG. The reason is simply that to work in RAW consistently, your tools need to be working at a higher bit-depth than the existing RAW images you are trying to use.

Some RAW formats are 14 and some 12. High-end Hasseblads are 16. It is not unlikely that future sensors will go beyond 16 bit. Therefore it makes sense to design the workflow around 32-bit processing. Otherwise you are stuck re-architecting your entire workflow. Witness the number of Photoshop filters which only work at 8-bit.

Here's another strange effect: on my monitor if I look at the top picture while standing, it's a grey box. If I move my head down and look up at it, the grey is gone and it's a saturated image. I know my monitor's not great at different viewing angles, but this surprised me.

Yes, most LCDs are still pretty bad at keeping colors constant with viewing angle. To see the effects being talked about precisely, get a calibrated CRT. :)

Hrm.. I just tested out Gimp-2.6.8 and Eog-2.28.2 - both had wrong, although cool looking results :S

Sounds like a job for a photoshop plugin

What? Plugins can’t modify the behavior of built-in tools, and are therefore absolutely powerless to fix this problem, which affects all of the compositing of layers in Photoshop, and many of its other tools.

(As for scaling in particular: many plugins already exist using better methods than bicubic interpolation, and avoiding problems like the lack of gamma correction, etc.)

plugins cannot modify built-in tools, but they can replace/duplicate their functionality

Somehow the whole thing reads like pointless wanking to me.

But I'm not a graphics designer or whatever

Reply on news.ycombinator.com