Porter/Duff Compositing and Blend Modes (2013)

The classic reference is the A-Buffer by Loren Carpenter (SIGGRAPH 1984).

One interesting possibility is to go from a binary matte or averaged anti-aliased layer to subpixel geometry by generating a contour (e.g. Marching Squares).

Then the coarse layers (e.g. green screen) can be incorporated into a high-resolution A-Buffer rendering pipeline with 3D CGI elements.

https://graphics.pixar.com/library/Compositing/paper.pdf

This research was sponsored by Lucasfilm because at the time, alpha compositing of bitmap images wasn't really a concern for desktop computers. CPU, memory, and display hardware wasn't there to do anything useful with alpha (other than 1-bit masks used for game characters and GUI icons).

OTOH Lucasfilm (and other film visual effects companies) had a long history of using mattes, the physical photography equivalents of alpha compositing. When you see a group of Stormtroopers marching inside a Death Star hangar in 1977 "Star Wars", the hangar is a matte painting. They were typically created by artists directly on a large glass that was carefully placed in front of the camera to create the illusion.

Porter and Duff came up with a compositing model that's appealingly simple: it's a single formula, three variables, and you get 12 compositing operators by simply enumerating the combinations.

You can still find these Porter-Duff operators pretty much anywhere that 2D graphics are processed. They're in HTML Canvas, PDF, Apple CoreGraphics...

But, as neatly mathematical as these operators are, most of them aren't really practical for rendering actual scenes consisting of graphics layers. They're not terribly intuitive for designers, and they don't even attempt to simulate the effects of combining light.

So when Photoshop was created, its designers needed a set of blending operations (initially for drawing tools, and soon after for combining layers, which were introduced in PS version 3.0). Exposing the Porter-Duff operators to designers in the Photoshop GUI wouldn't have been very useful.

Instead they came up with a set of blending modes that were loosely derived from operations that you could do on physical photography negatives, since that was the inspiration for the application. (The PS blending modes still retain names like "dodge" and "burn" which refer to darkroom techniques that no designer aged under 50 probably remembers.)

These PS blending modes are conceptually pretty much the opposite of the Porter-Duff modes: they're not mathematically rigorous, but just a collection of algorithms that could be executed in reasonable time on a 68k processor back in the early 1990s. These original limitations are visible in the design of the algorithms: for example, they don't even try to account for gamma correction in the source images, which means that the algorithms do a rather poor job of simulating the behavior of actual light. (Photoshop has later added "linear" versions of some blending modes to address this.)

Still, they are the closest thing we have to a standard for blending modes. Many 2D graphics APIs that originally only supported Porter-Duff have added support for PS modes too, including the aforementioned PDF and CoreGraphics.