Colorful Image Colorization

From the abstract: "We evaluate our algorithm using a “colorization Turing test”, asking human subjects to choose between a generated and ground truth color image."

And later in the article: "However, the results from these and other past attempts tend to look desaturated. One explanation is that [1,2] use loss functions that encourage conservative predictions. [...] We instead utilize a loss tailored to the colorization problem. As pointed out by [3], color prediction is inherently multimodal – many objects, such as a shirt, can plausibly be colored one of several distinct values."

This suggests that adversarial training might be a good fit. It transforms the goal from "reproduce the original colors", which is in general impossible, to "produce as convincing a colorization as possible", which is the real goal.

Demonstration of results: http://richzhang.github.io/colorization/#vgg_res, EDIT: Also, previous discussion: https://news.ycombinator.com/item?id=11403653

"Our method successfully fools humans 20\% of the time, significantly higher than previous methods."

I suspect this number will only increase over time, as humans lose the ability to recognize original colors in images thanks to Instagram and photo filters.

Maybe this could evolve into an artist's tool for digital painting, where you often do colorization as the final step in the workflow. As for adding back color to photos, I don't see much need for that today, except as a machine learning benchmark.

The examples in the PDF are impressive. I especially liked the colorized Ansel Adams photos. Those colorizations are quite good but seeing them in color only highlights how much more powerful they are in B&W.

It's interesting work, but what are the the applications?

Feed forward == machine learning ??