Colorful Image Colorization

This is amazing. Looking at the example comparison list, I love seeing the algorithm "correct" the color based on its biases. "No, people do not have pink hair: I made you blond." "Shoes are always a shade of brown (including black); the idea of shoes that are pink, purple, or even grey, morally offends me." "I don't understand why you would paint the frame of this window that crazy shade of green... let's go with a dark stain; doesn't that look classier?" "I've seen this insect before: I don't remember its name, but it was definitely brown, so stop trying to trick me into thinking it is bright blue with your silly Photoshop shenanigans." "Also: I've seen muzzles before, and they are made out of rusted metal; I don't know where you got this blue rubber-coated comfort muzzle, but back in my day we didn't have those extravagances." "On a similar note: I don't understand how your watch is glowing blue... back lights are supposed to be a dull and mottled yellow." "One red stripe on your truck is enough to make the point: three red stripes makes you look like you have no taste :/." "And why would you go to the trouble of forcing people to take a photo in front of a custom backdrop and not take the opportunity to print the name of your company in an eye-catching color, such as red?" "I am pretty certain all radiators everywhere are colored using the same aging white paint that has started browining from the heat; it gives a comfortable feeling of familiarity that your bright green radiator lacks." "Mittens are supposed to be festive, so why did you get grey ones?! I've colored them red: that should help."

One thing very noticeable in both works is also the tendency to overshoot borders when coloring. In many images it can be seen that the color of the main subject bleeds into the background. A good example is e.g. the fifth picture in http://richzhang.github.io/colorization/resources/images/exs... , where parts of the sky get assigned the same color as the mesa. This looks like something that should be fixable if one tries to detect these kind of edges and gives less weight to information from the other side of such an edge.

Does anyone know what the differences between this and the previous neural net approach are on a technical level?

Either this or it's an advanced April Fool's release (with back-dating and references)!

You can see some of the other results, including many there didn't work well at all, here: http://richzhang.github.io/colorization/resources/imagenet_c...

I shudder when I see someone/something colorize the works of masters of black-and-white like Ansel Adams or Henri Cartier-Bresson. Just a horrible mangling of their work.

Adams for example could have used color, and did take many hundreds of color photos[1]. But in his work for display he deliberately and consciously chose black-and-white and put in much effort in the lab to get the final images just they way he wanted them.

So while I truly appreciate the technological achievement here, I still cringe at the what I see as distorting the artists' work.

[1]http://www.smithsonianmag.com/arts-culture/ansel-adams-in-co...

Maybe there is a way to transform noisy recordings into ones that sound as if they were recorded in a modern studio. They could first try to emulate noise and distortions and generate a dataset of distorted recordings, then train a NN to reconstruct the original.

Aside from how impressive this is from a technical standpoint, I love that this algorithm seems to be a little colorblind in the human sense -- looking at the military trucks and uniforms in particular, it seems to have trouble with reds vs greens.