Color-Diffusion: using diffusion models to colorize black and white images

It's easier to get a sense of what's going wrong with a pixel space model though. With latent space, there's always the question of how color is represented in latent space / how entangled it is with other structure / semantics.

Starting in pixel space removed a lot of variables from the equation, but latent diffusion is the obvious next step

The last time I checked, “the source is public domain” is not a valid defense against the pro-DRM parts of that law.

For instance, fake blood in B&W was often produced with black liquid. Colorizing it correctly just doesn't make sense. Or a green or blue dress can be chosen because of the way it looks on film, not because it's supposed to BE a green or blue dress.

Tbh most cost effective would be a conditional GAN though

> penny per frame

Where did this come from?

Also horrified.

The point I am making is that colourisation is subjective art, and that alone.

Colourisation cannot fail to enforce contemporary biases based on poor understanding of the materials. It will darken or lighten skin inappropriately, and mislead in any number of ways.

Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation.

Automating it is banal at best and dangerous at worst; colourised images risk distorting history.

But it reflects the fact that an accurate colourisation of a black and white image without access to every possible detail about the scene and processing from the photographer's perspective is impossible.

Black and white film is substantially more complex and varied than people understand. Its sensitivities are complex and vary from processing run to processing run, and people at the time knew of the weaknesses of black and white and often used false colour to get an acceptable rendition.

Colourisation is a form of expression, not a form of recovery.

https://github.com/TencentARC/T2I-Adapter

But also you have to consider that bias is being introduced in the colour rendition. That causes damage.

For example, you could see a photograph of an African American woman in the 20s or 30s, and your AI would say, this is an African American woman and colour her skin in some way.

But a lighter-skinned-looking African American woman in a pre/early-post-war photo is a challenge. She may have had darker skin -- been unable to "pass" -- and the film simply didn't get that across because of its colour sensitivity.

Or she may actually have been light-skinned and able to "pass" (or wearing makeup that helped).

Automatically colouring that image introduces risks to the reading of history; you can read that woman's entire life completely wrong.

It's also common with photos of men from that era who worked outdoors. Many of them will come across much darker-skinned in photos than they actually would have appeared in real life, because not-readily-visible sun damage can look odd in mono. But if you colourise all those sun-baked people the same way, what happens to those of mixed heritage among them? (A thing that is already rather "airbrushed out" of history.)

Without knowing about the lighting, the material, the processing and the source of the positive (is it a negative scan? was it a good one? or is it a scan of a print?) you cannot make accurate impressions of skin tone.

And given the power and importance of photography in the history of the USA in particular -- photography coincides with and actually helps define the modern unified US self-image -- this is not something to blaze through without care.

This is a far less tricky problem in more homogeneous societies, obviously. But even then, there is this perception from photographs that British women in the 1920s were all deathly pale; colourisation preserves that illusion that actually comes in part from photographic style.

Colourisation cannot be done accurately from a black and white image without context that is almost always lacking. Hand colouring is less dishonest.

I fully agree that being able to generate an aesthetically pleasing image with an AI that has been optimized to do exactly that is a banal application of creativity.

I do think that AI has incredible potential to make (and become art).

The best AI artists don't just throw art into midjourney, they experiment, create their own secret sauce.

Training models has become an art form in and of itself: ai artists curate incredible datasets and devise recipes for training stunning models. Their workflows span multiple companies / tools / models.

AI just means that the goalposts for creativity are shifting. Boring people will use AI to make boring art, artists will find completely unexpected ways to use the tools we build to create art forms we've never imagined before.