If you're going to write code, have the decency to do it on punch cards. If possible by hand punching, rather than using a keypunch machine.
The point I am making is that colourisation is subjective art, and that alone.
Colourisation cannot fail to enforce contemporary biases based on poor understanding of the materials. It will darken or lighten skin inappropriately, and mislead in any number of ways.
Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation.
Automating it is banal at best and dangerous at worst; colourised images risk distorting history.
Well, faces still have a certain tint, the sky is mostly blue, the grass green, water is blue, mud pools are brown, the ground too, a lot of historical fabrics are certain inherent colors, known flowers have known colors, brownstones have red/brown color. A lot of it, is just not that subjective.
Besides different color film stock (or camera sensor "color science") can already result in dozens of widely different colorings of the same exactly scene.
Do they? A certain tint?
You cannot accurately colourise skin from photographic film without an _enormous_ amount of knowledge of the taking and processing of the film, and of the lighting and subject.
An AI can't do it any better than a painter. You can't take a scan of a print or a negative and get skin tones right.
Think about how weird the skin tones are from scans of wet-plate photography plates compared to the same process used in antiquity with the aim of producing a carbon print.
Yes. There's just not a single one across all faces - but I wasn't meaning that.
What I mean is, we know the kind of tints a face will have. A face is not suddenly going to be blue or green or poppy red. And by how light a black and white face appears, we can tell quite well if it's a darker one (oilish to brown) or lighter (pinkish towards more pale).
If we get it wrong within a range it's no big deal. Color film stocks would also vary it widely.
Hell, even actual people who met the person we colourise in real life will remember (or even experience in real time) their face's hue somewhat differently each.
This is an enormously important issue.
Black and white films of different technologies and manufacturers and eras actually lighten or darken skin tones. Really very significantly.
And it's not going to be obvious from the final positive, unless there's _extensive_ data with those images about how the photography was done. And there never is.
Editing because I can no longer reply: the question of whether a skin tone is a dark one or a light one has had severe real life impacts on people whose lives are now only represented in photographs. You can't write this off as micromanagement; it's about the ethics of representation.
Is it?
If 2 colour film stocks took the same image of them, it would show their hue a little (or a lot) different.
Even if two different people actually met the same person, they will probably describe their face as slightly different tones from memory. (And let's not even get into different types of color-blindness they could have had).
Hell, a person's hue will even look different to the same person looking at them, in real time, depending on the changes in lighting and the shade at the scene as they talk (e.g. sun behind clouds vs directly sun vs shade vs bulbs).
It's not really "enormously important" to micromanage the (non-existent) exact right brown or right pink.
You shouldn’t write this off as micromanagement; it's about the ethics of representation. It is better to leave the original image uncoloured than to colour it automatically based on some fundamentally ill-informed model.
Hand colouring that image based on individual knowledge (for example that someone could or could not pass as white) is ethically better, if colourised images are needed.
Important nuances of culture and history, important and complex stories of discrimination and survival, are damaged by automatic colourisation by models that have no knowledge of the source of the mono image they are colourising.
Which makes it mostly american baggage. Other places who didn't have that history don't have much of an issue with whether a person is shown this or that exact shade in a photo, as it doesn't change anything, the same way making a white guy a little pinker doesn't change anything.
If anything, an AI trained on a large and diverse dataset is probably going to wind up being much more accurate with regards to skin color than a human colorist would be in most cases.
The problem here isn't whether colorization is done by man or machine; it's just ensuring that colorized photos are identified as such. Which they usually are -- that's not a new problem to be solved.
A diverse data set of black and white images doesn't have any kind of knowledge of the colour sensitivity of the medium in that moment.
What film was it? How was it processed? Is it a scan of a negative or a print? What was the colour of the lighting? Was a particular colour tint filter used on the lens? Was the subject wearing makeup optimised for black and white photography?
The black and white image, standing alone, cannot tell you this, I think. Sure, it might get a bit better at, say, identifying a 1950s TV show. But what is the "correct" accurate colour representation of that scene, when televisual makeup was wildly unnatural in colour?
And the dataset an AI is going to train on should be using original color photos that are then converted to B&W across a wide variety of color curves. So it should be fairly robust to all sorts of film types. So again, I repeat that it's probably going to wind up being more accurate with regard to skin tone than a human (with their aesthetic biases) usually would.
No, indeed. Which is why doing it by hand is more respectful of the notion that it is subjective.
Automatic colourisation is and will be viewed differently, as more "scientific", when it's still absolutely beholden to the same biases and maybe misconceptions that we can't unpick because they come from poor training data.
Finally: "original colour photos" are also a problem. Not only for the part of the history where they don't exist. But also for the part of history (until the early 1960s) when the colour rendition of those photos was false or incomplete. You can get a little closer to understanding what that colour looked like, but it's important to understand that colour emulsions vary in the way they work: it's not black and white film with extra colour sensitivity.
So at best you will be colourising the black and white film to look like the colour film, which is not reality. And there are well-understood problems with correct representation of skin tones with colour film until the mid-eighties.
I can see your point; I just think there's a bigger picture here (pun not intended) that you're not seeing.
Then the solution is to correct that misperception, not deny ourselves a useful tool.
> I can see your point; I just think there's a bigger picture here (pun not intended) that you're not seeing.
My overarching point is that this is a tool like any other. And the idea that "doing it by hand is more respectful of the notion that it is subjective" I will push back on 100%.
There is nothing disrespectful about colorizing a photo, automatically or by hand. But it should always be clearly communicated that it is subjective not objective, whether human or machine.
Again, if someone believes the colorization is somehow "real" or "scientific" because a computer did it, then correct their misbelief. Don't stop using the tool. That's the bigger picture here.
So if you colourise an image of someone who appears to be a light-skinned 1930s African-American with colours that appear to conform to our contemporary understanding of light-skinned Black people of our era, you might be getting it right, of course.
But you might be getting it quite, quite wrong, in a way that matters.
The model doesn't need to touch the lightness channel at all, only predict the noised added to the color channels at train time.
At inference time, we start with a real lightness channel (b/w image), and initialize the color channels to random noise. The model iteratively denoises the color channels while keeping the lightness channel locked.
No, doing it by hand doesn't acknowledge that your interpretation is a fallible interpretation shaped by bias, just like translating a written work (e.g., the Bible, for a noted example where this has been done often without any such acknowledgement being conveyed) by human effort doesn’t do that.
Acknowledging bias in translation of either kind is an entirely separate action, orthogonal to the method of the translation itself.
There's a lot of irony in acknowledging this but not acknowledging that each and everyone of us has their own biases inherent to our perception and experiences.
Like the blue and white dress; we all perceive things differently even on identical images, monitors, screens, etc.
You're imagining this irony to suit your personal requirement that I am wrong or foolish.
But if you look at what I am talking about elsewhere in my comments on this topic, it is human bias that concerns me.
One of the things automated colourisation cannot get right is historical depictions of human skin. In a way that really matters.
Human biases will creep into automatic colourisation because they can't _not_ creep in: the test data cannot fully describe the subject matter so contemporary bias will take over.
One of the areas where this really matters is historical depictions of Black people. Automatically colourising a black and white photo of a Black person's skin will almost certainly get their skin tone wrong in a way that might well very significantly misrepresent their history.
The same is true in mixed cultures all around the world; colourism is as much an issue as racism.
People had different lives based on how their skin tone was perceived. Automatic colourisation will not (cannot!) automatically produce a colour image that fits that experience. Because dark skin can appear light (and light appear dark) depending on complexities of reproduction.
This stuff matters. Hence my position: if you wish to ethically colourise an image, first consider not doing it at all. Second, consider doing it by hand based on real knowledge of the subject (their lived history etc.).
I am fully cognisant of the bias issues here (well, as fully cognisant as a white amateur student of photographic history can be)