Hacker News Reader

Comment by ip26 | Hacker News Reader

It's still got all the noise and shadows frame-to-frame. It wouldn't be historically accurate, but it strikes me that an ML algo could probably smooth the noise and shadows, colorize, and even add frames into a more immersive video to bring Twain even closer to the viewer.

There's always the ethical and artistic arguments around colorizing and so forth, but if one of the values of old video reels is to connect our conscience to the past, it's my opinion that smoothing, colorizing, de-noising and so forth can be benefits in that goal, even if the frames are no longer "authentic". (It's not like the real Twain was choppy and black-and-white)

View on HN

I'd like to see ML colorize, sharpen, etc., older B+W movies. Silent movies would be greatly improved with the addition of a soundtrack (music[2], foley effects[1], and replacing the dialog cards with voice actors).

1. Yes, they already do add some foley effects. But very little.

2. Yes, they do add a music soundtrack. But it's almost always awful period music.

> the ethical and artistic arguments

Those are all baloney. Doing what I suggest does not destroy the originals in any way. It's simply making the movies more enjoyable. Apparently, a lot of purists want people to suffer watching them: suffer with grainy, blotchy film, difficult to make out flickering black & white, and bad piano music.

There's a reddit newsgroup where people colorize old B+W photos. They really do an amazing job. By colorizing key frames in a movie, ML can take care of the rest.

BTW, I've seen many bad attempts at colorizing old footage, like those "WW2 In Color" documentaries. You can always tell because the teeth & gums of the people are grey :-/ and the faces are a uniform brown. ML ought to be able to do a lot better.

>> the ethical and artistic arguments

>Those are all baloney. Doing what I suggest does not destroy the originals in any way. It's simply making the movies more enjoyable. Apparently, a lot of purists want people to suffer watching them: suffer with grainy, blotchy film, difficult to make out flickering black & white, and bad piano music.

I think there's 2 separate issues actually: there's the "artist's intent", and then there's refusal to utilize any modern technology. The latter I agree is baloney: if you can use modern technology to clean up and improve old stuff (such as taking an original Star Wars film print and removing the scratches in the film, improving flaws in the FX, etc.), I think it's ridiculous to argue against that kind of thing. Then there's the artist's intent: if the original artist intended that the movie have "bad piano music" as you call it, should you be changing that? It's like changing the dialog in Lord of the Rings so the characters all talk like modern American hipsters: how is this helpful? These movies are a product of their time, and the music is too.

If you could bring the original artists back from the dead, and ask them if they'd like to use modern technology to clean up their old silent movies to sharpen them, remove the noise, etc., I'm sure they'd agree: why wouldn't they want the print to look as good as possible? If you ask them about colorizing, there might be some debate; some might like it, others might not. It's not like they had a choice back then, but even today some people like B&W photos for artistic reasons. If you ask them about adding modern music, I don't think they'd like that much at all.

How many movie arteests today would choose black and white? A number indistinguishable from zero. How many would choose dialog cards over voice? Zilch. How many would make a film and says to the movie theater "play any music you want for the show"? Zip.

If I made a movie, I'd be pleased if people far in the future watched it. If that meant using their technology to improve it, that's fine with me. After all, I've suggested to many people that they improve my game Empire with modern graphics and AI and sound.

I don't think you can change a work that drastically without the change itself being a piece of art. And the original artists might not want to be left out of that process. See David Simon's experience with the HD remaster of the Wire for similar themes: https://davidsimon.com/the-wire-hd-with-videos/

The original artists for silent and B+W are all dead.

Arteests like… George Miller, Stephen Spielberg, Bong Joon Ho, Steven Soderbergh…

Look up Fury Road: Black and Chrome. And take a look at https://www.youtube.com/watch?v=Gz-30KI2bsw

Worth noting, among the “number indistinguishable from zero” are contemporary classics such as: Schindler’s List, Dead Man, Good Night & Good Luck, Ed Wood, Clerks, Raging Bull, Nebraska, et al…

Anyone who studied Media Theory will be well aware of the ubiquitous Marshall McLuhan quote, “The medium is the message,” and understands that the means with which a piece of media is produced and distributed tells a story in and of itself. Sometimes that’s in service to the broader narrative, and sometimes it’s simply a product of circumstance, but either way it carries its own value for the audience, and I would argue that it would be arrogant for one to unilaterally decide whether such things are important or not to preserve.

>> the ethical and artistic arguments

> ... Apparently, a lot of purists want people to suffer watching them

The tech and information of the time is worth preserving, as is the experience. I, for one, enjoy seeing the original, more analog versions of things. I cringe inside when I watch much of ML filtered or otherwise artificially colorized footage. Many films were shot with the intent of leveraging the strengths of the medium. Limitation often promotes its own form of innovation which can only be recognized in the context it was created in.

Saying that purists want people to suffer is quite misleading. A similar statement in the other direction might be, ML enthusiasts want people to forget what art actually is.

Ever listen to a Gregorian chant? They are cool, partially because they had no idea what they were (musically) doing... much like the product of an ML model today.

> Saying that purists want people to suffer is quite misleading

Not at all. I've watched enough movies where the director clearly wanted the viewer to suffer, based on the idea that only through suffering can the viewer be enlightened. The suffering is in the form of making a bad movie.

Solaris comes to mind. I've tried to watch it multiple times, but the director (and the director's fans) clearly revel in the notion that one must suffer through the movie, and that's the point of it. The movie is boring on every level - bad dialog, bad plot, bad cinematography, bad pacing, bad acting, everything is bad about it. And it's (small) number of fervent fans love it for its wretchedness. They can keep it :-)

I saw a colorized version of Metropolis a few decades ago. The technology to do it right wasn't there, but it was enjoyable in a way the original was not. I've since watched the full version when the missing footage was found, and there's an ideal film that would benefit from the application of modern technology. I bet Fritz Lang would love it.

TCM also ran a version of the silent "Wings". They added some color in a few spots, and some airplane engine sounds. Even just that little bit made the picture much more watchable.

I'd like to see the FM Busby musicals fixed up.

Some movies I wouldn't touch would be Top Hat. It's perfect as it is :-)

Did you like Stalker?

Never heard of it.

> ...I'd ike to see ML colorize, sharpen, etc., older B+W movies.

In case of old media, I think their historical value is not in their own existence but in their effect on people of the time.

Older movies were perceived back in time through the old technology, that is b/w, small screen, choppy, silent or with musical accompaniment.

It's a document, similar to language dialects used back in time - localizing this for present time would shave away the cultural skin it had for its time.

I can recommend a film festival and silent movies with live bands. If it works, it works great. For me the silent movie is always the highlight at the midnight sun film festival.

It's a bit closer to theater or reading a book. It's less directly realistic. Somehow your brain fills the gaps more and you get even more invested.

But if that ML tool is trained on modern video, depending on its capabilities it may subtly modernize aspects of the video in ways that lessen the "past is a foreign country" effect, leading us to believe that the past was more like the present than it in truth was. It's a trade-off between removing technological barriers between us and the person, so we can connect to their recorded legacy, and removing sociological barriers, which we probably want to preserve.

> leading us to believe that the past was more like the present than it in truth was

So what? I just don't get this. Can't we just enjoy a much improved film experience?

> so we can connect to their recorded legacy, and removing sociological barriers, which we probably want to preserve.

Nobody is suggesting destroying the originals.

But what’s the point then? If you’re going to make fake videos might as well just generate a completely new one with actors.

Would you rather see a colorized picture of your grandma or a color picture of another woman?

If the colors are just someone guessing I would honestly not want to see it. I find those kind of wishful fantasies harmful. Not much different than when favorite book is converted to a movie and actors stomp out your imagined characters.

If you hadn’t had a picture of your grandma at all, would you want AI to generate one based on a description of her and tell you it’s a picture of your grandma?

There is a short movie on Google which is a film of a train rolling into a station, and then the same one sharpened by AI. The second one is marvelous to watch.

There's another of a colorized ancient movie of a trolley ride in Europe. It's very fun to watch.

Ya know, when people restore old cars, if they want to drive them, they'll usually do a few upgrades to make the car safer and more driveable. And there's nothing wrong with that.

I'm curious. Do you ever watch and enjoy movies from the 20's and 30's? For the silent flicks, do you think that the music played during the screening by some random starving artist plinking on a crummy piano or squeaking away on a violin is part of the "vision" of the director?

Lots of people today simply turn away from any movie that is black and white.

I’d like to see rolling data on how people are turning away from lower quality media. Or what premium is being placed on 720p+ video versus standard.

There’s a reason video sites offer a filter for HD and YouTube videos get keyword stuffed for 8k even when they aren’t.

I am very curious how Apple’s spatial video lands.

From reviewer descriptions, the experience is such a major change from standard HD video I am wondering if depthscaling “old” video will become a thing too.

Personally, I find the constant breaking of the flow in a silent movie so the dialog card can be shown very irritating after a while. It's also annoying that the card is shown for too long (for slow readers) and there's a fair amount of dialog you can lip read that is not on the cards.

I seriously doubt dialog cards are part of any director's "vision".

The bad music slapped on the DVDs is also a serious turnoff. Music easily makes or breaks a movie. For example, the music on Lord of the Rings adds greatly to my enjoyment of it. The Hobbit music is utterly forgettable, and makes the poor movie even worse.

For another example, the movie "Hawaii" has a most excellent music soundtrack. Its sequel, "The Hawaiians", has a completely forgettable soundtrack, which contributes to the movie falling flat on its face.

I suspect much of "Star Wars" success came from the music.

“Fun to watch” and “historically accurate” are different things.

We’re not that far away from doing things like have AI remove “icky cigarettes” and “poisonous alcohol” to make it lots of fun for zoomers to watch.

If the colors are just someone guessing I would honestly not want to see it. I find those kind of wishful fantasies harmful

I think disclosure is high priority here. If you know it's been cleaned up, and colourized, then you can watch it as you would the original.

Because, what I think people are saying here is that older cameras already have made the equivalent of "someone guessing". The framerate is variable and jumps, the colour is off and variable, rhe film is degraded, and even the people in the films act quite unnatural, for they are very aware of being filmed.

I get what you mean about authenticity, but I think full disclosure takes care of that.

Another woman than a fake grandma

What they’re saying is that it’s your opinion that that would be an improved experience, and that’s not an objective opinion that other people share.

Because it's not real, red pill man

The past was more like the present than we like to believe. The past was no more sepia than it was black and white. Just like the 80s wasn’t as bold and neon as pop culture portrays.

In media we use colour to exaggerate the differences of the past, to make it feel different and foreign to our senses, rather than represent them accurately. So if you want a more accurate representation then you absolutely should be training your models on modern video.

I'll do this and let you know how it goes, I don't think I can easily add colour but I can work some other magic. ETA is likely 2-3 hours + whatever it takes to upload (assuming I get back to it tonight - otherwise I’ll upload tomorrow morning).

Looking forward to this. You can probably submit it to HN separately and it'll get heavily upvoted!

I haven’t tried enhancing anything this old and crusty before so don’t hold your breath!

Done yet?

Yeah I shared yesterday - https://news.ycombinator.com/item?id=37898748

It’s quite hard with such low quality footage. I think could be done is use some sort of algorithm to stop that sort of wavy chemical look that might be beyond me anyway that video did clean up a bit!

Very cool, it definitely gives an increased feeling of "being there".

First attempt has processed at 1080p 60fps, uploaded to YouTube - it will take some time for YouTube to finish processing it in 1080p - https://youtu.be/hlhn9HSH4OI

I think I can do better though, so will try another method in the morning.

Here is a colorized version .. but with all the noise:

https://m.youtube.com/watch?v=EIeNnDee-Ok

AI could do a 1080p with sharp details. Someone just needs to process the frames, and use something like DALL-E or stable diffusion with the right settings.

It’s not destroying the old footage to make it more viewable, so long as it’s preserved in its originality in some way.

Reply on news.ycombinator.com