Hacker News Reader

Comment by jedberg | Hacker News Reader

On the one hand, as a lover of film, ewww gross.

On the other hand, as a lover of film, if this will bring more audience to foreign films, then let's give it a try.

This is very much in the same vein as colorizing black and white movies, pan-and-scan for video releases (and now zoom for 4:3 content on widescreen), and dubbing instead of subtitles. Every one of those things brought in more viewers of the content, exposing more people to those films.

View on HN

I want to be on board with the “bring more audience” but the truth is knowing how this industry works, any success with AI on this front will lead almost immediately to a total drying up of funding for professionals engaging in this work currently. I hope I’m wrong, I really do.

What professionals are doing this work already? As I understand it, this is something that can only be done by AI (or by shooting the movie twice in different languages).

audio dubbing and voice acting is a thing. They even do it in a way that takes into account the mouth movement length/timing, to make it look slightly more natural a dub.

of course, it's not super good imho, and personally i prefer subtitles with the original audio. Even if ai did a perfect job, including changing the film's frames to suit etc, i dont believe the outcome is sufficient. I want the original actor's voice timbre and intonations, which has meaning, and is lost when translated to a different language.

This is using the original actor. They re-recorded it. The AI just modifies the lips.

From TFA:

> Notably, the original actors recorded their own dialogues in English in a sound booth — Flawless AI's technology merely altered the movements of their lips in the movie.

Imo the real situation is that US distributors suck at dubbing and/or rely too much on celebrity voices rather than good voice actors that can dub properly.

I have seen some US dubs that would have been improved by a polish-style voiceover dialog

German dubs were (or still are – if an appropriate (time) budget is allocated) probably some of the best.

I dont get why anyone would want dubbing over subtitles?

There’s a reason ‘may your hobby hit mainstream audiences’ is an insult. Foreign films are foreign because they don’t attract a mainstream (read: anglophone) audience. As soon as there’s mainstream money, clueless producers will decide that they want that mainstream money, and make changes to accommodate said audience. This ends up turning them into a slightly different flavour of Hollywood.

Context: I think most recent use cases of AI is slop, generally people who identify as pro AI think I’m an AI doomer. But there are some good use cases of AI.

This sounds like a good use. It reminds me of the early days of CG, it took a while for it to look good, and now it’s a tool in the toolbox alongside practical effects. This is gonna look bad for years, but I think ultimately will be a good thing. Anything that helps shrink the world and understand other cultures is a plus for me.

But man, as a film lover I know this gonna annoy me over the next 5 years.

Agree on every point here, apart from your very last one. I am not watching movies with AI dubbing. Subtitles and original audio is fine. I do not speak a word of Japanese but I have never had issues at all with dubbed anime or films from that country. If this is a boon to anyone out there who finds foreign film utterly intolerable without it, genuinely, I'm happy that you will hopefully now have this option and I hope it's everything you need, but yeah. Hard pass for me thanks.

Cool, I also almost always prefer the subbed, but I meant a good thing for a wider audience for foreign films and TV shows. Imagine wider theatrical releases for, say, Let the Right One In, instead of a bad American remake. Think of possibly more money and recognition for the original creators. This is a good thing, even though the film lover in me is going to a visceral reaction against it.

Edit: I think I misread you and you actually agree with what I just said, so never mind. I have noticed with streaming more obscure movies sometimes dubbed is the only option, though

> I think I misread you and you actually agree with what I just said, so never mind.

Correct! I’m a very live and let live type of girl lol

If it looks good it's fine, but I'm not as optimistic as you.

I would argue that pan-and-scan failed long-term, perhaps even short term. I remember widescreen releases being available on VHS because a significant amount of people didn't like it.

Colorizing black and white movies...I can't think of any time I've seen that done in a commercially successful way.

I also think that a dub with lips that don't match is something people barely notice in the first place.

Did you watch the video posted in the article? Looks pretty good. The swedish actors recorded themselves speaking English and really the tech is just changing the lip movement. I think it would be more weird if they weren't speaking with a swedish accent.

I would argue that pan-and-scan solved a practical problem, and while it wasn't ideal, it was also just flat out necessary for a while. Your average TV in the '80s would have been around 20", and VHS only gave you 240 lines of vertical resolution. Letterboxing a 2.39:1 movie meant cutting both of those metrics in half. Preserving the original framing doesn't mean much when a sizable portion of your audience is stuck squinting at the screen.

Some filmmakers kept this in mind when making their movies in the '80s and '90s. Films like Terminator 2 were shot open matte, allowing each shot to be framed for 4:3 and the target theatrical widescreen aspect ratio simultaneously.

I think these facts got lost on a lot of people who were die-hard widescreen LaserDisc fans. LaserDisc had twice the vertical resolution of VHS. And if you were a big enough moviebuff to have a LaserDisc collection, you probably also had a bigger than average TV.

The rest of these techniques, like colorization, seem more about making older content more palatable to newer viewers, and don't really do anything to solve a technical problem.

It may have begun as a practical problem, but the fact that the opposite now happens (zoomed 4:3 content on widescreens which are all at least 720p and where letterboxing doesn't lose anything) tells me that there is enough of a population that doesn't like "the black lines" to warrant publishing those.

I wish they would just split the difference and zoom to 14:9. That minimizes the worst of the cropping with less intrusive bars.

David Simon went into great detail in his blog about why they had to completely re-edit The Wire scene-by-scene from source material to suit the HD format. There were scenes where the meaning was changed and details lost if they simply cropped without discretion.

https://davidsimon.com/the-wire-hd-with-videos/

By 2002 widescreen HD was clearly on its way. The video cameras weren't ubiquitous for shows shot on tape, but for film shoots, it's on them for not framing with awareness of a future HD release in syndication. The only excuse for composing shots in full frame 4:3 in the 00s would be retro-themed programs like That 70s Show.

I think you mean it's a technical problem. Language translation and screen size translation are both solving for the practical problem of reaching a wider audience and on more devices.

Subtitles and dubs already solve the practical problem of making content produced in one language accessible in another, and in a non-destructive manner.

This AI lipsyncing nonsense is more like the rash of colorized black and white movies that came out in the '80s, or the modern day scourge of 4:3 shows getting pan-and-scanned to 16:9. The "problems" these solve are viewers being closed-minded about watching movies and shows that don't have color or "look old". The final product is generally going to be inferior to the original.

My pedantry—like most pedantry—was focused on the problem, not the solution. Ergo, you could accurately say "AI lipsyncing nonsense" is a highly-technical solution to a practical problem, and subtitles a pragmatic low-tech solution.

Reply on news.ycombinator.com