Hacker News Reader

Comment by BoredPositron | Hacker News Reader

It's still on the wav2lip[1] level from 5 years ago (look at the teeth) just slightly higher resolution. The only real player with a moat is probably flawlessai. [2]

[1] https://github.com/Rudrabha/Wav2Lip

[2] https://www.flawlessai.com/

View on HN

Actually, our work not only targets lip movements, it can also produce more realistic head movements and facial muscle movements. ps: It also achieved a good syncnnet score on 200 randomly collected wild image-audio pair. And it is open sourced.:)

I get exactly what you are doing but I don't understand what would be novel about it? It really looks just like a simple SD/wav2vec/insightface/animatediff pipeline, everyone can plug together in comfy or with diffusers. The muscle claim is also a bit dubious...

PS: With insightface models in your reqs the oss aspect is also pretty much void for any use other than research and your readme should reflect that.

The flawlessai is interesting, but even in their formal example for marketing there are weird tells. The actresses eyes are too jerky and weird during the changed sections. It's strange, but in a way that I'm not sure I could tag without the context of the rest of the video.

Is this what we're in for? Human movement that is just normal enough to pass, but not really natural enough to be comfortable with?

We're sure to be in an uncanny valley for a little while first, the real question is whether it is followed by a winter or some of that sweet verisimilitude.

What's SOTA open source?

Reply on news.ycombinator.com