Hacker News Reader

Comment by ge96 | Hacker News Reader

What I want is to be able to feed in a bunch of videos and generate an animatable (from talking) 3D face from that data. I suppose you in theory only need 3 images (front and sides). But mapping pixels to motion is interesting (facial expressions).

There wouldn't be depth data so it would be inferred from shadows

View on HN

Replicate has Hunyuan video training now. https://replicate.com/blog/fine-tune-video

Also, Kling 1.6 Elements works pretty okay if you use the same person/face for each element.

Kling also has lip sync.

Or this lip sync with replicate: https://replicate.com/bytedance/latentsync

Or there is HeyGen or D-ID or Synthesia, or tavus.io for full interactive digital twins.

Thanks

Why do you want to do that?

My case is not directly nefarious, for example an old popular YouTuber that streamed in the early 2000s taking their content and making a model of them for personal use like a 3D chat bot but with that person's quirks

Edit: when I say "nefarious" I mean you can use that tech to impersonate someone (eg. political reason) but for my case it's more the creeper type cloning someone for personal use eg. Replika

Tangent, the holo vtubers industry is interesting since they build up these characters with some unique persona/theme and then people follow that specific model, they could make themselves into an AI easily since it's a rigged 3D asset but of course it would be boring compared to the real thing

>they could make themselves into an AI easily since it's a rigged 3D asset but of course it would be boring compared to the real thing

The most popular vtuber on Twitch is an AI tho

You talking NeuroSama? I haven't kept up with it in a bit

I'm not sure if that's truly AI since the Turtle drives her

Edit: if the source was open I'd believe it

>I'm not sure if that's truly AI

It has always been a LLM. There is no human typing at insane speed to the TTS.

I'm referring to live interception of messages which I guess has to be done to be compliant with Twitch's terms -- there is a human there

edit: but yeah the fact that so many people interact with her shows generated content can keep people occupied

Reply on news.ycombinator.com