NPGA: Neural Parametric Gaussian Avatars – high-fidelity digital faces | Hacker News Reader

NPGA: Neural Parametric Gaussian Avatars – high-fidelity digital faces

arxiv.org

60 pointssamspenc14 comments

Their github page has some videos: https://simongiebenhain.github.io/NPGA/

Um, wow. These are really, really good. They are not perfect, but the improvements on fidelity, open mouth, eyes over a GAN-based approach are .. real high.

This is the first paper I’ve seen with videos that compare a person with a re-render side by side, and it’s a nice way to see what the model’s good at, and what it’s not.

Some perf numbers (which they say are unoptimized): 30-60 hrs on a 3080 for the avatar model, and rendering in the 20-40fps range on the same hardware. Basically good enough for a commercial implementation. They don’t mention latency of the CNN side that I can find, which is obviously a big question for chat scenarios, although not a big deal for pre-rendered scenes.

Agreed, very impressive results. It's both ~worrying and amazing that I'm sure an AI agent could just directly trace a path in the expression latent space to produce a photorealistic and real-time rendered head.

it feels to me like the gaussian-based algorithm improvements will likely force new render pipelines away from triangles sooner rather than later. It's hard to imagine giving up fidelity like this. And re-rendering to textured triangles is not fast right now. Should be a fun couple of years!

The real issue is the dependency in a real life enactment. Not even considering the performance.

In the novel Snow Crash, one of the characters, Juanita, does just this - she is known for making realistic facial expressions possible in the metaverse.

Given the now clear abuse potential (and other issues surrounding AI generally), why do papers like this discuss the motivation as if it’s a for sure requirement to do such work?

“The creation of high-fidelity, digital versions of human heads is an important stepping stone in the process of further integrating virtual components into our everyday lives.”

It’s not clear to me that’s a desirable outcome.

It is kind of a requirement. Scientific papers will typically describe the motivation for a work to start the introduction. The real motivation of course is “I need to publish something”.

The real motivation is "I want to research this field", publishing is part of the way. Nobody is doing research "to publish something", there are much simpler fields to enter if that was the motivation.

If you decide to research in $field, then you’ll typically need to keep publishing papers to keep your job

Indeed. That's what I'm saying - you decide you want to research and so you publish. Nobody decides to publish and so does research.

How is this a useful critique? The publishing being a subgoal doesn’t mean it’s not still a goal.

How is that a useful critique? Yes, researchers publish research.

I don't understand your question. There are some cases of abuse and so the researchers shouldn't discuss applications of the research?

Reply on news.ycombinator.com