Hacker News Reader

Comment by aantix | Hacker News Reader

Why would we want an avatar vs a real video stream of the actual person?

Being able to have an avatar that fits your voice without having to actually look like that has many applications.

Whether you're trans or you just want to join a video call early in the morning without dressing up, the applications are endless.

In many situations we demand that people dress or present a certain way, just out of bullshit social expectations. This is one way to eat your cake and have it too.

For those use cases you should be able to get much more accurate results using a base video stream. This more fits use cases where you're lacking a video stream but not necessarily because you just don't want to turn it on.

A video stream isn’t volumetric.

This is for the metaverse.

I think metaverse is their primary target in making it as well, as I mentioned in a sibling comment, but modeling an avatar then rendering it is probably the easiest computational way of generating a video, even if the video itself isn't volumetric. See: traditional 3D avatars already in meeting apps which use the video feed.

I'm not sure if there would be any other potential use cases beyond these two. Or rather, I'm not able to think of them at so far.

I get that this is to drive the avatar, but I'm curious as to why. There's stronger signals with video, which I'm certain taking in even a very low resolution image would have stronger signals to convey movement than audio does (either the network is memorizing a lot (which is fine, but limited), or this is an iteration towards a 3D high sensitivity audio driven for precise sound? Something else?). I mean the quest has cameras in it, so why not use those? Computation? They aren't big models (largest is 1.42G, smallest 0.58GB)

Either games or its just interesting research that mostly ties in with what FB is doing. Cause there are problems like, e.g. imagine the bandwidth requirement of streaming 3D copies of like 20 people in a room

it's simply not possible within the near future, even today zoom/teams video conferencing is somehow highly compressed and shit quality with just low res 2D video.

You could generate the avatar clientside and save a ton of bandwidth vs a compressed video stream...

Because my avatar can more easily travel to a made-up photorealistic 3D render of an exoplanet

I’m imaging video game applications where the avatars are controlled by both online users and LLMs

Given it's by meta I'm guessing it's related to their metaverse goals.

Old recordings of people without pictures, for one!

Reply on news.ycombinator.com