Hacker News Reader

Comment by ryandamm | Hacker News Reader

Can anyone who has seen these demos confirm if they are 6 DoF (the perspective changes with viewer position) or merely stereo?

The latter is boring and easy to implement, the former much more compelling and technically difficult.

View on HN

They are stereo using the MV-HEVC format https://developer.apple.com/av-foundation/HEVC-Stereo-Video-...

And while stereo video as a format is simple, IMHO it’s still quite a feat of engineering to capture it from the current phone generation. There have been stereo Android phones before but they were engineered from the get go with stereo in mind, and sacrificed other elements to achieve it

How can they be 6 DOF if they're recorded with two cameras?

AI magic. (They aren't 6dof but it is possible)

AI is not magic, and still constrained by math. If you need 3 variables and only have 2, AI isn't going to do it.

Unless we're talking about different things - it already exists. There's two parts to it - estimating a depth map ("monocular depth estimation") and inpainting the occluded parts.

They have a lidar. So they have very accurate depth. This gives some limited possibility at extrapolating and showing things from different viewing angles. I think this is also how they can give a good stereo experience from cameras that are closer than the average human IPD.

It would have to be AI generated, right? The cameras can't see behind what they can't see behind.

Maybe, but not necessarily. Good oldfashioned computer vision can go a long way.

When it comes to photography "AI" is a spice best used in moderation, cf eg https://www.theverge.com/2023/12/2/23985299/iphone-bridal-ph...

Only for objects up to 5 metres away, though those might be where the most is gained from accuracy.

Human depth perception only works up to ~7 meters so should be ok.

I like to imagine with 3D gaussian splatting of the videos. Not compute possible yet.

something like this? https://guanjunwu.github.io/4dgs/

Reply on news.ycombinator.com