Hacker News Reader

Comment by lvl102 | Hacker News Reader

What’s the status on companies building AI models to build actual 3D backend behind these generative videos. Anyone working on something similar? Imagine that’d be far more productive. For example, lookdev mlop is pretty low hanging fruit. Not sure why we don’t already have models from Autodesk, Epic or even Adobe (with resources ie A100/H100) where you upload an image/video and the model spits out workable 3D scaffolds.

View on HN

This is a good question, and the answer is that from a tech side it is surprisingly easier to solve the problem in the reverse direction.

As in, making workable 3d models is harder than making video.

And it is easier to make a 3d model by generating a video of the object instead.

Why is that? I don't know. But that's the current state of the industry. 3D model generation is simply harder.

I am thinking reinforcement on top of Blender would be straight forward with unlimited synthetic data potential. I’ve come across people incorporating SD into rendering workflow so tools are all there.

Probably also helps that there's way more image/video data to train on than 3D data.

If I’m not mistaken, Stability just released something like that a few days ago.

Yes, but it works by generating a video first and doing photogrammetry on it to produce a 3D model.

Looks like I completely overlooked threestudio released last year. Thank you for pointing it out.

It’s something I’ve been interested in too. I do a bunch of CNC woodworking so would love the ability to atleast generate close enough 3D models I can then refine.

Nerfs and splatting

Reply on news.ycombinator.com