Hacker News Reader

Comment by j_m_b | Hacker News Reader

What I want to see is a model that can generate 3D models for use in applications such as Blender. It would provide a good starting point for someone with talent to make beautiful. Or just save people like me time for making games.

View on HN

I've been looking into some of the 3D model generators this past week, and there is some work happening in that field. See the following non-exhaustive list:

https://github.com/snap-research/NeROIC

https://github.com/threedle/text2mesh

https://github.com/AutodeskAILab/Clip-Forge

https://nv-tlabs.github.io/GET3D/

Have you by chance tried out NeROIC? I'm a 3D printing enthusiast, mostly video game stuff, and it seems like like it would be excellent for that purpose.

I actually have been trying it out this week, and in fact it's currently trying to process the video generation, like their example shows. While I was able to follow their steps for training using their dataset, and generate the lighting/depth maps for the milkcarton example, the video generation is taking a long time (over 24 hours so far, using a 3070Ti with 8GB VRAM).

From what I understand with NeROIC, it's not particularly meant to be able to generate an 3D model that can be imported into Blender (or other software). It requires more work to take the meshes it generates to do something with it. See https://github.com/snap-research/NeROIC/issues/10

I too was looking into it to generate 3D models for some software I've been working on.

Thanks! I imagine one could use something like ZBrush's dynamesh to create a usable mesh from the output. Shame the library doesn't provide it by default, though.

> 3D

Then you're in luck today as this was just submitted: DreamFusion: Text-to-3D using 2D Diffusion

https://news.ycombinator.com/item?id=33025446

Funnily enough, you might not even need a 3D model if the txt2video is good enough. Whatever you wanted to render in Blender could just be rendered via text prompt (when this becomes 100x better).

I recently saw a company that seems to be doing just that[0], but I’m not sure if their model is open source

[0] https://www.kaedim3d.com/

Agree! And it seems like this should be the easiest thing to train!

Let the AI generate random blender files.

Render them.

Train on source => render mapping (which is 1:many)

Repeat.

You should be able to get an incredibly high-fidelity decoder to go from image to blender source.

Here's an attempt at doing that: https://dreamfusionpaper.github.io/

Reply on news.ycombinator.com