DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior | Hacker News Reader

DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior

mrtornado24.github.io

134 pointsphil9l29 comments

This is incredible.

All that remains is a network for rigging (and maybe retopology)

Indie gamedev would be revolutionized

There would still be a long way to go, I believe. Creating a consistent art style, with assets that fit into each other well for a whole game world would still be very challenging, for example.

We are definitely getting warmer. Exciting times. I would love a metahuman like diffusor for my characters. This would also be great for quickly generating props from photos or midjourney conjures. I still feel like all of these techniques and tools still require an artist, but the artists job is now more macro than at the micro. Awesome!

This is very impressive, the best one in the space I've seen, although I'd like to see some of the less cherry-picked examples.

Yup, I had the same thought. Click through the gallery to the last page, and you can start to see artifacts in the examples. The chicken & waffles one particularly. Although I must say, I am still extremely impressed by this tech.

Interestingly most of the artifacts, like the chicken and waffles one, are on the texture and not on the mesh.

Unlike other areas of AI that are immediately applicable to use in products (LLMs, Stable Diffusion, RVC, etc.), 3D sculpting has years of further optimization before it finds its fit. These sculpts are not "game ready". Their object geometry is massively bloated and inefficient.

They're not rigged with a skeleton or morph targets. That's a hard problem that some products (like rigify) can sort of solve, but not for assets in this poor shape.

The massive VC funds being poured into this "game ready asset AI" should sit on the sideline for 3-4 years and invest in a new player with a fresh cap table and no baggage.

I'm looking to jump into this area, but the timing is off.

But isn't the point of publishing intermediate results to present what is currently possibly and set the agenda for future inquiries?

> Their object geometry is massively bloated and inefficient.

This came out recently which looks like a leap forward in optimization: https://github.com/nv-tlabs/FlexiCubes

I mean, alternatively, the timing is now. It's a great time to get a head start on it, because once it hits market-readiness, there are going to be massive opportunities.

What causes all of these papers published on github without code?

Massive receptive AI audience.

The papers published on GitHub are typically presented in a much more accessible and beautiful way than arXiv.

The downstream social traction is further signal and affirmation for your results, lab, and/or company.

There is also Zero123++ which seems to be doing the same thing but has code available https://github.com/SUDO-AI-3D/zero123plus

EDIT: Just realized this one only outputs multiple images which are 3D consistent, not a 3d model.

I believe Zero123++ gives you the ability to generate consistent images from multiple camera views, where as DreamCraft3D seems to output 3D models with geometry and textures if I'm understanding it right. Like I could take the output of DreamCraft3D (if I had the code) and drop the result into Blender for example.

It is not unusual for code release to follow paper release with a lag of days to weeks. The top priority for a researcher is to get the paper out. Code is supporting material of secondary importance.

I was wondering why all of those papers have PDF downloads with the most useless, non-descriptive, names.

> Paper_high_res.pdf

Probably the same kind of oversight that just allowed me to compile hundreds of books in a catalog each as "book.pdf"

It was on exactly the second download where I realized how restrictive my choice of namespace was

The authors aren't thinking about you when they name the 87th revision of their paper.pdf They just want to get some sleep.

I used to give descriptive names to directories that would each carry a main document, and associated materials like references, diagrams, etc. Since the directory already had a descriptive name, I used to name the main document like "-- paper.docx" so that it stays on top when sorting the files by name.

However, realizing the problem you have mentioned, I started giving descriptive names to the main document also, in most cases by re-using the name given to the directory, still adding the leading "--".

The fourth line of the readme

> Code will come soon.

Proof is in the pudding, so to speak.

You know how many of these bullshit things we’ve had in the past year? Lots. All of them promise something, have a long (maybe even AI generated) research paper.

And,, most of all, NO CODE.

Wtf. Push your code. Or slse it doesn’t exist. I have a ton of these bookmarked. Still waiting for code.

I call bullshit.

I think you'll soon be wrong on this one based on the owners other projects

https://github.com/MrTornado24

All the rest have code. I'd expect to see it coming here in a few days

Any ETA for releasing the source code please .

i would love to test this for my current game project .

So, at the code link

> Code will come soon.

Disappointing, but also:

> 6 forks