Hacker News Reader

Comment by avaer | Hacker News Reader

In case it's not obvious, this isn't a model, this is python scripts that call out to models (code not available).

It's mostly "did you know you can attach an LLM to a PCG system?", but there is one idea here you don't see much of: an image model performs the composition (which image models are really good at), and then you extract the objects into 3d via things like SAM3D before placing them in the world, which is pretty interesting.

The rest is standard stuff you'll find in your favorite PCG system/game engine. Still, think most people don't realize how good LLMs are at 3D these days, especially Fable 5 (which this project predates).

View on HN

LLMs are a PCG system. This is one of my pet peeves, I've been working on a procedural generation constraint system for quite a while (way before LLMs) as a hobby, that came with a domain specific language, where you could specify constraints.

Things like the red key must be reachable before the red door, and there should be no path that bypasses the red door, etc. written as a set of constraints.

I was working on is because I realized, constraints tend to be similar in nature (graph cut, minimum/maximum shortest path etc.) algorithmically, but its usually quite hard to control designer intent.

The reason I shelved the project, was I asked an LLM a couple years ago, to generate a dungeon room graph, following a similar set of constraints, but stated in plain English, and it did a perfect job.

Which brings me to my final point. LLMs are amazing and flexible constraint solvers, and are woefully underutilized in scenarios like this.

The trouble is the LLMs non-determinism and the "You're absolutely right, I placed the red key behind the red door rather than in front of it. This was load bearing for the whole dungeon and I blew it" moments.

With a hand authored constraint system if the red key is in the wrong place you have a bug in the constraint system that can be deterministically fixed, with a LLM you have to roll the dice on prompting again and go through expensive verification.

Very humorous and accurate contemporary AI slop quote, thank you

I had a validation script in the background that checked the generated graph - constraint based PCG often fails (it gets into an unsatisfiable state) anyways, and you have to (partially) retry it.

> Still, think most people don't realize how good LLMs are at 3D these days, especially Fable 5

Can you explain more? Can I use Fable 5 to generate a 3d model from a couple of photographs?

> most people don't realize how good LLMs are at 3D these days

It seems to work well generating static 3D models. However typically with way too many triangles and no skeleton unless it is a human.

That's my experience anyway. I would be happy to be proven wrong.

Reply on news.ycombinator.com