This is an interesting question where the answers feel tied to the training data, because with 2D, you primarily have declarative data with HTML/CSS, frontend frameworks, scene graphs. With 3D though, I would think most of the training data is imperative, because of libraries like Three.js and Python scripting for Blender.

For your first point though, I don't think raw vertex data necessarily needs to be the output, it could instead be based on shape primitives with raw points as an escape route, with an architecture similar to the one Quiver/StarVector uses for SVG generation. I think llama-mesh and mesh-gpt prove this semi-viable for 3D?

Even some of those smart 3D object ideas you mentioned could still work with declarative, where you build constraints and resolvers and such into the compiler instead of treating each design as a program.

In my own experimentation with 2D, I haven't actually seen a major gap between the outputs from the agent for imperative vs declarative but I'm curious if you've done any testing with declarative languages for 3D and whether you see a quality gap. It should in theory be much more token efficient?