Have you formalized this into a compiler infrastructure yet? I think Python on its own would be too slow to build complex parts, especially since for triangle mesh, accuracy inversely correlates to performance.
Vision is generally not the most reliable form of checks for LLMs, even on GPT 5.6 Sol, so a recommendation I would have is to instead emit JSON or CSV of the color/topology data for the LLM to inspect directly, and this is the instance where ray query for topology checking will greatly improve accuracy in general. SDFs are a bit more complicated right now, I have a full implementation designed for 3D analysis
My own experimental compiler generated mesh suffers from the spiderweb effect: it's very polygon efficient but not very friendly towards UV unwrapping in general, and I'm struggling to find the correct approach for that. If you have any suggestions, I'd love if you can point me towards the correct approach.
Definitely very interesting though.