Ask HN: Anyone working on natural language interface to 3D CAD?
This seems like a powerful next step for automated design and manufacturing? Thanks for contributing…
So it won't be just for CAD, but more general 3D (material creation, physics, particles, etc). And it is very early in my research, so I'm still unsure how far things can go, or how usable could a scene created by this technique be. But it would at the very least help discover features, and try ideas even faster.
The biggest win really is to avoid allocating memory when the scene plays. So as you set up your scene, I have all sorts of tricks to pre-evaluate what the scene will need, and allocate that once only. That's probably what made the biggest difference to keep a smooth FPS. But that's only on the javascript side, there is more pitfalls when generating shaders.
And for the CAD part, it's powered by opencascade/wasm, so that's reasonably fast for now. But I'm sure this will get battle tested soon, with larger models and nodes setups.
https://github.com/gd3kr/BlenderGPT - actually an attempt to make something useful, though its working with meshes and not step files.
https://github.com/JennyList/Breakfast-by-ChatGPT - project to get GPT to output OpenSCAD. Mostly just messing around but interesting attempt nonetheless.
If I were gonna work on a tool in that space, I'd try to get it running with eg FreeCAD macros or OpenSCAD before trying to get it to work with something like Solidworks.
For anything sufficiently complex, a natural language interface is going to be hard though. How do you get it to understand you want it toleranced in Y way vs X way? How do you get it to fix the non-manifold mess it created by destroying all the constraints when you asked it to fillet something? Not impossible certainly, but a lot of work for the library.
It's not using natural language, but now it could easily be modified to support it.
I suspect someone else is working on the accuracy necessary for actual CAD work.