Hacker News Reader

Comment by jpatten | Hacker News Reader

Out of curiosity I gave Fable (on max effort) a CAD task yesterday, which was to design a space efficient carrying case for a set of fasteners in my repair kit for work. It used CadQuery to generate a STEP file. The result was pretty much exactly what I wanted, without needing any manual edits. I did go back and forth with it on the design, but was really impressed with the result. Without prompting it included nice touches like ribs on the bottom of the lid to stop fasteners from migrating to adjacent compartments, and the right tolerance for the fit between the case and the lid. This is a dramatic improvement from Opus 4.8.

View on HN

Well the thing about CAD files is that through reinforcement learning you can basically ask the AI to generate the CAD file an arbitrary item - say it’s a rabbit. It might have examples of this already in its training set and it’s essentially a similarity lookup - but for sake of argument assume we are giving it examples at the edge of the distribution (the whole point of RL). It guesses and you render the file. You pass that image to another AI (not being trained) and ask it if it resembles the description you gave the AI in training. If it does, you have a positive example. If it doesn’t, negative. In that way you can essentially apply transfer learning from the image recognition functionality to the description -> CAD functionality.

But is that actually spatial reasoning? Or is it effectively image generation? Because there’s a difference. Spatial reasoning implies that you could drop it in a video game, give it rules, and let it run. And it would play the game well. Like a flight simulator. That would be true spatial reasoning because spatial reasoning is not just identifying objects but understanding how they interact with one another in a highly quantitative way.

I'm not sure I see the distinction you're making between 3D design and other spatial reasoning tasks. You can use RL to teach navigation or video game play too. Does that mean these tasks are not spatial reasoning? Additionally, 3D CAD is all about understanding how objects "interact with each one another in a highly quantitative way." I mean, not in the rabbit example, but the container Fable designed for me holds around 30 different types of objects. It figured out a way to arrange them that was more space efficient than what I'd originally described. It considered the best way to stack the fasteners in each bin to pack them as densely as possible. It identified the risk that some very thin objects could slide between compartments in transit, and modified the design to prevent that. It correctly solved for the tolerance between objects that needed to snap together. These all feel like understanding how objects interact with each other. The model didn't just talk about these concerns, but created two 3d models for the case and its lid that accurately reflected them. I hadn't seen that before.

Reply on news.ycombinator.com