If I ask a SOTA model to just implement some functionality, it doesn’t necessarily do so using a great architectural approach.

Whenever I ask a SOTA model about architecture recommendations, and frame the problem correctly, I get top notch answers every single time.

LLMs are terrific software architects. And that’s not surprising, there has to be tons of great advice on how to correctly build software in the training corpus.

They simply aren’t great software architects by default.