Yes. You're expecting too much. Generative AI models don't "understand" your problem, they don't even "understand" how to program. They're just fitting whatever data they've seen to your input.
To his defense, that is how it’s marketed. That this new model can reason.
Yeah, but "reason" is not a well-defined term. It means different things to different people in different contexts. It's just marketing speech. You can easily argue that all ML models, even those from 50 years ago, can reason to some extent.
Fully agree that’s kind of my point though. It’s a very tall order for some people. Like the OP