I just asked Google's Gemini the following question:

Q: How many N's are there in Normation?

A: There is one N in the word "Normation"

Note that the answer is the same when asked n's instead of N's.

And this is but one example of many simple cases demonstrating that these model are indeed not reasoning in a similar manner to humans. However, the outputs are useful enough that I myself use Claude and GPT-4o for some work, but with full awareness that I must review the outputs in cases where factual accuracy is required.