I also use o1 and sometimes it just gets stuff confidently wrong. For example it often makes up command line arguments or optional parameters to methods or configuration keys. Anything that is even remotely niche, anything that hasn't been posted 1000 times on the internet, GPT will get wrong without ever telling you that it does not know.

Anything else, you can just google it because 9/10 there's a SO thread about it, and when there isn't, the documentation is usually good enough. GPT is not going to help you there.

But you can see it in the small things. For example if you ask an intelligent person "how do I install X on docker?" it will search google and then find a tutorial or some resource to reference. Then they will break it down and adapt it to your scenario.

GPT instead will give you... whatever it feels like giving you in that moment. It will not look on the f*n internet to make sure that what it says is correct and up to date. It does not update an internal knowledge base with factual information that it can then reference to produce a coherent plan. It has no concept of truth so it can't use logic.

You can throw as much compute and chain of thought as you want to the problem, it's the architecture itself that is disappointing.