My brother in law who is extremely smart, and very well trained -- MIT PhD -- and exposed to orders of magnitude more data than ChatGPT, given that the human senses ingest gigabytes of data per day, has -- like everyone else -- persistent errors in his thinking. I single him out because of his credentials, but everyone has these.
As a particularly funny example, we were talking about steers and bulls and he was like "Oh, right, I remember, some male cattle are born bulls and some are born steers". My wife, who is similarly educated, believed for a very long time that watermelon's grew on trees. At my undergrad school, a woman who was getting her physics degree and then who eventually went on to get a PhD, fundamentally believed that yeast would not rise if you talked around it -- I guess her parents had told her that and she never questioned it.
That is to say, humans lack full knowledge. We're trained on significantly more data than ChatGPT et al. We still have persistent inaccuracies despite all of that. We're not perfect.
That doesn't make us less useful. Half the time we just rely on other humans trained on similar, but slightly different data, to iron those out, and indeed AI systems that employ those methods get better accuracy.
Now again, my brother in law has fewer errors than most people since he is very smart. However, the vast majority of people are really not that bright (rumor has it that fifty percent have below average intelligence), and yet, simply through verification with other humans, they are able to accomplish useful things.
Sometimes even very capable people have persistent thinking errors. This doesn't make them less smart. No one can claim perfect intelligence. That's the only thing that's bullshit.