> ChatGPT has no idea what they're talking about. It lacks it's own thought and heuristic patterns, and only gives you the most-likely response to your prompt

Funny thing is, same could be said about lots of people. Try listening to any political debate.

Which leads me to believe that the thing that's missing from AI is the same thing that we miss in those political debates: ability to explain and justify your own thought process.

As long as AI is a blackbox, we won't consider it to be a real intelligence.