Yes. The root here is that large language models produce human-like passages of text, but they are devoid of any specific intelligence, per se. They can sound quite convincingly like your average internet denizens, but fundamentally - and critically - lack the ability to differentiate between truth and fiction.

There are a lot of cool things you can do with models that don’t require you to differentiate fact from fiction. For example, creative prompts, and to some degree customer service chats etc. But these are not intelligence, not do they demonstrate anything resembling it.

It seems as though there are a lot of very smart people who have a hard time wrapping their heads around this, including, somewhat incredibly, some of the top researchers in the field. I think there is, in some researchers, a deep, Frankenstein-like desire for their creations to truly be something, and we see this manifest in unfounded, religion-like delusions. All despite these same folks carrying a deep academic understanding of the true nature of these models.