> LLMs do not have corporeal experience. But it's not obvious that this means that they cannot, a priori, have an "internal" concept of reality, or that it's impossible to gain such an understanding from text.

I would argue it is (obviously) impossible the way the current implementation of models work.

How could a system which produces a single next word based upon a likelihood and and a parameter called a "temperature" have a conceptual model underpinning it? Even theoretically?

Humans and animals have an obvious conceptual understanding of the world. Before we "emit" a word or a sentence, we have an idea of what we're going to say. This is obvious when talking to children, who know something and have a hard time saying it. Clearly, language is not the medium in which they think or develop thoughts, merely an imperfect (and often humorous) expression of it.

Not so with LLMs!! Generative LLMs do not have a prior concept available before they start emitting text. That the "temperature" can chaotically change the output as the tokens proceed just goes to show there is no pre-existing concept to reference. It looks right, and often is right, but generative systems are basically always hallucinating: they do not have any concepts at all. That they are "right" as often as they are is a testament to the power of curve fitting and compression of basis functions in high dimensionality spaces. But JPEGs do the same thing, and I don't believe they have a conceptual understanding of pictures.