This is wrong. Human thinking and speech isn't autoregressive like LLM inference.
> LLMs appear to learn distributions of representations, they both develop a hierarchy of those representations, both have early layers that process simple features, with later ones processing more abstract concepts, both predict missing information.
This type of superficial comparison isn't very meaningful, it's trivial to liken anything to a human biology in this manner.
A plane and a bird both use wings to produce lift, it doesn't then follow that a bird and a plane are meaningfully similar.
The use of Bernoulli's principle to achieve lift is a fundamental and meaningfully similar function of both airplane and bird wings. That functional similarity is well known.
> This type of superficial comparison isn't very meaningful
The comparisons I provided are fundamental to both the human mind and LLMs.. that's pretty darn relevant.. and whether you find that trivial or not is a matter of opinion.
Natural language is, however, a prerequisite for the existence of LLMs. It’s more similar to methods for storing and retrieving information, like the printing press or a database, than it is to a sentient being.
That’s not to say that LLMs can’t do crazy things, because they already have. Our language can encode a whole lot of information, and it’s incredible that we’ve found a way to distill that so effectively.
I think they’re not talking about cognition, but about output: regardless of what may be happening inside your brain, ultimately one word at a time comes out of your mouth, right? And you can’t then unsay it.
When you put it in those terms, LLMs are in exactly the same boat.
You're conflating being autoregressive with being sequential.