I don't know how people can still be making this misguided comment. Predicting tokens well requires reasoning and planning.
"Brains are impressive and have their uses, but we shouldn't pretend like a muscle controller can somehow reason and plan."
"Brains are impressive and have their uses, but we shouldn't pretend like a muscle controller can somehow reason and plan."
Reason and planning regarding token prediction
It cannot reason about the context of its output. It only infers the most likely token to follow up.
Can you give me an example of a prompt that requires your definition of "reasoning"?
This is what I am talking about. It can reason and plan for what is the likely token based off its training data. It is completely unable to evaluate the logic of the code it was generating. It can not reason on the context (in this case, programming).
The same is true for other domains. Law, Medicine, etc. the stricter the field, the less reliable LLMs are, because it cannot reason about the context of what it is writing.
That said, I like LLMs, and I think it was an interesting productivity tool, only having been hyped beyond any reasonable expectations. I find it more useful for less strict contexts (for example, creative writing).
There are many examples of slightly modified popular riddles that are easy to solve by reasoning about them, but LLMs always fall back to the most likely output from their training data.