I think a better argument for them not reasoning is that they are incapable -- seemingly, for now, that I know of -- of independently starting a reasoning task.
They do not ask whether or not they dream of electric sheep. Unless prompted to do so.
That's part of human reasoning, being prompted, but so is independent thought.
So at best they are partially reasoning, unless we philosophically argue these are two separate things. And frankly, shrug, I'm not a philosopher.
If somehow independent reasoning then we're getting very close to something alive that would need rights.
In my view.
And I have yet to have an experience with an LLM where I came away thinking -- this thing is alive and me using it this way is unethical. I've had that experience with animals, even people. Never -- yet -- an LLM.
I hope you agree that this question is too important to leave to gut feelings.
Humans have a poor track record in that regard.
This is a simple and intentional design choice though. Biological lifeforms are always on and always receiving sensory input. LLMs don't functionally exist out side of when we decide to run them. An always on agent with a looping prompt of "if you aren't doing anything else, ruminate" overcomes this limitation.
In fact, if your brain were removed from your body, and you were locked in a room that was completely empty and precisely calibrated to your brain's ambient temperature, and you were then ordered on your life to not ruminate. Well -- I would posit that you wouldn't last very long.
There's something else going on with human cognition -- and reasoning by extension -- that LLMs are, to date, not replicating.
With the big caveat of AFAIK. I'm not in one of the labs close to this stuff.
Why do the internal mechanics need to mirror how it works in humans?
I have agents that receive sensory input (video/audio) which run continuously processing it, receiving events as they occur. When something strange happens they notice and take action (i.e. there's an unrecognized person in the living room -> ask who they are.)
Presumably you've seen the agents that play video games - build things to progress in Minecraft, plan routes, avoid adversaries, etc.
Why doesn't that count?
I feel like you're failing to consider anything outside the chatbot UI.
EDIT: The first morning my agents had access to an outside video stream they commented on the sunrise. While I was sleeping through it.
> they are incapable ... of independently starting a reasoning task.
I have provided several examples of them independently starting reasoning tasks.
... and I'm wondering why you seem to be entirely unaware of such examples. The most likely explanation being you are only aware of the chat interface.
... and I have absolutely no idea of what you mean by "... and feel".
Or put it other way, lack of feedback loops between the “brain” and the environment.