It's also immediately clear to me when I look at the architecture of transformers that reasoning is not in the cards. I could be convinced otherwise if, again, someone showed me an indication of reasoning behavior. Since there is no such evidence and the systems theory approach tells me it does not reasonably reason, I have a pretty darn good reason not to believe it's reasoning.
I'm not saying that's incorrect, but thb that's exactly the tautology I was talking about!
If no statement is ever evaluated, it's not logical reasoning, because logical reasoning requires the evaluation of truth values of statements.
You could argue that the attention part of the network is some form of truth validation of the next predicted token? But indeed, the current chat interfaces don't change their previous text output retroactively.
Still, I am not really convinced. If we assume a human can reason, what does "evaluation of the truth value" mean? Thinking about something? This is still performed with our implicit mental model of the world, coming from shadows on a cave's wall, right?