"How do you know we're not just next token predictors" is the thought terminating cliche . We know that's what LLMs are. It was certainly eye opening to see how far that gets you. But any deeper claims about intelligence or reasoning need real evidence or at least a proposed line of reasoning. "We don't know how intelligence works so it might be that" doesn't count.

It’s no more thought-terminating than “intelligence,” which is extremely loaded and causes people to make assumptions about these models that work backwards from the “intelligent” label rather than forwards from the tech itself