> It's not clear they ever will be;

It seems to me that the ability to reason ( though primitive it may still be) is an emergent property of sufficiently large differentiable hash/lookup tables (and if you think about the mechanism of reasoning, that seems... Reasonable).

According to "folks in the know", the scaling limits of GPT architecture seem to be nigh, but if it were a bit higher I wouldn't be surprised if LLMs would have made it.