It is impossible to accurately imitate the action of intelligent beings without being intelligent. To believe otherwise is to believe that intelligence is a vacuous property.
If the idea is that something cannot accurately replicate the entirety of intelligence without being intelligent itself, then perhaps. But that isn't really what people talk about with LLMs given their obvious limitations.
But LLMs are already beyond that in writing code that passes actual tests, proving theorems that are check able with formal methods etc.
The people who still say LLMs are just parrots in 2026 will just keep saying this no matter what, so I don't think it makes sense to argue this point further.
Wait what? So a robot who is accurately copying the actions of an intelligent human, is intelligent?
If it's just basically being a puppet, then no. You tell me what claude code is more like, a puppet, or a person?
But that is the key insight, how can you tell when an imitation of intelligence becomes the real thing?
They might misremember, but they can know, for sure, if they have NOT come across some information. So if you ask someone if they know where `x` is, they might have came across that info, and still be wrong. But they will know if they have never come across it.
A neural network will happily produce an output when when the input is completely out of range of the training data.
False memories are super common. "I thought I had seen this thing there, but turns out it's not" is a perfectly normal, very frequent occurrence.
Just tell me..
---
Q: How do I reverse an array in the Navajo programming language?
A: I'm not familiar with a programming language called "Navajo." It's possible you might be thinking of a different language, or it could be something very niche that I don't have information about.
---
As for your question, the chances go from 0 to 100% depending on how many languages I already know and whether I have an idea (or I think I have an idea) of how python looks like. And LLMs have seen (and tried to "learn") pretty much everything.