It could have been hallucinating, of course. But it does seem like it occasionally alters already generated words as it goes, at least I think I've seen it do that.
The slow progression of ChatGPT output is just a property of the output layer. The language engine doesn't work slowly like that, and once a token has been generated it can't backtrack.
In other words, it outputs one token at a time.
My take on why they have build the output layer like it is, is that next to feeling more human, it also forces you to be a bit more thoughtfull with your requests, and thus spam the system less. In the end it is still really expensive to run these models..
No, it's literally a for loop in python that runs the whole thing from scratch[1] after appending each new token. No artificial slowdowns, what you're seeing is literally what it's spitting out in real time
Here's an example
https://github.com/karpathy/minGPT/blob/master/mingpt/model....
[1] Some things can be cached such as kv, but still