Tiny camera shakes on a static scene gives away more information about that scene, it's effectively multisampling of the static scene. (If I have to hunch it, any regular video can "easily" be upscaled 50% without resorting to interpolation or hallucination.)
Our wetware image processing does the same - look at a movie shot at the regular 24fps where people walk around. Their faces look normal. But pause any given frame, and it's likely a blur. (But our wetware image processing likely does hallucination too, so it's maybe not a fair comparison.)
By introducing a new source of information (the memory of the NN) it can reconstruct things it has seen before, and generalise this to new data.
In some cases this means hallucinations, true. But in other times (eg text where the NN has seen the font) it is reconstructing what that font is from memory.
So if we are reconstructing letters from a known font we essentially are extracting 8 bits of information from the image. I'm pretty certain that if you distort the image to an SNR equivalent of below 8 bits you will not be able to extract the information.