Can we say humans also hallucinate and continue parroting a bad idea for 1500 years? How does this support the concept that humans have real understanding unlike AI? Looks to me we have hallucinated theories that eventually get invalidated. We don't really understand, we're like hallucinating LLMs testing their outputs in the world.

Similarly to this we had no idea about the germ theory of disease even when 30% of our population was dying (Black Death). A life and death scenario and we couldn't figure it out, that's how intelligent we are in reality - not enough to save our asses. Our intelligence is social, an environment based discovery process not something coming from a single brain.

All we know comes from the environment, and no agent is too smart alone. We should reexamine intelligence as a property of the brain, better see it as a property of language.