See my comment in this thread about how ChatGPT thinks that sailfish are mammals. I possibly have never read the classification of sailfish explicitly in a book or a documentary, but I could have classified it as a fish any time of day. For one it's in the name, and furthermore I can imagine with high probability that baby sailfish do not drink milk from their mothers.

I don't know if the knowledge in an LLM is stored like in my neurons - I know nothing about neurobiology. But after having used ChatGPT extensively I can certainly tell that ChatGPT does not mobilize knowledge like humans.