Is this a good or bad thing? We hear "hallucination" this and that. You can't rely on the LLM. It is not like a search engine. But then you hear on the other side "it memorises PII".

Being able to memorise information is demanded when we want the top 5 countries by population in Europe or the height of Everest. But then we don't want it in other contexts.

Looks more like a dataset pre-processing issue.