How do we know which sources contain factually right things? What happens when the facts change? It used to be a "fact" that the Sun revolved around the Earth, and that stomach ulcers were caused by stress...
One is that objective truth is internally self-consistent. If one AGW denier claims it's the sun, and another AGW denier claims the NASA falsifies the data, and they support each other, then you can judge these are conflicting claims and decrease your trust.
Also, false claims usually focus on attacking competing claims than to come up with a coherent alternative. And they tend to be more vague in specifics (to avoid inconsistency), compare for example vague claims about all scientific institutions faking data vs Exxon files containing detailed reports for executives.
"Statistically correct" gobbledygook that signifies nothing.
...in the context of the texts that the LLM is built on. Not in the context of the real world, where P('Peru is a country') = 1.0 and P('Peru is a cat') = #cats named Peru / #things in the world (or something).
But if a LM looks up a topic and sees contradictory answers, and none of them is much more reputable, maybe it can still use that information to say it is inconclusive. Knowing a topic is controversial or not present in search engines is useful information. chatGPT would hallucinate, a search + chatGPT solution would refrain from hallucinations. It could also give references.