Hacker News Reader

Comment by dcrazy | Hacker News Reader

A “hallucination” is an authoritative counterfactual statement returned as a response. Why do you think it is impossible to engineer an LLM (by which I am including tool usage and RAG) that catches and prevents such statements?

View on HN

Because they are word generators without any concept of quality save what is in their weights and they have been trained on the internet, much of which is wrong or inappropriate for any given context. They have also been trained to be people pleasers and do as they are told.

The popular answer is sometimes the wrong answer.

If I cite an incorrect Wikipedia article, I didn’t hallucinate it. The citation points to a real article that happens to be incorrect.

https://en.wikipedia.org/wiki/Entscheidungsproblem

Undecidability is not relevant here beyond e.g. a model incorrectly claiming something is decidable, and potentially even following through. It is no different to the model incorrectly claiming anything else.

These are statistical systems, so guaranteeing any particular high level behavior is not possible because of that. But given that they're working with natural language, that was never going to happen anyways, for the obvious language theoretic reasons.

The more appreciable interpretation of the claim is that they can be nevertheless tuned so that this issue becomes practically resolved. Contending guarantees and theoreticals is simply misplaced, these are not formal symbolic reasoning systems being buggy.

GP was asking for a guarantee against emitting false decidable statements. We agree that this is impossible. You can slap layers upon layers of heuristics on top, yes, but you will not eliminate all hallucinations because Church and Turing proved it fundamentally impossible nearly a century ago.

I might be slow today but I'm pretty sure that the Church-Turing thesis is about undecidable statements, not false decidable statements. If a decidable statement is false... then that's it, it is just false, that's how we can decide so in the first place.

Why do you think that will happen, and why do you think going for slop in the meantime could possibly bring us closer to that?

Nobody said anything about “going for slop.” You can watch today’s models actively trying to check themselves. Just use Google’s AI mode, for example. It’s far from perfect—it doesn’t fact-check every single claim, nor does it correctly understand 100% of the sources it does cite. But I’ve found it pretty useful for research as long as I use my brain and check its citations.

> Nobody said anything about “going for slop.”

I did. I used that phrase.

"not slop cuz pretty useful" = slop argument

Reply on news.ycombinator.com