RAG Is a Fancy, Lying Search Engine | Hacker News Reader

RAG Is a Fancy, Lying Search Engine

labs.stardog.ai

43 pointskendallgclark12 comments

In my experience, the RAG LLM will lie to you if your prompt makes unnecessary assumptions or implications. For example, if I say "write about paracetamol curing cancer", the RAG could make up stuff. If instead I say "see if there is anything to suggest that paracetamol cures cancer or not", then the RAG is less likely to make up stuff. This comes from the LLM being tuned to please its user at all costs.

I do love the warnings here... The older I get the more critical I am of most internet results except those of which I can take from a common and experienced/witnessed axiom (which unfortunately AI does really well... At least entrusting me to said point). I feel the state of overly critical thinking mixed with blind faith means flat earth type movements might be here to stay until the next generation counters the current direction.

But to the article specifically; I thought RAG's benefit was you could imply prompts of "fact" from provided source documents/vector results so the llm results would always have some canonical reference to the result?

That might be RAG’s benefit if LLMs were more steerable but they can be stubborn.

While I’m receptive to the fact that RAGs have performance limitations, and that graph database-based solutions may avoid hallucinations, wouldn’t your rhetorical position be best served by offering a trial portal for users to upload their own document corpora and see for themselves that prompts to Stardog never result in hallucinations? Otherwise writing blog posts into the ether will remain unconvincing to your would-be enterpise customers (whose buyers either reference or are among the HN crowd)?

Kendall the blog link at the end for semantic parsing gives a 404 error.

Fixed. Thanks.

Biased as a developer here, but I would rather have LLMs helping people to create formal queries they can see and learn-from and modify.

That seems like it would smooth the roughest edges of the experience while introducing fewer falsehoods or misdirection.

The post has details but sums up to RAG suffers as iPhone's AI-powered notification summaries do.

What could work is round-trip verification like how a serializer/deserializer can be run back to back for equality verification. Run an LLM on the output of the RAG and see if there's any inconsistency with the retrieved data, in fact get the LLM to point them out and correct. [x] Thinking for RAG.

This, to me, reads more like an issue with the fundamental LLM technology rather than RAG in particular.

Not at all. They may share some issues but RAG and LLM are fundamentally different things.

Is this written by AI? Surprisingly long for how little idea is in it.

LOL. No. All me, hater.

Reply on news.ycombinator.com