Show HN: RAGless – similar to RAG, but $0 LLM API costs at runtime | Hacker News Reader

Show HN: RAGless – similar to RAG, but $0 LLM API costs at runtime

github.com

6 pointsemilianoc8 comments

Haven't looked at the code, but it's unclear from your ReadMe how you turn search queries into embeddings for the semantic search itself. In my experience, this is the problem to solve for local semantic search, since your stored embeddings need to be aligned with the queries (the more expressive your stored embeddings, the more expensive each query will be; your question generation pipeline does not address this). Correct me if I'm wrong though.

The $0 runtime API cost is an interesting angle. A lot of AI products I've seen are fine with the initial demo, but the economics get very different once you have real usage. I'd be interested to see how the quality compares with a traditional RAG setup on less predictable queries. The cost reduction is obviously attractive if the quality stays close.

Im not deep into the space but surely zero LLM cost would be default for search???

In traditional RAG (Retrieval-Augmented Generation), you use an LLM at runtime to read those retrieved documents and synthesize a direct, conversational answer for the user. That generative step is what incurs an API cost and adds latency per query.

RAGless gives you that direct Q&A experience (the synthesized answer) but removes the runtime generation by pre-computing all the answers offline.

That sounds ... magic. Will give it a go.

This is interesting, especially if the data doesn't change much.

thanks :)

Hi HN,

I built RAGless to remove the LLM from the critical query path. Traditional RAG is powerful, but introduces non-deterministic behavior (hallucinations) and recurring API costs at runtime.

RAGless shifts the LLM entirely to build/ingestion time. It processes your documents offline to generate a deterministic artifact of Question-Answer pairs, strictly grounded in source quotes.

At runtime, there is no LLM in the loop. It relies solely on semantic search (Q-Q matching) between the user's query and the pre-computed questions.

$0 LLM API costs per query.

Deterministic outputs: it only serves pre-validated answers.

Low latency: zero generation wait time.

The hardest bottleneck in this "compile-time" architecture is validating the generated artifact before deployment. Currently, I use strict JSON schema enforcement and an optional --judge pass. The judge acts purely as a tripwire/veto to discard ungrounded blocks, keeping the ultimate trust on the deterministic layer.

I'd love to hear your thoughts on this pattern and how you tackle artifact validation.

Reply on news.ycombinator.com