Hacker News Reader

Comment by l5870uoo9y | Hacker News Reader

RAG doesn’t necessarily give the best results. Essentially it is a technically elegant way to semantic context to the prompt (for many use cases it is over-engineered). I used to offer RAG SQL query generations on SQLAI.ai and while I might introduce it again, for most use cases it was overkill and even made working with the SQL generator unpredictable.

Instead I implemented low tech “RAG” or “data source rules”. It’s a list of general rules you can attach to a particular data source (ie database). Rules are included in the generations and work great. Examples are “Wrap tables and columns in quotes” or “Limit results to 100”. It’s simple and effective - I can execute the generate SQL again my DB for insights.

View on HN

RAG without preprocessing is almost useless, because unless you are careful, RAG gives you all the weaknesses of vector DBs, with all the weaknesses of LLMs!

The easiest example I can come up is imagine you just dump a restaurant menu into a vector DB. The menu is from a hipster restaurant and instead of having "open hours" or "business hours" the menu says "Serving deliciousness between 10:00am and 5:00pm"

Naïve RAG queries are going to fail miserably on that menu. "When is the restaurant open?" "What are the business hours?"

Longer context lengths are actually the solution for this problem, when context is small enough and the potential of ambiguity is high enough, LLMs are the better tool.

Reminds how few years ago Tesla (if i remember - Karpaty) described that in Autopilot they started to extract the 3rd model and use it to explicitly apply some static rules.

What do you mean by "RAG SQL query generations"? Were you searching for example queries similar to the questions the user's asked and injecting those examples into the prompt?

Yes exactly. That way the user could "train" AI on a particular data source.

Semantic search is a powerful tool that can greatly improve the relevance and quality of search results by understanding the intent and contextual meaning of search terms. However, it’s not without its limitations. One of the key challenges with semantic search is the assumption that the answer to a query is semantically similar to the query itself. This is not always the case, and it can lead to less than optimal results in certain situations. https://fsndzomga.medium.com/the-problem-with-semantic-searc...

Don't post AI slop on HN comments please.

this is not AI slop. do you have a proof of what you just wrote ?

Reply on news.ycombinator.com