Ask HN: Challenges with RAG | Hacker News Reader

Ask HN: Challenges with RAG

1 pointofermend8 comments

RAG seems to be the winning methodology for building Chat-with-your-data applications.

HakcerNews: what do you find to be the most challenging issue when migrating RAG applications from simple prototypes to enterprise scale?

View on HN

Knowing how to do it?

AI / GPTs have splatted so much additional user-space-knowledge requirements its astounding.

What we should be asking is "how are we going to teach the youngsters faster" and that looks like a path diverged between two woods.... The one less followed by.

@ofermend - in the words of Jony Ive, if its so perfectly designed, you wouldnt need a case" (Tell that to IfixIt (YC alum) you Muppet. (Ive is the muppet in this, not @ofermend.)

And then the billions spent on putting more plastic and electronic boron in the waste. Visionary.

You always have to recognize a Visionary who brings amazing products to market without an exit plan for planned obsolescence.

Yes I agree it's complex. So large companies with large tech teams and expertise can handle this and will likely build teams to develop and maintain their own RAG pipelines. I believe simplifying this to non-experts is a necessary next step. I'm curious though about more specific challenges - what is the most difficult part of building RAG applications? Is it educating yourself about the various components (embeddings, vector databases, LLM, prompts), is it scaling, data ingest, security, data privacy? Something else?

So simplicity and ease-of-use. I assume you mean for builders/developers here, right?

llama_index puts out lots of great content on how to do RAG, definitely my recommended starting point

What about data types? Are your RAG pipelines mostly using text data from structured data, from document stores, or unstructured data like PDF files, website content and the like? Or maybe enterprise applications like Salesforce, HR, project management, etc?

even with RAG, it could hallucinate badly

Do you think there is a level of hallucination that may be "acceptable" for an enterprise deployment? if so - what would that be?

We do see that hallucination varies between LLMs https://huggingface.co/spaces/vectara/leaderboard

Reply on news.ycombinator.com