Hacker News Reader

Comment by jonahx | Hacker News Reader

Is the architecture they use in this diagram currently the best way to train LLMs in general on custom data sets?

https://raw.githubusercontent.com/vanna-ai/vanna/main/img/va...

That is, store your trained custom data in vector db and then use RAG to retrieve relevant content and inject that into the prompt of the LLM the user is querying with?

As opposed to fine tuning or other methods?

View on HN

All the podcasts I've been listening to recommend RAG over fine-tuning. My intuition is that having the relevant knowledge in the context rather than the weights brings it closer to the outputs, thereby making it much more likely to provide accurate information and avoid hallucinations/confabulations.

Do you have any podcasts you would reccomend with this type of content?

I listen to Data Skeptic and Practical AI. It's interesting content, and I find Practical AI has guests who are particularly relevant to what I want to learn, but you should keep in mind that the guests are usually trying to promote their own products, so everything they say should be taken with a grain of salt

> All the podcasts I've been listening to recommend RAG over fine-tuning

I'm always suspicious that is just because RAG is so much more accessible (both compute wise and in terms of expertise required). There's far more profit in selling something accessible to the masses to a lot of people than something only a niche group of users can do.

I think most people who do actual fine tuning would still probably then use RAG afterwards ...

One added benefit of RAG is that it's more "pluggable." It's a lot easier to plug into newer LLMs that come out. If and when GPT-5 comes out, it'll be a one character change in your code to start using it and still maintain the same reference corpus.

We can get a lot done with vector db + RAG before having to finetune or custom models. There are a lot of techniques to improve RAG performance. Captured a few of them a while back at https://llmstack.ai/blog/retrieval-augmented-generation.

Yes from what I gather. And just to emphasize there's no LLM (re)training involved at all.

Reply on news.ycombinator.com