LLMs: RAG vs. Fine-Tuning | Hacker News Reader

LLMs: RAG vs. Fine-Tuning

winder.ai

36 pointschhum5 comments

Glad the author ended with they can be better together. Generally a good piece worth reading.

I'd add that Agents are becoming another piller. They can take actions both inside and outside the system. For outside, you can think of a code assistant using an interpreter, or flights querying a database. For inside, most systems are becoming more sophisticated. You might need to decide what data set to RAG with or ask if the system should try again or refine. There's also the Mixture of Experts (ensembles generally), CoT, ToT, and more to come.

The symphony is getting really complex! Truly an exciting time to be working in this field.

That's correct. I would even add that this will give rise to the need for a standardized Agentic focused interface (called it "Agent Programming Interface" for confusions sake) that will be massively necessary sooner than later.

I think this is a great primer on how to make LLMs enterprise/domain specific, but is still couching this technology's main use-case in the context of "question-answer" machine toward the service of a user.

While this may be the (brilliant) hook that enabled the massive conceptual adoption of the tech it is not going to be the value proposition of it. In the next 6 months we're going to see the deployment of Agency-based solutions completely powered by LLM[1][2].

The real value LLM provide will be that they replace the "humans-as-integrations" and RPA automations.

1. https://github.com/microsoft/UFO 2. https://github.com/microsoft/TaskWeaver

Perhaps they will freeze their LLM api, but these systems appear.incredibly fragile and rely on dumb parsing of LLM output'

They'll be a toy until a purpose built and reliable LLM is available and its primarily local'

I installed danswer and connected it to our internal systems to see how well it would work. And it was pretty bad.

And I found that a lot of rag is applying AI lipstick on a rudimentary keyword search pig. No “understanding” of raw data.

I couldn’t ask it things like “is our SSH key policy in line with the latest NIST best practices” if the word SSH wasn’t in our key management policy, since it would just say I can’t find anything.

It seems to me that the true power will only come with extremely large context windows because then we will actually apply “understanding” to our raw data.

Reply on news.ycombinator.com