Hacker News Reader

Comment by p1esk | Hacker News Reader

It’s strange: most answers here assume the next gen models won’t be able to perform RAG on its own. IMO, it would be wise to assume the opposite - anything humans currently do to make models smarter will be built in.

View on HN

A Large Language Model itself can't perform RAG: a model is a big binary blob of matrices that you run prompts against.

Anything that can do RAG is, by definition, a system that wraps an LLM with additional code that performs the retrieval.

It's the difference between ChatGPT (software that wraps a model and can extra features such as tool usage, Code Interpreter, RAG lookup via Bing etc) and GPT-4 Turbo (a model).

Why can’t a model explore its environment (given access) and find and use those tools? Why can’t a model fire up a query (sql, http, or whatever) to do the retrieval if it determines it needs more information?

I think it's just a matter of semantics - "model" usually refers to a neural network or some similarly pure, deterministic computation, or so I thought.

I'm talking about Large Language Models - the architectures behind most of the current generative text AI boom.

In order to use tools they need to be run as part of a system that grants them access to tools, eg via the reAct pattern. https://til.simonwillison.net/llms/python-react-pattern

Are you talking about a simple script which executes model’s instructions? It can literally be as simple as connecting the model to my computer command line. I will tell it what I want done and give it my cc number or website creds.

Obviously I’m talking about next gen models, like gpt5/6.

For that a model needs to recursively parse its output. That is not possible with the current architectures, we need Agent Frameworks for that.

Though the terminology is incorrect (a model by itself cannot 'perform' word other than encode/decode, the llm agents are developed to do what you're stating. But when you think about an llm agent doing all sorts of things like sql etc, it effectively becomes a RAG (which in itself is a process than a technology)

Does the LLM get trained as the database get builded?

As you train a model it should build it's knowledge database at the same time. I guess that's how it work?

When an LLM get trained on next word guessing, can it search the web to improve his results?

I want an LLM that can perform self-inspection; attribute each response to its sources. And then I would replace RAG with continuous learning.

I mean you can’t build rag into a model, it’s part of a system or application.

You wouldn’t say a steering wheel is built into an engine. People just build cars with engines.

I guess I’m a bit confused - what would be an equivalent to a “steering wheel” in the context of a human brain as an “engine”? Or would this steering wheel be part of the brain? If so, why do we need to separate the two in the context of an LLM?

Reply on news.ycombinator.com