Hacker News Reader

Comment by simonw | Hacker News Reader

A Large Language Model itself can't perform RAG: a model is a big binary blob of matrices that you run prompts against.

Anything that can do RAG is, by definition, a system that wraps an LLM with additional code that performs the retrieval.

It's the difference between ChatGPT (software that wraps a model and can extra features such as tool usage, Code Interpreter, RAG lookup via Bing etc) and GPT-4 Turbo (a model).

View on HN

Why can’t a model explore its environment (given access) and find and use those tools? Why can’t a model fire up a query (sql, http, or whatever) to do the retrieval if it determines it needs more information?

I think it's just a matter of semantics - "model" usually refers to a neural network or some similarly pure, deterministic computation, or so I thought.

I'm talking about Large Language Models - the architectures behind most of the current generative text AI boom.

In order to use tools they need to be run as part of a system that grants them access to tools, eg via the reAct pattern. https://til.simonwillison.net/llms/python-react-pattern

Are you talking about a simple script which executes model’s instructions? It can literally be as simple as connecting the model to my computer command line. I will tell it what I want done and give it my cc number or website creds.

Obviously I’m talking about next gen models, like gpt5/6.

For that a model needs to recursively parse its output. That is not possible with the current architectures, we need Agent Frameworks for that.

Though the terminology is incorrect (a model by itself cannot 'perform' word other than encode/decode, the llm agents are developed to do what you're stating. But when you think about an llm agent doing all sorts of things like sql etc, it effectively becomes a RAG (which in itself is a process than a technology)

Does the LLM get trained as the database get builded?

As you train a model it should build it's knowledge database at the same time. I guess that's how it work?

When an LLM get trained on next word guessing, can it search the web to improve his results?

Reply on news.ycombinator.com