> they don't have "memory", they have no capacity to learn outside of the training stage
The underlying LLMs don't, but the agent frameworks around them do.
The underlying LLMs don't, but the agent frameworks around them do.
We are perfectly capable of running LLMs in a way that does a backward pass to update some or all of its weights after every user message. But, naively implemented, you only get partial, fragmentary absorption of the info in those messages, it costs three times as much compute, and you lose out on the ability to implement a ton of optimizations that making modern LLM serving economical.
If you want to do it, though, ask your friendly neighborhood robot to get it working with a tiny model (whose full precision weights fit several-times-over on your machine's resources).
Technologically the LLMs we use today don't implement this behavior, but you could take the weights of Sol and add a couple (very large) patches to vllm (or whatever OpenAI has today) and have a version of Sol that does have "memory"
That's been my experience, anyway. Memories from 100 prompts ago tainting what I'm trying to do right now.