Hacker News Reader

Comment by vb-8448 | Hacker News Reader

> Global memory

I don't get why we need global memory for code? Aren't code comments (even if invented for humans) the ideal place where to put "memories"?

View on HN

LLMs also do not have anything like memory, by design. Everything bolted on that smells like memory or is being called memory is a crutch, at best. Not being pedantic, I just think this aspect is lost on a lot of people.

Also, when we humans are tricked into perceiving a remote mind, we automatically assume it has a memory like us, or at least like large animals.

In other words, there's a story-document between a SherlockHolmesBot and UserPlaceholder that's growing like a crystal formation. Some software sees "Sherlock Homes Says" and then "performs" the dialogue at us, and we then assume Sherlock Holmes exists with a mind and memory, rather than being a facet of a text-story.

Exactly, this is why we see such minimalistic memory systems being used. Information retrieval is bad, but improving information availability by pre-computing information retrieval is not the obvious fix people think it is.

Sometimes, but I think there are a few problems with this:

1. Some context doesn't have an obvious place to write it in the code. If you're explaining why a tricky function is implemented a certain way, you can leave the explanation above the function or within the function. If you're explaining something more general, there might not be a natural place to put it.

2. Relatedly, some context doesn't have an obvious location to read in the code. E.g. you can comment on a schema that a table is intended to be append-only, but an agent could easily miss this comment if it doesn't gather context all the way down to the raw schema. Any research process has unknown unknowns. There may not be a canonical place to look.

3. Adding comments for every single human intent might be a bit noisy. E.g., if a code reviewer flags something that I know isn't a bug, I want the system to learn from this, and I want that knowledge to take effect outside of just code review, but I might not want every single code review thread to yield a codebase comment.

1: There are corner cases, no doubt, but the main source of information should remain near the code.

2 & 3: How to be sure that the agent will pick the "append only hint" from the memory? Especially when it starts growing and cannot be part of the context?

I think the bigger need is for outside concerns, like: what is the infrastructure like? How much volume does this feature handle? How should changes to live code be rolled out to prevent running processes from failing?

In addition to all the product details that aren't in the codebase or docs, like "keep this logical path because it is used by our one big client."

Mostly what I think is needed is a richer worldview available to the LLM so it understands not just the code but can understand the product and its real world usage and constraints.

But how you are going to link the "keep this logical path ..." with the code? You cannot feed all the memory to the agent because of the context rot, and you are not sure the agent will always pick it when needed.

Reply on news.ycombinator.com