Comment by nzach | Hacker News Reader

Comment by nzach | Hacker News Reader

I'm starting to think that 'memory' may be the wrong analogy for what we want.

I do think that having a set of token that are highly personalized to your project and to way you work is beneficial. I also think that the idea that this set of token will be constructed in the background without any work from the user is really appealing. So it's understandable that the 'memory' analogy became so popular.

But in my experience having a really good AGENTS.md file almost always produce better results than enabling memory.

Maybe we should start to think about how we 'train'/'onboard' agents into our projects, in a similar way that we do for new co-workers. Imagine if we could send the agent to our repo and ask it to learn our patterns and in the end we could quiz the agent to gauge how much it actually understood the project. Once he 'understands' the project we can start to use it to help with development.

In a very small scale (example, individual new features) I will sometimes ask the agent to explain me how things work (even though I already know how it works) so I can 'prime' the agent context with good data before starting any real work. But I'm not sure if this approach could be reliably scaled to work with any repo for any kind of work.

When I was doing a really large refactor across the codebase I told Claude Code to explain how certain things worked currently and how I wanted things to look like after the migration and a plan on how to get there. Then I did several further clean sessions where I always started along the lines: "Using this plan {link} do X."

Works pretty well for "non-permanent" instructions (you don't want to put this info into your committed markdowns).

My main motivation was simply to save tokens, but it actually worked really well and improved speed as well.

we want some kind of fractal knowledge graph. It starts coarse at some zoom level and you can move in and out. You can insert knowledge at any level and it's out/in levels adjust accordingly. The search semantics at every level are the same, but whats in the visible area changes based on what your focus is.

One way I've toyed with a graph outline with it using "whitening" (https://arxiv.org/pdf/2104.01767v3) for embeddings rather than just text, so you add things like file path, nearest title/method/const etc. You have to have dummy text though because it fails with "null"; all embeddings need to carry some kind of text and of the same size.

so when the agent rembembers something, the memory would be an embedding that includes where they found the file, what method or const or whatever they're in, what the task they're working on is, etc. That all becomes a single embedding. You could imagine a metadata tag that also describes the tools they're using etc.

Map out a complete space of tags for whitening an embedding and there's surely a proper mix. Then when you're searching for things in the embedding, you also store some of the other metadata as plane strings & edges, which gets you some useful granularity.

It's really about storing institutional context and on-task learnings. The AGENTS.md can do the same thing as memory, but if you have a good memory system, in theory you never need to do any ongoing maintenance of AGENTS.md and the system will do it automatically. And because it's automatic and has a whole harness system built around maintaining the memories, you can engineer it to have not just more convenience of maintenance but also better correctness / recall quality.

Memory systems can also let AI load memories on demand so it's actually analogous to maintaining AGENTS.md plus twenty different "read this if you need to do X" markdown files.

I think the name "memory" makes sense given that the AI system is one singular system with central context (rather than a software org of multiple distinct humans with distinct memories), so the equivalent of institutional knowledge in the software org really is just akin to memory for the AI system.

Reply on news.ycombinator.com