Hacker News Reader

Comment by KeplerBoy | Hacker News Reader

How else would it work? Less technical people don't even watch their context usage.

In the olden times, aka like two years ago, AI chats would just stop working or just start slicing off the oldest parts of the context to fit the model's window.

That said, compaction feels like an idea that should work reasonably well, but across all of the major providers and agent tools I've used has never actually produced compelling results, to where if I see I'm getting close to the token limit I prefer to start putting a bow on the project and readying it for a fresh start. Even when I provide a detailed compaction prompt it usually focuses on the wrong stuff.

Do you have experience with OAI's, it's been known to be to be good for a while now, going off public consensus and my experience.

Yes, and I think it has improved some, but just this week 6 Astra lost the most key details of a project across a compaction and got confused about what we were actually trying to do. I would have preferred to stop at 85%, interactively develop a next-steps prompt and continue from there when ready, rather than seeing it compact and become 5x dumber from one turn to the next.

I have a "wrap up the session" skill that I use when the session gets >50% of its token use. It commits everything, updates documentation, writes a handoff doc, makes sure the todo.md is up to date, etc.

Still works better than compaction.

I do something similar - as I approach the context limits I have a pre-compact flush skill that extracts anything useful from the context, updates the MEMORY.md and my Obsidian vaults (set up as a poor man's graph DB) and so forth. Once everything's been stored I run /compact to keep the general session flow intact. Recently I added a small embedder/vector search setup to the same skill, which seems promising so far.

On another environment I've been doing something roughly similar, but have integrated Hindsight as a kind of all-in-one of the above and am still trying to suss out the best compaction strategy.

This is what I've been working toward as well. It's interesting how having the agent do its own reasoning about what it thinks is the most relevant knowledge to carry forward into the next pieces of work is vastly more effective (and even fast sometimes) than whatever the mystery-meat "compaction" process is.

Mentioned by the author in a recent HN thread, I'm also experimenting with automating this through a tiny issue tracker called epiq [1] that basically lets the agent sessions themselves file tickets with the follow-on tasks and relevant handoff right in them, and then a dispatcher automatically launches those tickets into new agent sessions.

[1]: https://ljtn.github.io/epiq/

The idea to kill "mystery-meat compaction" and use an external handoff primitive is brilliant, but doesn't letting the agent author its own handoff tickets re-introduces the same failure mode?

You would think so, right? But as with others in the thread, I'd found directing the agent itself to prepare the handoff does deliver much better continuity.

I assume Anthropic & friends have noticed this as well and will change how they handle long running sessions, so the gap will likely close over time, but this is definitely where things stand today.

i have active disagreements with teammates on the value of compaction/months-long sessions.

Same teamates also post 'Sol deleted my git repo!' or 'Sorry, ignore those 300 PR comments i was just looking!' ~once a month.

No compelling results because summarization is really hard.

Reply on news.ycombinator.com