Hacker News Reader

Comment by wrs | Hacker News Reader

Lately I'm trying a variation on this: Have the main agent make a phased implementation plan, then for each phase, have it start an implementation subagent with a focused prompt, then review that agent's work in the main session. The theory being that the main session still contains all the research, but it can review just the diff rather than have the entire implementation session in context as well.

The post doesn't include a review in the cost comparison, but I find that immediately doing a review catches lots of mistakes.

View on HN

Actually it does include an important level of review. They say "init a TODO list with a validation step for each item", and then the last several sections of the execution are within the TODO - where it is validating the code.

To me that was the heart of the article and of what they did - create a self-validating todo list that deterministically forces the agents to stay on task, and then trim out just the right extraneous context (the actual planning) without messing with the read-file context which is where they found subagents burning most of their tokens.

Just a few days ago I was speculating about forking context to implement instead of spawning subagents here on hn. This is similar to that just several more steps more sophisticated. You could even fork the trimmed context here for very large tasks.

It wasn’t clear to me (and still isn’t, after slogging through the inhuman article again) that the verification here is done by a separate agent from the implementation. That’s a second trimming step, but in reverse: at that point you want the new code in the context, but not the implementation reasoning.

No idea, but yes it seems like good practice to trim context there as well. A good candidate for v2 if it's not present.

At the same time, even if that is not done, the todo is both specification and tests. If it passes those when it should not, then either you approved something you should not have, or else there should only be smaller improvements to be made. I like the concept enough and I am not very concerned about this, so I will give it a test on a project I have had on backburner for a bit.

Yes, this is a core part of my workflow; use the smartest model (e.g. Fable) to generate a large hierarchical implementation plan, then clear the context and have a lesser model (e.g. Sonnet) track and execute the plan [0], often in parallel.

It all depends on how much work you're doing; if it's a simple task, there's virtually no need for plans at all; just have the smart agent do it. But if the work is going to take 5, 10 or 100 sessions, there's real value in the "smart agent plans, cheap agent executes" model.

[0]: https://github.com/bensyverson/jobs

This is exactly what the post argues against though, as it leads to higher overall cost.

Yes, and I'm arguing that the post is over-simplifying. As I mention, for small plans, it's probably easier to have the big model just do it. But for large, multi-session plans, it's cheaper to let smaller models execute the bulk of the work.

Perhaps, there's a small change that might work better here - if we use a different agent (by creating custom agents) for plan and implementation, then the switch to the implementation agent wouldn't end up re-reading a lot as the new agent would have fresh empty context. I implemented this[1] from one of the posters on HN and I think it'd work well with this problem too. Would be great if you could point out if my understanding is wrong.

[1]: https://www.stavros.io/posts/how-i-write-software-with-llms/

The issue with this is just how much context it consumes. Generating the plan could easily get me to 30% of a 1M context window—the point at which I normally clear. Most of that context fill is not necessary for execution, and the model degrades as context fills. So starting from 30% (or passing a fresh agent all of that context) is not ideal. But maybe I'm misunderstanding what you're proposing?

Yes, sorry let me explain a bit more. The idea is that we have custom agents each having a model version of their own. So, if we have a planning agent that uses Opus and a worker agent that uses Sonnet, then the planning agent writes the plan and hands it over to the worker agent. This way, the handover here is not going to retain the context.

When the worker agent picks it up, it'd have a new context and only the todo list from the plan, so it shouldn't be reading all the files again.

Ah, got it. Yeah, that's what I do using my tool, Jobs. When the plan is big enough, it's definitely cheaper & more efficient than having the "big" agent do everything.

My main point was not about the implementation stage, but the verification stage. Letting the smarter agent verify the work of the cheap agent finds a lot of little oversights and sloppiness that I used to have to find myself. So I’d be interested in that cost comparison.

I think it would depend on how you measure it. Adding a review step will definitely increase cost, but theoretically that cost is lower than re-doing the subagent's implementation later when you discover it's broken.

I’m using Claude code dynamic workflow like this. I tell Fable to use a workflow. He is the planner, orchestrator and reviewer. Opus agents are implementers. Works unbelievably well.

“He”

I call him "my son"

In many languages that’s the only way to express that since there is no “it”. It doesn’t mean speakers of such languages anthropofy everything and I think that’s the case here.

can you explain this in a bit more detail? super interested

It's literally all there is to it. Write your prompt normally. then, at the last paragraph, you write:

Use a workflow to implement this. You (Fable) are the orchestrator, planner and reviewer. Opus agents are implementers.

That's all there's to it. it will create the dynamic workflow - has a nice interface native to Claude Code - and do all the coordination to deliver what you asked.

I do the same, but delegate to codex instead via the /smux skill. It's expensive, but I get one long-running Claude context window and one frequently compacting codex, and this saves me from needing to re-gather context all the time.

Per the OP's technique, I'll use Fable for intensive planning moments, then switch back to opus when things are going well.

Reply on news.ycombinator.com