Build a Basic AI Agent from Scratch: Long Task Planning

Maybe I spent 2-4 hours reviewing it, checking things with colleagues etc.

Then I press "go" and maybe an hour later I have a tested system ready for manual review.

It's plans are at least as good as any I've seen. Their weakness is if there are unstated assumptions I have about how things need to be done, so most of my time is now getting those assumptions stated properly and then reviewing.

Why wouldn't I use this? It's the best tool I've used in my 30 years of professional programming.

You're wrong about what i mean with delegating architecture to coding agent.

I'll let the coding agent take the first shot at it, already having in my mind a decent idea about how i'd do that. Worst case its wrong i need to correct it, more than half the time it comes up with the same sort of design, sometimes it comes up with a better alternative.

Additionally, the same pattern of: "sometimes wrong, mostly good, sometimes better" also plays out wrt naming things. I thought i was decent at naming things, but an the LLM is literally build on turning 'concepts' in a vector space into words.

And in a very real way the names its choosing will 'compress' the ideas so that the next time an LLM reads it is more likely to understand.

For this to work though you need your complete system accessible and well structured.

You say "I always knew how the system worked and could extend it as needed". If an AI can't learn how your system works then that's a problem with the system setup, not the AI. An AI can find its way in the linux kernel or chromium source code just fine.

If you're in a role where you only spend time planning / architecture, then i assume things are pretty gnarly to begin with. The thing i can only guess at - and which is on a spectrum - how much of our role exists to support the weight of accidental vs essential complexity.

i.e. can the engineers not do the planning because: they're not that good, or its very broad things that need to expertly interplay with each other, or because the org has a mountain of buried bodies.

In my experience some of the more fanatic AI people are blind to the mountain of buried bodies covering a lot of essential complexity, but others can be blind to how well AI works when you can just shoot of a prompt to unbury a body and actually reduce the debt.

But in one sentence:

> Why can't you do the planning ?

This way lets me do more planning - planning is basically all i do now.

Even in that situation, I think I would still only feel comfortable approaching the task as I would do it without AI, and using the AI to accelerate the parts that would be time-consuming. E.g. finding where/how feature X is implemented, how it would affect the overall system if I were to change it this way, etc.

…but it’s than just that (in most cases; I am just lazy sometimes); but fundamentally there’s a limit to how much complexity people can comprehend.

We are good at working at high level abstractions, modules with clear apis that can be sprung to together into some kind of feature.

You don’t need to look inside the black box of the module if you trust the implementer; Ive never opened up the internals of a calendar be like “how does this work?”. I just don’t care. It’s a calendar. I use the api.

I think most people are using these tools in this way; very few people are having an agent write a plan, then a sub agent review it, no human in the loop. Those are for prototypes and are yolo cowboys using open claw and playing with the phones instead of working; we have a few at work, but their PRs are regularly rejected as slop.

…but, realistically; many people aren’t software architects. They may not even know coding patterns, forget architecture patterns.

Having an agent spit out generic software architecture is probably better than what they were producing before.

Writing a module / feature using generic architecture and planning is probably better than random code spaghetti right?

It’s easy to lament the loss of craft here, but at the end of the day, the models today do an ok job of this. The models of tomorrow will probably be better at it than many people.

Architecture is easy composed to actually implementing things. You just wave your hands from your ivory tower and say “more event sourcing”.

The strengths are that the design forces Chain of Thought as a memory buffer and the TODO list in an FSM style. I think those are fine. The recovery strategy is also pretty good.

However, the problem is that the business logic does not run as Python code but lives inside the prompt. And it does not support parallel execution. But as a single run script, it is helpful enough for understanding the concept.

Of course, if I were to do the code properly, I would use a separate storage instead of in memory, and more carefully verify tool constraints and the actual scope limitations of the tools. But still, I think this is helpful enough.

I don't think the content is low quality, though.

The original post is also available at the poster’s own blog [1], so the question is a very valid one. Clearly, “posting articles for free” is a hurdle already cleared by the author.

[1] https://www.ruxu.dev/articles/ai/build-an-ai-agent-planning/