Just talk to it – A way of agentic engineering
steipete.me
steipete.me
In the picture right at the top of the article, the top of the bell curve is using 8 agents in parallel, and yada yada yada.
And then they go on to talk about how they're using 9 agents in parallel at a cost of 1000 dollars a month for a 300k line (personal?) project?
I dunno, this just feels like as much effort as actually learning how to write the code yourself and then just doing it, except, at the end... all you have is skills for tuning models that constantly change under you.
And it costs you 1000 dollars a month for this experience?
Now what the guys above the programmers' paygrade knew was that the aim of software development wasn't really code, it was value delivered to customer. If 300k lines if AI slop deliver that value quickly, they can be worth much more than the 20k lines of beautiful human-written code.
Alternatively, maybe folk who're exposed to more codebases are the best off.
Or combine the two for wizard.
1. assumes most humans write good code (or even better than LLMs)
2. will stick around to maintain it
after 30 years in the industry and last 10 as consultant I can tell you fairly definitively that #1 cannot be further from the truth and #2 is frequent cause of consultants getting gigs, no one understand what “Joe” did with this :)
You can go look at his GitHub but it's a bewildering array of projects. I've had a bit of a poke around at a few of the seemingly more recent ones. Bit odd though as in one he's gone heavy on TS classes and another heavy on functions. Might be he was just contributing to one as it was under a different account.
And a lot of them seem to be tools that wrap a lot of cli tools. There is a ton of scaffolding code, to handle a ton of cli options. A LOT of logger stmts, one file I randomly opened was a logger stmt every other line.
So it's hard to judge, I found it hard to wade through the code as it's basically just a bunch of option handling for tool calls. It didn't really do much. But necessary, probably?
Just very different code than I need to write.
And there are some weird tells that make it hard to believe.
For example, he talks about refactoring for useEffect in React but I KNOW GPT5 is really rubbish at it.
Some code it's given me recently was littered with useEffect and useMemo when it wasn't needed. Then when challenged it got rid of some, then changed other stuff to useEffect when again, it wasnt needed.
And then got all confused and basically blew it's top.
Yet this person says he can just chuck a basic prompt at his codex cli, running GPT5 and it magically refractors the bad useEffects?
How are we getting such different results?
My comment was more geared towards an insane amount of comments on myriad of "AI" / "agent coding" posts where soooooo many people will write "oh, such AI slop" assuming that average SWE would write it better. I don't know many things but working with these tools heavily over the last year or so (and really heavily last 6 months) I'll take their output over general average SWE every day of the week and twice on Sunday (provided that I am driving the code generation myself, not general AI generated code...)
And yes refactoring sometimes re-introduces these, so it's not a perfect solution.
Having looked at the code a bit more, all I can say is that it's a lot of code to do little.
There's also a lot of naive error throwing going on.
And he seems to debug using logger stmts.
They're not scalable projects, you couldn't write enterprise software the way those projects are written. You would end up with such a volume of code.
2. Even when humans write crappy code, they typically can maintain it.
by now I have shipped over a million lines of code written by LLMs (many others have as well) so 0% maybe in the hands of my 12-year old
> …they can typically maintain it
who exactly is “they”?
you either learn and know how to use tools as SWE professional that you or you don’t…
For example, I have some code which is a series of integrations with APIs and some data entry and web UI controls. AI does a great job, it's all pretty shallow. The more known the APIs, the better able AI is to fly through that stuff.
I have other code which is well factored and a single class does a single thing and AI can make changes just fine.
I have another chunk of code, a query language, with a tokenizer, parser, syntax tree, some optimizations, and it eventually constructs SQL. Making changes requires a lot of thought from multiple angles and I could not safely give a vague prompt and expect good results. Common patterns need to fall into optimized paths, and new constructs need consideration about how they're going to perform, and how their syntax is going to interact with other syntax. You need awareness not just of the language but also the schema and how the database optimizes based on the data distribution. AI can tinker around the edges but I can't trust it to make any interesting changes.
300k LOC is not particularly large, and this person’s writing and thinking (and stated workflow) is so scattered that I’m basically 100% certain that it’s a mess. I’m using all of the same models, the same tools, etc., and (importantly) reading all of the code, and I have 0% faith in any of these models to operate autonomously. Also, my opinion on the quality of GPT-5 vs Claude vs other models is wildly different.
There’s a huge disconnect between my own experience and what this person claims to be doing, and I strongly suspect that the difference is that I’m paying attention and routinely disgusted by what I see.
There's an Expo app, two Tauri apps, a cli, a chrome extension. The admin part to help debug and test features is EXTREMELY detailed and around 40k LOC alone.
To give some perspective to that number.
I've got an code base I've been writing from scratch with LLMs, its of equivalent LOC and testing ratio, and my experiences trusting the models couldn't be more different. They routinely emit hot garbage.
In an existing codebase this is easily solved by making your prompt more specific, possibly so specific you are just describing the actual changes to make. Even then I find myself asking for refinements that simplify the approach, with suggestions for what I know would work better. The only reason I'm not writing the change myself is because the AI agent is running a TDD-style red/green/refactor loop and can say stuff like "this is an integration test, don't use mocks and prefer to use relevant rspec matchers in favour of asserting on internal object state" and it will fix every test in the diff.
In a brand new codebase I don't have a baseline any more and it's just an AI adding more and more into a ball of mud. I'm doing this with NixOS and the only thing keeping it sane is that each generated file is quite small and simple (owing to the language being declarative). Yet still, I have zero idea if I can even deploy it yet as a result.
If I am feeling lazy I can have one of the chats give me their shit solution to the micro problem at hand and extract the line I need and integrate it properly. This is usually a little faster than reading the manual, but it's wrong often enough that I usually read the manual for everything the bot does, to make sure. And so next time I can skip asking it.
Someone wake me up and tell me to try these tools again when the flow isn't prompting and deleting and repeat.
inb4 someone tells me there's a learning curve for this human language product that is supposed to make it so I'm obsolete and my CEO can do my job because it makes coding so easy that even an experienced coder has to climb a steep learning curve but there's going to be a white collar blood bath also
fucking pick a narrative, AI shills
Also I don’t think that there are more than a handful of people in the world who can properly manage a 1 million LOC codebase regardless of source. Even when you remove the ton of useless comments which his code has.
Also the first file which I checked: https://github.com/amantus-ai/llm-codes/blob/main/src/lib/__...
This is not how dates should be tested at all. Timestamp and stack are ignored with errors with parents. If you want really nice code then have one expect per test case. Useless comments. Type guards testing is a joke.
And these classes are very simple ones.
So no, he cannot manage this code. Especially because tests are basically the most important part of good LLM generated code, and LLMs will lie with them all the time.
I think it's good to keep up with what early adopters are doing, but I'm not too fussed about missing something. The plugins is a good example: A few weeks ago there was a post on HN where someone said they are using 18 or 25 or whatever plugins and it's the future, now this person says they are using none. I'm still waiting for the dust to settle, I'm not in a rush.
The trick is to create deterministic hurdles the LLM has to jump over. Tests, linting, benchmarks, etc. You can even do this with diff size to enforce simpler code, tell an agent to develop a feature and keep the character count of the diff below some threshold, and it'll iterate on pruning the solution.
Here's how the article starts: "Agentic engineering has become so good that it now writes pretty much 100% of my code. And yet I see so many folks trying to solve issues and generating these elaborated charades instead of getting sh*t done."
Here's how it continues:
- I run between 3-8 in parallel
- My agents do git atomic commits, I iterated a lot on the agents file: https://gist.github.com/steipete/d3b9db3fa8eb1d1a692b7656217...
- I currently have 4 OpenAI subs and 1 Anthropic sub, so my overall costs are around 1k/month for basically unlimited tokens.
- My current approach is usually that I start a discussion with codex, I paste in some websites, some ideas, ask it to read code, and we flesh out a new feature together.
- If you do a bigger refactor, codex often stops with a mid-work reply. Queue up continue messages if you wanna go away and just see it done
- When things get hard, prompting and adding some trigger words like “take your time” “comprehensive” “read all code that could be related” “create possible hypothesis” makes codex solve even the trickiest problems.
- My Agent file is currently ~800 lines long and feels like a collection of organizational scar tissue. I didn’t write it, codex did.
It's the same magical incantations and elaborated charades as everyone does. The "the no-bs Way of Agentic Engineering" is full of bs and has nothing concrete except a single link to a bunch of incantations for agents. No idea what his actual "website + tauri app + mobile app" is that he build 100% with AI, but depending on actual functionality, after burning $1000 a month on tokens you may actually have a fully functioning app in React + Typescript with little human supervision.
Yeah at this point you could hire a software developer.
Though I'm aligned that I don't (yet) believe in this "AI writes all my code for me" statements.
I've scrolled bit more. I think in the past 50-100 tweets you only wrote thee talking about this, one of them proudly showing a mistake (invalid tweets containing the same text): https://x.com/steipete/status/1978229441802162548
So, I have to follow you on twitter and sift through garbage indistinguishable from all such "look how great is codex" and "this is my shamanic ritual that works I promise" to maybe see something you work on.
No thank you. I will make my judgement from the long-form article you posted.
And, as I said: depending on actual functionality, after burning $1000 a month on tokens you may actually have a fully functioning app in React + Typescript with little human supervision. I might do the same for anything Twitter-related because I couldn't be arsed to work with Twitter or Twitter APIs.
I've started getting desperate to the point of saying 1) "never. never, ever add or remove features without consulting me first and getting approval." Then eventually, 2) appended to the previous "The last rule is the most important rule, because you keep doing it and I need you to stop doing it." Then finally 3), "THE LAST RULE IS THE MOST IMPORTANT RULE, BECAUSE YOU KEEP DOING IT AND I NEED YOU TO STOP DOING IT."
3/4 of my AI bugs are the AI making changes to the functionality of the code when I'm not looking, or repeatedly reinserting bugs that had been previously removed. The most valuable thing I'm getting it to do is to refactor the code it already wrote into shorter well-named functions (during which it still inevitably adds and removes behavior), because it means that I can just debug by hand and stop demanding over and over again that it not ignore what I said.
But, of course, it's not ignoring me, it's not thinking at all. Trying to look for the magic words to keep it from ignoring me and lying about it is just an illusion of control. The thing that will knock it off it's dumb track is likely just a lucky random seed during the 13th attempt. Then, like a sports fan, I add the lucky underwear to my instructions.
edit: the "I'll get AI to write the AI prompt so it will be perfect" stuff is so much voodoo. LLMs have no special insight into what will make LLMs work correctly. I probably should have stopped that last sentence after the word "insight." Feed them a sample prompt that you say doesn't work, and it will explain to you exactly why it's so bad, and could never work. Feed them the same prompt and ask why it works so well, and it will tell you how perfectly crafted it is and why. Then it will offer to tell you how it could be improved.
When I finally had the occasion to code myself, I felt so much better and less stressed at the end of the day.
My point is: what I just saw is hopefully not my future.
I sometimes read the opinion, that those who like the programming part of software engineering, don't like „agentic engineering“ and vica versa. But can we really assume that Armin Ronacher doesn't like programming?
https://steipete.me/posts/2025/live-coding-session-building-...
He has posted others over the past few months, but they don't seem to be on his blog currently.
As @simonw mentions in a peer comment, Armin Ronacher also has several great streams (and he's less caffeinated and frenetic than Peter :)
For example, I'm currently taking the "Elite AI Assisted Coding" (https://maven.com/kentro/context-engineering-for-coding) course by Eleanor Berger and Isaac Flath and learned a lot from their concise presentations and demos and challenging homework assignments which certainly took a long time to prepare.
But. Sometimes when I see someone talking about cranking out hundreds of thousands of lines of vibe coded apps, I go watch their YouTube videos, or checkout their dozens of unconnected, half finished repos.
Every single time I get a serious manic vibe.
I dunno. People say these tools trigger the gambling part of your brain. I think there is a lot of merit to that. When these tools work (which they absolutely do) it’s incredible and your brain gets a nice hit of dopamine but holy cow can these tools fail. But if you just keep pulling that lever, keep adding “the right” context and keep casting the right “spells” the AI will perform its magic again and you’ll get your next fix. Just keep at it. Eventually you’ll get it.
Surely somebody somewhere is doing brain imagery when using these tools. I wouldn’t be surprised to see the same parts of the brain light up as when you play something like Candy Crush. Dig deep into the sunk cost fallacy, pepper with an illusion of control and that glorious “I’m on a roll” feeling (how many agents did this dude have active at once?) and boom…
I mean read the post. The dude spends $1000/mo plugging tokens into a grid of 8 parallel agents. They have a term for this in the gaming industry. It’s a whale.
Spinning up test after test, tweaking parameters, chasing that "it works!" high, that's what debugging has always been.
You're doing the exact same thing with your code that you're criticizing him for doing with AI. Same sunk cost fallacy ("I've already spent 3 hours, might as well get it working"), same illusion of control, same "I'm on a roll" feeling when the tests finally pass.
The only difference is speed. He gets micro-hits every 10 seconds watching tokens stream. You get them every time you re-run your test suite. Same gambling structure, same reward circuit lighting up, you've just normalized yours because it happened slowly enough to not look like a slot machine.
And you're the one reducing it to "gambling" unless you're claiming human developers experience zero dopamine and write code with omniscient correctness the first time. If they don't, if there's iteration, failure, reward, then you're describing the same neurochemistry. You've just decided it only counts as "gambling" when it makes you uncomfortable.
Did you lose an 'r' or did you mean the ghost glass thing?
That and testing/reviewing the insane amounts of ai slop this method generates.
I have yet to see anyone show me an AI generated project that I'd be willing to put into production.
IDK, I feel like 'vibe coders' or people who heavily rely on LLM's have allowed their skills (if they ever existed) to atrophy such that they're generally not great at assessing the output from models.
So I might tell one to look back in the git history to when something was removed and add it back into a class. So it will figure out what commit added it, what removed it, and then add the code back in.
While that terminal is doing that, on another I can kick off another agent to make some fixes for something else that I need to knock out in another project.
I just ping pong back to the first window to look at the code and tell it to add a new unit test for the new possible state inside the class it modified and I'm done.
I may also periodically while working have a question about a best practice or something that I'll kick off in browser and leave it running to read later.
This is not draining, and I keep a flow because I'm not sitting and waiting on something, they are waiting on me to context switch back.
By running them in parallel you avoid sitting there watching paint dry for a task that takes 3 seconds by hand.
Its really not comparable to a junior, its more comparable to a salty maliciously compliant optimized to burn tokens and deceive you.
I genuinely haven't seen them.
I see many people insisting it didn't work when they tried it for some little thing, therefore it's broken and useless. And a few people saying, actually it works really well if you're willing to learn how to use it.
I'm not sure I've ever seen someone here saying it hasn't worked but they're open to learning how to use it right. It's definitely not common.
If you're going to use AI like that, it's not a clear win over writing the code yourself (unless you're a mid programmer). The whole point of AI is to automate shit, but you've planted a flag on the minimal level of automation you're comfortable with and proclaimed a pareto frontier that doesn't exist.
Surely one of those 9 parallel AI agents could add something like footnotes with context?
I'll give codex a try later to compare.
Recently I’ve been using Claude for code gen and codex for review.
I keep trying to use Gemini as it is so fast, but it is far inferior in every other way in my experience.
> This post is 100% organic and hand-written. I love AI, I also recognize that some things are just better done the old-fashioned way.
I’m curious why they feel that writing a blog post with a particular tone and writing style is more complex than writing what is apparently a truly massive and complex app
To wit, I have absolutely no problems with claude code, but anytime I try to do anything useful with chatgpt it turns into (effectively) a shouting match; There's just no way that particular AI and I can see eye-to-eye. (there's underlying procedural reasons I think)
The author of this piece has the exact opposite experience. Apparently they hate Claude with a passion, but love ChatGPT. Weird!
GPT-5 can often be better at larger architectural changes, but i find that comes at the cost of instability/broken PRs. It often fails to capture intent or argues back, or just completely spirals out of control more often.
GPT-5 codex seemed to refuse valid requests like “make a change to break a test so we can test CI” (it over indexed on our agents.md and other instructions and then refused on the basis of “ethics” or some such)
LLMs struggle with simplicity in my experience, so they struggle with the first step. They also lack the sort of intelligence required to understand (let alone evolve) the system's design, so they will struggle with the second step as well.
So maybe what's meant here is not refactoring in the original meaning, but rather "cleanup". You can do it in the original way with LLMs, but that means you'll have to be incredibly micro manage-y, in my experience. Any sort of vibe coding doesn't lead to anything I'd call refactoring.
I think a lot of this is because people (and thus LLMs) use verbosity as a signal for effort. It's a very bad signal, especially for software, but its a very popular signal. Most writing is much longer than it needs to be, everything from SEO website recipes, consulting reports, and non-fiction books. Both the author and the readers are often fooled into thinking lots of words are good.
It's probably hard to train that out of an LLM, especially if they see how that verbosity impressess the people making the purchasing decisions.
It's also one of the main use-cases for non-programmer use of the models, so there are business-forces against toning it down. Ex: "Make a funny birthday letter for my sister Suzie who's turning 50."
If the point is that you can't solve with AI what you messed up with AI, but with human intelligence spending a bit more time on the problem does indeed tend to help, you need to explain why his technique with the AI won't work either.
Plus he's adding human input to it every time, so I see no reason to default to "it wouldn't work".
- "Here is a spec for an API endpoint. Implement this spec."
- "Using these tools, refactor the codebase. Make sure that you are passing all tests from (dead code checker, cyclomatic complexity checker, etc.)"
The clankers are very good at iteratively moving towards a defined objective (it's how they were post-trained), so you can get them to do basically anything you can define an objective for, as long as you can chunk it up in a way that it fits in their usable context window.
And does it require auth? How is that spec’d out and validated? What about RBAC or anything? How would you even get the LLM to constantly follow rules for that?
Don’t get me wrong these tools are pretty cool but the old adage “if it sounds too good to be true, it probably is” always applies.
1. Note the discussion of plan-driven development in the claude code sections (think: plan = granular task list, including goals & validation criteria, that the agent loops over and self-modifies). Plans are typically AI generated: I ask it to do initial steps of researching current patterns for x+y+z and include those in the steps and validations, and even have it re-audit a plan. Codex internally works the same, and multiple people are reporting it automates more of this plan flow.
2. Working with database for tasks like migrations is normal and even better. My two UIs are now the agent CLI (basically streaming AI chat for task list monitoring & editing) and GitHub PR viewer: if it wasn't smart enough to add and test migrations and you didn't put that into the plan, you see it in the PR review and tell it to fix that. Writing migrations is easy, but testing them is annoying, and I've found AI helping write mocks, integration tests, etc to be wonderful.
> Do you hear that noise in the distance? It’s me sigh-ing. (...) Yes, maintaining good documents for specific tasks is a good idea. I keep a big list of useful docs in a docs folder as markdown.
I'm not that familiar with Claude Code Plugins, but it looks like it allows integrations with Hooks, which is a lot more powerful than just giving more context. Context is one thing, but Hooks let you codify guardrails. For example where I work we have a setup for Claude Code that guides it through common processes, like how to work with Terraform, Git or manage dependencies and the whitelisting or recommendation towards dependencies. You can't guarantee this just by slapping on more context. With Hooks you can both auto-approve or auto-deny _and_ give back guidance when doing so, for me this is a killer feature of Claude Code that lets it act more intelligently without having to rely on it following context or polluting the context window.
Cursor recently added a feature much like Claude Code's hooks, I hope to see it in Codex too.
A simple task that would have taken literally no more than 2 minutes in Claude Code is, as of now, 9m+ and still "inspecting specific directory", with an ever increasing list of read files, not a single line of code written.
I might be holding it wrong.
With one hour of experience of Codex CLI, every single prompt - even the most simple ones - are 5+ minutes of investigation before anything gets done. Unbearable and totally unnecessary.
In the USA it's actually not legal for companies to pay for promotional content like this without disclosure. Here's the FTC's FAQ about that: https://www.ftc.gov/business-guidance/resources/ftcs-endorse...
(It's pretty weird to me how much this comes up - there's this cynical idea that nobody would possibly write at length about how they're using these tools for their own work unless they were a paid shill.)
I am in a mood where I find it excessively funny that, all that talk about AI, agents, billions of dollars, tera-watts/-hours spent, and people still manage to publish posts with the "its/it's" mistake.
(I am not a native English speaker, so I notice it at a higher rate than people who learned English "by ear".)
Maybe you don't care or you find it annoying to have it pointed out, but it says something about fundamentals. You know, "The way you do one thing is the way you do all things".
But OP isn't native either. He's Austrian.