My experience creating software with LLM coding agents – Part 2 (Tips)

Can you give me the full content of the ideal context of what you mean here?

It’s not though, one of the key gaps right now is that people do not provide enough direction on the tradeoffs they want to make. Generally LLMs will not ask you about them, they will just go off and build. But if you have them ask, they will often come back with important questions about things you did not specify.

I do that as well, with Wispr Flow. But I still forget details that the questions make obvious.

Workflow improvement: Use a repo bundler to make a single file and drop your entire codebase in gemini or chatgpt. Their whole codebase comprehension is great and you can chat for a long time without the api cost. You can even get them to comment on each other's feedback, it's great.

There are however human developers that have built enough general and project-specific expertise to be able to answer these open-ended, poorly-defined requests. In fact, given how often that happens, maybe that’s at the core of what we’re being paid for.

A senior programmer does not suggest adding more complexity/abstraction layers just to say something. An LLM absolutely does, every single time in my experience.

Most definitely, asking the LLM those things is the same as asking (people) on Reddit, Stack Overflow, IRC, or even Hacker News

Caching sounds fine, and it is a very potent method. Nevertheless, I avoid using it until I have almost no other options left, and no good ones. You now have to manage that cache, introduce a potential for hard to debug and rare runtime timing errors, and add a lot of complexity. For me, adding caching should come at the end, when the whole project is finished, you exhausted all your architecture options, and you still need more speed. And I'll add some big warnings, and pray I don't run into too many new issues introduced by caching.

It's better for things that are well isolated and definitely completely "inside the box" with no apparent way for the effects to have an effect outside the module, but you never know when you overlook something, or when some later refactoring leads to the originally sane and clean assumptions to be made invalid without anyone noticing, because whoever does the refactoring only looks at a sub-section of the code. So it is not just a question of getting it right for the current system, but to anticipate that anything that can go wrong might actually indeed go wrong, if I leave enough opportunities (complexity) even in right now well-encapsulated modules.

I mean, it's like having more than one database and you have to use both and keep them in sync. Who does that voluntarily? There's already caching inside many of the lower levels, from SSDs, CPUs, to the OS, and it's complex enough already, and can lead to unexpected behavior. Adding even more of that in the app itself does not appeal to me, if I can help it. I'm just way too stupid for all this complexity, I need it nice and simple. Well, as nice and simple as it gets these days, we seem to be moving towards biological system level complexity in larger IT systems.

If you are not writing the end-system but a library, there is also the possibility that the actual system will do its own caching on a higher level. I would carefully evaluate if there is really a need to do any caching inside my library? Depending on how it is used, the higher level doing it too would likely make that obsolete because the library functions will not be called as often as predicted in the first place.

There is also that you need a very different focus and mindset for the caching code, compared to the code doing the actual work. For caching, you look at very different things than what you think about for the algorithm. For the app you think on a higher level, how to get work done, and for caching you go down into the oily and dirty gear boxes of the machine and check all the shafts and gears and connections. Ideally caching would not be part of the business code at all, but it is hard to avoid and the result is messy, very different kinds of code, dealing with very different problems, close together or even intertwined.

Care to show what you've built?

> The software I've been able to build on a whim over the last 6 months is beyond my wildest dreams from a a year or two ago.

If you don't mind sharing, I'm really curious - what kind of things do you build and what is your skillset?

I would personally never. Do I want to spend all my time reviewing AI code instead of writing? Not really. I also don't like having a worse mental model of the software.

What kind of software are you building that you couldn't before?

> Am I alone in spending $1k+/month on tokens?

I would if there were any positive ROI for these $12k/year, or if it were a small enough fraction of my income. For me, neither are true, so I don’t :).

Like the siblings I would be interested in having your perspective on what kind of thing you do with so many tokens.

I’m unclear how you’re hitting $1k/mo in personal usage. GitHub Copilot charges $0.04 per task with a frontier model in agent mode - and it’s considered expensive. That’s 850 coding tasks per day for $1k/mo, or around 1 per minute in a 16hr day.

I’m not sure a single human could audit & review the output of $1k/mo in tokens from frontier models at the current market rate. I’m not sure they could even audit half that.

You're not alone in using $1k+/month in tokens. But if you are spending that much, you should definitely be on something like Anthropic's Max plan instead of going full API, since it is a fraction of the cost.

I'm starting to notice a pattern with these kinds of comments. They almost never provide any actual evidence for the work they mention.

You're absolutely right - I explained my reasoning poorly. Let me try again: a README.md marks a conceptual project root. It's a way of flagging to developers: "The code in this directory builds, deploys and/or runs separately to other code in this repo." It's a marker that says you need to think of this directory as something that could meaningfully be split out into a separate repo or storage, but isn't because it only makes sense in the context of this repo. It's common in public repos with docs and examples dirs in the project root. The docs are built separately. The examples are meant to be standalone and could be implemented that way even if they actually import code from the parent repo instead of requiring it as a third party dependency.

In this case the lunch is being paid by VC money.

I acknowledge that and get like $400 worth of tokens from my $20 Claude Code Pro subscription every month.

I'm building tools I can use when the VC money runs out or a clear winner gets on top and the prices shoot up to realistic levels.

At that point I've hopefully got enough local compute to run a local model though.

OP here. I was not solicited by anyone nor did I solicit or accept compensation from anyone for this or any other post.

It’s not an advertisement; I apologize if I come off as a Claude Code fanboy.

If you read part 1 of my post (linked in my OP) you will see that I disclosed exactly how much I paid for my usage, and also the reasons that I ended up choosing Claude Code over other agents.

Especially if you hit the rate limits, you should be on the Max plan. It will save you at least $2000/month if you hit the rate-limits on the $200/month plan, and then you can go and spend however much more you want on the API after hitting the rate limits. There are many people showing how they use 3, 4, or even 5 thousand dollars of API credits a month using the Max plan. You're just going to pay those extra thousands of dollars for the sake of it?!

It is insanity to spend thousands of dollars a month when you could be spending hundreds for the exact same product.

It's an absolute no-brainer. And it's not even "either or". You can use both the plan and fallback to the API when you get rate-limited. A 30% discount on tokens cannot match the 90% discount on tokens you get using the plan. The math is so unbelievably in favour of the plan.

There are probably people who are not heavy users where the plan may not make sense. But for heavy users, you are burning piles of your own money by not using Anthropic's Max plan. You only need a week of moderate usage a month and already the plan will have paid for itself compared to paying for API credits directly.

RFC speak is a good way to put it.

Also, there's a big difference between giving general "always on" context (as in agents.md) for vibe coding - like "validate against existing code" etc - versus bouncing ideas in a chat session like your example, where you don't necessarily have a specific approach in mind and burning a few extra tokens for a one off query is no big deal.

Context isn't free (either literally or in terms of processing time) and there's definitely a balance to be found for a given task.