Why are coding agents so dumb?

> All these problems are solved.

> True, communication could be better.

Nitpick, but bit of a contradiction.

Just using subagents was a pretty useful gain. My only issue is sometimes I forgot to tell the agent to use them. Probably need a hook or skill to do it so I don't need to remember.

Funny the sandbox is a real security benefit but it just turned into an extra confirmation for me. I'd rather have it than not.

what other sandbox do you need besides running a harness in a container with the right folder mounted and some due diligence like -nonewprivileges? are we talking host network sandboxes here? I think he's just referring to a rogue agent running rm -fdr --no-preserve-root, and that's safe in a container.

Same here. I have a feeling that once the AI hype cycle blows over to a degree (not that it will go away, just calm down), we'll be left with an enhanced version of pre-boom development. LLMs are hitting a dead end. I'm not convinced they can be bandaid-fixed to become AGI or SI or whatever the cool new trendy term is. They're good at many things, but sustaining a lean and directed software project's architecture over long periods of time doesn't seem to be one of them. That still remains to be further proven over time, but I've already seen plenty of examples of this.

Also, "esoteric abandonware" is a great way to put it. I keep looking for solutions to problems, finding dead projects that haven't been committed to in months, they have all the indicators of AI slopification (the primary one for me being emoji-filled READMEs). I don't think these types of projects will be around for the long-haul. These projects are just as much about community and people as they are about code. And people are much less likely to care about a short-lived project that doesn't have the backing of people that intuitively understand the internals of it. It's like building on sand.

Personally, I've decided that allowing the "agents" (these buzzwords are quickly becoming pet peeves of mine) to write the code for me is a bad idea, because the faster the LLM constructs architecture than I can, the less I'm absorbing the details of it, and the less likely I am to intuit obvious shortcomings. They do help me work much faster than before though, because I still use them like a research/learning assistant chatbot. Learning what I'm doing wrong faster, while still being the one putting the pieces together, is very great.

A couple of suggestions from the other side of this:

* Hold on to that isolating feeling. Those of us who feel AI is an incredible accelerator also feel just as isolated as you do. We wonder why you aren't seeing what we are seeing - and that isn't criticism, we just genuinely don't understand where the gap is and what would help other people jump across.

* I wouldn't bother looking at GitHub, or using that as any kind of measure. Personally getting away from all dependencies is my goal, and avoids a lot of supply chain issues.

* I don't use any plugins or external skills. I do use a ton of self built local MCP tools though, and it drives much of my workflow. That's me though.

* You sometimes need to work up to it. I've had periods of hitting a wall this year with LLMs. Once I end up building a framework, and the framework is in place, everything flows. The models can learn from looking at your adjacent projects, and they just fly. That's for coding, but it's similar for solving other problems... record everything, give it a ton of context, etc.

* If the software "kinda works kinda doesn't", then someone isn't doing enough testing. You should be dogfooding your software every day. Use your AIs to help you find edge cases, sure, but they're alien minds. Your human mind will sometimes think in ways they don't, but that your customers will. Sometimes we write software for AI to use, sure, but if you're writing for humans, you still should add some human mind thinking just to double check the model's work. "That's right, it goes in the square hole."

It's okay to struggle, some of us have been building toolkits over the last two years that make AI work so much easier. But don't get sucked into thinking "it doesn't work", especially in the Opus 5.5 era. On my side of the fence, these last two weeks have been the most intense and productive in a long time.

Lastly: some of us have tips we will not share. Some is slight selfishness - if others haven't discovered what the models can do, that is a small moat. But there are also some tips you just have to find on your own. I could tell you playing The Witness is some of the best training you could have for working with AI, and many will think that makes no sense at all. But I think a handful will nod and quietly understand, the lesson can't be taught, it must be learned.

Sorry for the long ramble, but I genuinely hope some of that helps.

I'm also in the boat that says things should be better, but instead of improving the agent I decided to try to implement the agent management (in: https://beolis.com).

It still leverages agents (people are used to them already) but can deal with executing it on a different vm, dealing with git worktrees, etc (not that coding agents themselves cannot be improved, but I think the sw development infrastructure around it has a lot to improve too).

A perfect polished artifact is not important. Every person can just vibe their own good enough solution. And that’s fine.

Came here to write exactly that. Especially about Pi, which they claim they've used.

Isn't this the point of Pi to implement all the features you need yourself? Complaining about lack of features in Pi doesn't make sense to me, it's their whole identity.

Although I must say that describing specific problems is valuable on its own. It's just (a) why stop there and (b) if stop there, why shape it as a complaint and not as a list of features that are worth discussing.

I am building a mostly autonomous coding agent [1] that takes a task from jira/github/gdocs and codes it to a PR autonomously.

It works pretty well with Opus/Sol or better types of models. But with Sonnet/Terra level, it chases its tail and is pretty crappy.

I'm just waiting until Chinese models get to the required level, its going to be great for the cost.

[1] https://github.com/obaqueiro/coderbot

These are very hard problems to solve.

It's often nearly impossible to know in advance how hard the task will be, so the model has to guess, and will inevitably miss. Just because you know how hard the task is, doesn't mean it's obvious. Often it's only obvious to you, because you have additional context that models are missing.

A bad guess can lead to either context rot (if subagent wasn't used), or slower execution (if subagent was used but it turned out to be a bad idea). Both are bad UX.

Now, we could improve accuracy by having the harness do a research on each task before executing... which leads to even slower execution, also bad UX.

Instead of trying to guess the difficulty level for each task, I think it’s better to define the model for each role in advance. It’s hard to know the difficulty level ahead of time, but we do know the roles in advance.

I've been thinking an AI could be trained to bid on jobs. Probably too many real world problems to make it practical, but I can't get it out of my head.

No, but it will do the opposite You can choose Sonnet and set "/advisor opus" and it will reach up when it thinks it needs to.

OTOH, would you trust the vendor to pick the best model for you? Would you trust them not to prioritize their own load-balancing concerns first?

The descriptions are near-useless and tend to flip around, as model families are not released in sync anymore, that's true, but fortunately, thanks in a big way to subscription pricing, the choice is simple: start with the best model on offer, and when you run out of quota, downgrade to the next best (or briefly switch providers).

OP here.

I didn't want to complain about OpenCode too much because it's open-source and kind of the scrappy underdog to Claude/Codex, but I do find its multitasking support pretty limited. I have to specifically remind it to spin up subagents, and when it does, it'll do 2-3 tasks in parallel and then wait until all of them finish. It can't seem to inspect tasks in progress, and if a subagent dies or hangs, the main agent can't seem to retrieve any information from it, even though I as the human can look at the session and see what happened.

Are you doing anything special to make OpenCode multitask well?

Yeah I regularly have 5+ agents working in 1 repo simultaneously even modifying the same file. Since they were all running on my singular local machine. Builds and tests were interfering at 1 point so 1 automatically proposed writing a single script that basically mutex locked it with appropriate wait and timeouts. It had even added this to my own personal agentic todo backlog. They've proposed new skills for me.

They even have split my decisions to human decisions. Proposed and approved work. They can iterate on approved work without me just fine.

And I only just started with agentic coding in last few weeks before that I was mostly a copy paste chat person.

I've now been trying for six months to get agents to produce decent code. As in readable / easy to follow, easy to change. I'm doing something really wrong. It's killing me. Everything it produces will work, buts it diabolically over complicated. I've built skills that have helped. But not massively. I've used other people's skills, in particular Matt pococks grill me and Dex hortlys show me. These have helped a bit. I work in enterprise, I want to be proud of what I'm producing, but trying to understand and then cajole and agents code into something that's good is exhausting. If anyone here has been through this and can share how they got through this, I'd really appreciate the help.

Edit: I have access to codex, vscode, GitHub co pilot cli and all anthropic and openai models (excluding mythos).

Hard disagree: 1. Writing every piece of software you want to use yourself is unreasonable, 2. The power to change doesn't make it wrong to complain about valid shortcomings. If my ISP is unstable, I can probably change but it's still a reasonable human reaction to complain. "Never complain" is a nice philosophy but really hard to stick to forever.

But you didn't just write the thing you wanted, you're relying on other software that happens to do it, and if you were happy with the communication bus you would've stuck with that.

Your shoe analogy also breaks down because really the shoe is the issue, not the stone. And expecting everybody to make their own shoes is, well, I mean we just don't do it that way anymore for good reason. Let the cobblers make the shoes, and the runners wear them.

I upvoted it and now my Anthropic account has been suspended

OP here.

Looks like this is a layer on top of OpenCode. Seems interesting. I'll check it out. Thanks for the tip!

So that's how model distillation is done. Just ask for the source code directly.

I think definitionally the model can't know about the harness, since the model is at least a few months out of date. So every model must be behind it's harness or something really weird is going on.

Makes me wonder about doing some archaeology and trying out really old harnesses on modern models...

OP here.

I don't expect the model to be trained on the latest features of the harness, but I think the harness should ship with its own docs so that the model can quickly search local docs rather than search online page by page for docs that don't necessarily match the local harness.

So include the docs like what Pi already has been doing since the beginning.