Hacker News Reader

Comment by sothatsit | Hacker News Reader

Agent performance depends massively on the work you do.

For example, I have found Claude Code and Codex to be tremendously helpful for my web development work. But my results for writing Zig are much worse. The gap in usefulness of agents between tasks is very big.

The skill ceiling for using agents is also surprisingly high. Planning before coding, learning agent capabilities, environment setup, and context engineering can make a pretty massive difference to results. This can all be a big time sink though, and I'm not sure if it's really worth it if agents don't already work decently well for the work you do.

But with the performance gaps between domains, and the skill curve, I can definitely understand why there is such a divide between people claiming agents are ridiculously overhyped, and people who claim coding is fundamentally changing.

View on HN

I feel there's a third reason.

When I see a pro-AI person insisting that they are fully automated, I often scour their recent comments to find code or git repos they have shared. You find something every now and again.

My thinking is that I want to use this stuff, but don't find the agentic AI at all effective. I must be doing something wrong! So I should learn from the real world success of others.

A regular pattern is they say they're using vibe coding for complex problems. You check, and they're trivial features.

One egregious example was a basic randomizer to pick a string from a predetermined set, and save that value into an existing table to re-use later.

To me that's a trivial feature, a 15-30 minute task in a codebase I'm familiar with.

For this extremely AI bullish developer it was described as a major feature. The prompts were timestamped and it took them 1/2 day using coding agents.

They were sharing their .claude folder. It had 50 odd md files in it. I sampled a bunch of them and most of them boiled down to:

'You are an expert [dev/QA/architect/PM/tester]. Ultrathink. Be good'.

Worse, I looked at their linkedin, and on paper they looked experienced. Seeing their code, they were not.

There's a subset of the "fully automated" coders who are just bad. They are incapable of judging how bad AI code is. But vocally, and often aggressively, advocate for it.

Some are good, but I just can't replicate their success. And they're clearly also still hand-writing a lot of the code.

Yeah, I definitely see this as well. These are the people with seven MCP servers, 5000-line AGENTS.md files, their own "memory systems" for the agents, and who try to hit their rate-limits on all their agents every 5 hours (regardless of whether or not they are actually getting useful work done). Having tried some of this stuff when I was trying to learn about agents, it almost always made their performance worse...

In web development, where I get the most out of agents, I am still only using them for implementing basic things. I will write anything even moderately complex, as agents often make the wrong assumptions somewhere. And then there's also manual work required to review and tidy up agent output. But there's just so much grunt work in web development from adding to a DB schema, writing a migration, adding the data to your model, exposing it in an API endpoint, and finally showing it on a page. None of that is complicated, so agents are pretty good at it.

Yea, these are the NFT/Crypto bros of the AI world. They don't really understand anything.

The best of them are rediscovering basic software project management and post about it on every social media site and their substack like they discovered something brand new :)

"Turns out if you plan first, then iterate on the plan and split the plan into manageable chunks, development is a lot smoother!!!11 (subscribe to my AI podcast)"

No shit, Sherlock. I wish they read a book once or twice.

Reply on news.ycombinator.com