Comment by dataviz1000 | Hacker News Reader

Comment by dataviz1000 | Hacker News Reader

Parent Full thread

Have you noticed that the coding agents get really close to the solution on the first one shot and then require tons of work to get that last 10% or 5%?

If we shift the paradigm of how we approach a coding problem, the coding agents can close that gap. Ten years ago every 10 or 15 minutes I would stop coding and start refactoring, testing, and analyzing making sure everything is perfect before proceeding because a bug will corrupt any downstream code. The coding agents don't and can't do this. They keep that bug or malformed architecture as they continue.

The instinct is to get the coding agents to stop at these points. However, that is impossible for several reasons. Instead, because it is very cheap, we should find the first place the agent made a mistake and update the prompt. Instead of fixing it, delete all the code (because it is very cheap), and run from the top. Continue this iteration process until the prompt yields the perfect code.

Ah, but you say, that is a lot of work done by a human! That is the whole point. The humans are still needed. The process using the tool like this yields 10x speed at writing code.

This was often true when writing code manually to be fair.

You could get to "something that works" rather fast but it took a long time to 1) evaluate other options (maybe before, maybe after), 2) refine it, 3) test it and build confidence around it.

I think your point stands but no one really knows where. The next year or so is going to be everyone trying to figure that out (this is also why we hear a lot of "we need to reinvent github")

When I hire fresh out of college… I can see them coming in and not having the slightest comprehension of the difference of the things that they did in school to get a grade and never touch it again versus a product that is supposed to exist and work for 10+ years.

The problem of life in general is the last 5-10% is always the hardest. And it makes no economic sense in many cases to invest in trying to make that last part mechanised.

I believe the llm providers went with the wrong approach from the off - the focus should’ve been on complementing labour not displacement. And I believe they have learned an expensive lesson along the way.

> Ah, but you say, that is a lot of work done by a human! That is the whole point. The humans are still needed. The process using the tool like this yields 10x speed at writing code.

Shame that what is left for the humans is the shitty, tedious part of the work.. It reminds me of the quote:

   I want AI to do my laundry and dishes so that I can do art and writing, not for AI to do my art and writing so that I can do laundry and dishes..

I tend to get something working and refactor my way out, which does work and you can use a coding agent to do it, but it takes time. Maybe starting over would have been better, but I didn’t know what I wanted the architecture to look like at the beginning.

I can go long session with it making great code.

But the first time I say “No, it should be …” it’s nearly game over. If you say it 3+ times in a row, you’re basically doomed.

Sure, you can get it to fix the bug, but it comes at the cost of future prompts often barely working.

I second that experience.

The moment I hit the "no, it should be.." point, I know it's the end of it.

Sometimes I can salvage something by asking for a summary of the work and reasoning done, and doing a fresh restart. But often times, it's manual corrections and full restart from there.

That will not work as cleanly as you described once a lot of code has been committed to the code base. You cannot just blow away an entire working code base and start over just because an LLM is struggling to make a feature work with existing architecture.

This happened on every single greeenfield project that I've started with AI, no matter how rigorous process I've had defined.

And it's not just easier because it's cheap, it's easier because you're not emotionally attached to that code. Just let it produce slop, log what worked, what didn't, nuke the project and start over.

It just gets incredibly boring.

People will get attached to code that works just right and they don’t want to mess with it too much.

Yes! Anthropic team calls this “regenerate, don’t fix.”

The person who builds an agentic IDE or GitHub alternative that natively does the process you describe will be a multibillionare.

> https://github.com/adam-s/agent-tuning

Do you want a demo of what this is capable of?

Reply on news.ycombinator.com