Hacker News Reader

Comment by giancarlostoro | Hacker News Reader

I can agree with this sentiment. It does not matter how insanely good LLMs become, if you cannot assess it quickly enough. You will ALWAYS want a human to verify and validate, and test the software. There could be a ticking timebomb in there somewhere.

Maybe the real skynet will kill us with ticking time bomb software bugs we blindly accepted.

View on HN

The threshold of supervision keeps rising - and it's going to keep rising.

GPT-2 was barely capable of writing two lines of code. GPT-3.5 could write a simple code snippet, and be right more often than it was wrong. GPT-4 was a leap over that, enabling things like "vibe coding" for small simple projects, and GPT-5 is yet another advancement in the same direction. Each AI upgrade brings forth more capabilities - with every upgrade, the AI can go further before it needs supervision.

I can totally see the amount of supervision an AI needs collapsing to zero within our lifetimes.

In the middle term, I almost feel less productive using modern GPT-5/Claude Sonnet 4 for software dev than prior models, precisely because they are more hands off and less supervised.

Because they generate so much code, that often passes initial tests, looks reasonable, and fails in nonhuman ways, in a pretty opinionated style tbh.

I have less context (and need to spend much more effort and supervision time to get up to speed to learn) to fix, refactor, and integrate the solutions, than if I was only trusting short few line windows at a time.

> I almost feel less productive using modern GPT-5/Claude Sonnet 4 for software dev than prior models, precisely because they are more hands off and less supervised.

That is because you are trained in the old way to writing code: manual crafting of software line by line, slowly, deliberately, thoughtfully. New generations of developers will not use the same workflow as you, just like you do not use the same workflow as folks who programmed punch cards.

No, it's because reading code is slower than writing it.

The only way these tools can possibly be faster for non-trivial work is if you don't give a shit enough about the output to not even read it. And if you can do that and still achieve your goal, chances are your goal wasn't that difficult to begin with.

That's why we're now consistently measuring individuals to be slower using these tools even though many of them feel faster.

"Consistently"? Is there more than just the one METR study that's saying this?

I have measured it myself within my organization, and I know many peers across companies who have done the same. No, I cannot share the data (I wish I could, truly), but I expect that we will begin to see many of these types of studies emerge before long.

The tools are absolutely useful, but they need to be applied in the right places and they are decided not a silver bullet or general-purpose software engineering tool in the manner that they're being billed at present. We still use them despite our finding, but we use them judiciously and where they actually help.

> No, it's because reading code is slower than writing it.

This feels wrong to me, unless we qualify the statement with: "...if you want the exact same level of understanding of it."

Otherwise, the bottleneck in development would be pull/merge request review, not writing the code to do something. But almost always, it's the other way around - someone works on a feature for 3-5 days, the pull/merge request does not really spend the same time in active review. I don't think you need the exact same level of intricate understanding over some code when reviewing it.

It's quite similar with the AI stuff, I often nitpick and want to rework certain bits of code that AI generates (or fix obvious issues with it), but using it for the first version/draft is still easier than trying to approach the issue from zero. Ofc AI won't make you consistently better, but will remove some of the friction and reduce the cognitive load.

I could see it happening in a year or two. Especially in backend. There's only so many different architecture patterns we use, and an LLM will have access to every one that has ever been deployed, every document, every gripe, every research paper, etc.

I mean, I think ultimately the state space in designing a feature is way smaller than, say, go (the game). Maybe a few hundred common patterns and maybe a billion reasonable ways to combine them. I think it's only a matter of time before we ask it to design a feature, and it produces five options that are all better than what we'd have come up with.

That doesn’t require developer time though.

Also that time is needed regardless, do you think it’s the majority of time related to releasing a feature?

In most scenarios I can tell you if I like or dislike a feature much faster than it takes a developer to build it

If it just came down to the "idea guy liking or disliking a feature" things would be quite easy...

why doesn't it? it doesn't have to be you or me personally, it could be a representative sample of our users

So if you wait to put together a representative sample of users and gather the data long enough for the numbers to matter, you’ve gated further changes. If you’ve gated further changes for a week, why does it matter that the feature change was done in an hour or a day?

Releasing it to users does not take a long time. Randomly select 5% of your user base and give them the feature. If your development process was mature, this would be a button you could push in your deployment env.

Deployment isn’t the problem. If you’ve pushed a new version of a feature that five percent of users use per week and you’ve pushed it to five percent of the userbase how long do you need to wait to have reliable data on that before you push another experimental version?

Reply on news.ycombinator.com