Comment by socalgal2 | Hacker News Reader

Comment by socalgal2 | Hacker News Reader

> We are 4 years into this ‘revolution’ and the industry has almost nothing to show for it. Software has not improved in quality,

Apple, Mozillia, and Firefox just released a record number of bug fixes because of AI. So this assertion that Software as not improved in quality is arguably if not provably false. There are 1000s of other examples. I have my own. Personal projects that were stopped because of dependency rot, having 100s of bugs fixed, nearly all dependencies removed, and lots of tests added just by asking. I know tons of others with similar experiences

> I do not feel like I’m falling behind by not using them

Obviously you are correct about "how you feel". But, this argument to me is more like a horse owner claming they don't feel like they're failling behind to trucks that can carry 10x, 100x, 1000x more than their horse. Sure, maybe the horse can go places the truck can't. For most uses the truck is better than the horse.

> The LLM-generated PRs I get are still rubbish.

I can't argue with that. I haven't seed the LLM-generated PRs "you" get. I can say though the LLM-generated bug reports I've seen are 1000x better than human ones. Human bug report "your software breaks, fix it!". LLM bug report "There is a bug in this specfic part of the code for these specific reasons and here's a repo I wrote to demonstrate the bug. Here's instructions on how to run the repo. There's the code to fix the bug. And, here's a test to show the bug is fixed."

Is it perfect? No, sometimes the bug isn't real because the code can only be reached in situtations that can never happen. But still, 7 of 8 times it's a real bug and the bug report are the best I've ever received.

you can go through this list: https://www.firefox.com/en-US/firefox/150.0/releasenotes/ and for those very few issues not still blocked you can read the examples

>For most uses the truck is better than the horse.

I think you're talking past the author here, who says:

>By not leaning into LLMs I’ve given myself the room to specialise in a smaller set of skills. So far that conscious decision is working out well, and I remain confident about my job security.

"Most uses" might mean boilerplate and simple frontend work that LLMs can do easily because it's formulaic and doesn't require much creativity. OP is simply not doing that type of work.

I don't think anyone sane is really claiming 100x or 1000x speedups anymore. Most people I talk to who use LLMs are closer to 1.5 or 2. Because their job is not mostly boilerplate, there is simply not that much room to be improved.

It’s not about speed-ups, it’s the fact that I’ll attempt a project far more ambitious than I would have in the past.

Over the past two months I’ve built out a very complex web app for a client. Without the aid of AI it would be far smaller, simpler and less capable. And honestly I probably wouldn’t have agreed to take on the job in the first place.

> far smaller, simpler and less capable

So, better, in other words?

Sure, in the same way that nano is better than Microsoft Word.

Unfortunately, sometimes nano is not enough.

In the case of this client project, we are only now getting to enough functionality to be generally useful. The complexity that exists in the project is necessary complexity. A lot of coders are simply wrong about complexity, and feel it can always be reduced.

Complexity in software in general is an evil thing, to be feared and fought and avoided at all costs.

Niklaus Wirth pointed this out over 30 years ago…

https://ieeexplore.ieee.org/document/348001

… and it has grown exponentially worse since.

I think that the software world ought to fear and dread complexity, the same way that the Unix designers feared and dreaded verbosity.

« Note the obsessive use of abbreviations and avoidance of capital letters; this is a system invented by people to whom repetitive stress disorder is what black lung is to miners. Long names get worn down to three-letter nubbins, like stones smoothed by a river. »

https://web.stanford.edu/class/cs81n/command.txt

We ought to be religiously obsessive about avoiding it. In the end, everyone would benefit thereby.

there's no real difference between an excess of boilerplate and slop.

if your job is just writing boilerplate you're probably doing it badly.

if your app is so simple that really all you need to do is sling boilerplate then it might suffice but it's probably still the most unreliable way to develop that software.

if flakiness, bugginess and unreliability are all fine then yea, boilerplate away.

>Apple, Mozillia, and Firefox just released a record number of bug fixes because of AI.

A hidden metric here is the number of new bugs created by these fixes. If an LLM creates 10x bugs but create 5x more new bugs, then it is not really an improvement. Because that is 5x more bugs that the user has to observe and report (unless it is a security vulnerability, which the LLMs might detect) before it can be fixed.

Also there's the "let's make time to work on those bugs by using this new tool". Time that wouldn't have been allocated otherwise.

...allocated on not fixing the bugs, sure. and AI does not decide what you spend your time on.

The LLMs find the bugs and the users will get exploited unless fixed. Sure am glad the choose fix them.

This is effectively a claim that Mozillia, Apple, Google's developers are idiots and irresponsible and just accepted the fixes without checking them. That's a pretty strong claim which I think you'd have a hard time proving or even finding any evidence for

I mean no such claim. One does not have to be an idiot or irresponsible to overlook an unintended consequence of some code change in a very large, legacy code base.

If that claim were even reasonable in the first place you would have to accept that they were idiots long before by allowing bugs to be introduced in the first place.

The presence or introduction of bugs in software does not mean it’s developers are ‘idiots’ unless practically everyone who has ever written code is an ‘idiot’

Yes I didn't say this. I'm using it to show how absurd the original claim was.

I think LLMs are particularly effective at turning once-competent developers into idiots. (“Just ask Claude.”)

> A hidden metric here is the number of new bugs created by these fixes.

If the project has proper test coverage, there should be no significant number of new bugs. This is no different than the possible regressions added by a human-implemented feature.

And LLMs have made implementing a massive number of tests far easier and faster than before the LLM era.

LLMs are very helpful as a debugging aid, yes, but in large part because the fixes tend to be small and verifiable. That this does not carry over to many other use cases is the crux of the problem.

I myself use them to accelerate programming tasks, so I'm not anywhere near as pessimistic as the author, but the claimed multiples of productivity definitely haven't materialized for me.

> Obviously you are correct about "how you feel".

I posted the link. I didn't write it. My name is Liam Proven, thus "lproven", while they are Joshua Barretto, thus "jsbarretto".

Not the 1000x more productive numbers again please. Is this a meme or something? If you said +17% I might have taken you seriously.

I gave the 1000x example because a single semi-truck can generally carry 240x what a horse can carry. A horse can carry 150-200lbs. A semi-truck can carry 42,000 to 48,000 pounds. The largest truck carries 2.2 million pounds or 11000x the horse.

To see an example of this 1000x, all we have to do is look at the flood of bug reports and fixes to browsers and OSes. Does it do 1000x for every project? No, and neither does a pickup truck do 1000x what a horse does. But, even a small pickup truck does more than a horse for most use cases (carrying people or stuff) and similarly, LLMs do more for most use cases.

Reply on news.ycombinator.com