AI isn't ready to replace human coders for debugging, researchers say | Hacker News Reader

AI isn't ready to replace human coders for debugging, researchers say

arstechnica.com

12 pointskristianp6 comments

It can spot the things you tell them to spot. But, natural language (used to tell the AI what to do) is imprecise and lacks unambiguity. Makes me remember Dijkstra's words: https://willem.com/blog/2025-04-15_vibe-coding/

AI cannot replace people for anything but shitting out large volumes of boilerplate code. It's really irritating that the grifters are so loud that actual smart people are believing them.

If you do, please take a cutting-edge model and ask it to write code for a slightly obscure use case. Unless your thing has been done to death, the model will fumble hard.

We have a word for programs that turn high level instructions into machine code: compilers. They don't lead to fewer software engineers, they lead to more, and a compiler that uses natural language and behaves unpredictably will not change that.

What if I need massive amounts of boilerplate code that is cheap? there are still so many use cases where relatively simple CRUD apps would save massive amounts of time. In my industry, there are still massive organizations using pen and paper doing DVIR logs (just one example)

Sure, but you didn't need more than a few engineers for a CRUD app anyways. The problem with building a useful business is that CRUD generally is not enough, and the actual value-add is something novel that an LLM will choke on since it's not in the training data.

Once you have one CRUD app, Copy/Paste is faster.

I have had AI models perform well on obscure use cases. Even two years ago, let alone the latest models, I've had ChatGPT generate correct code for bespoke requirements that I could not find anywhere else on the Internet. I actually spent a lot of time trawling GitHub and even SourceForge to convince myself this wasn't just a case of "stochastic parrots" regurgitating existing code from some forgotten corner of the Internet. (This also served as an exercise in researching prior art for what I as trying to do.)

In one case it made sense of my own gnarly, hacky, organically-grown, prototyping code that I myself had lost all mental context on and devised the correct solution to my prompt. Two minutes of prompting saved me an hour's worth of software archaeology.

Each time, the model seemed to "understand" the requirements, "grok" the existing code, identify the relevant concepts needed, and apply them to the task at hand. I certainly did the hard work of researching the problem and figuring out the high-level approaches, but the AI was very good at generating fairly complex code to implement those ideas.

While there certainly is a ton of overhyping going on, especially from the "AGI-is-nigh" quarters, the hype is compelling because these models really do succeed at useful tasks much more often than not.

Reply on news.ycombinator.com