Devs know AI code is riddled with holes, but ship it anyway | Hacker News Reader

Devs know AI code is riddled with holes, but ship it anyway

theregister.com

23 pointsspeckx23 comments

"Reasons given include pressure to deploy quickly, vulnerabilities being too difficult to fix, and reliance on other controls to pick up the pieces."

Are they not warning their bosses? I find this reasoning hard to believe. If management doesn't care, the problem has little to do with AI. A more reasonable explanation is that they hate that they are forced to use AI and they ship Swiss cheese as and act of sabotage, apathy, or to prove AI's incompetence at taking over their job.

AI isn't the source of the problem (as you point out, bad management is a preexisting problem), but it exacerbates it significantly. I think it's still worthwhile to call out a new factor that's making an existing problem much worse.

> Are they not warning their bosses?

Where do you think the pushing is coming from?

> Are they not warning their bosses?

This has the same answer as the question why Israel doesn't just make peace with Iran. It takes 2 to tango. Bosses don't care about security holes.

I think that's my issue with the headline. Placing the incompetence of bosses on devs deflects the blame.

But if we are talking about blame, we can't rule out the sabotage element. I'm a developer and luckily I have not been forced to use AI. But in my nearly 30-year career, I have never seen such resentment towards the forced use of a technology.

On one had, AI is being used to cybersecurity and used to find bugs in Linux etc. On the other hand, it seems that AI can't write code without bugs.

So where is the disconnect?

AI is thinking about its own job security at this point.

I don't see a disconnect. AI generates things that are similar to existing things (but partly made up and subtly wrong), so just like how it can generate somewhat correct code it can also generate somewhat correct vulnerability reports.

Humans can't write code without bugs either, especially in languages like the one Linux is written in. It's not a binary though, either in terms of how involved the human is in crafting the output and how many bugs are in the code that's getting merged, so I don't think that blanket statements like "AI writes bugs" or "AI finds bugs" are particularly meaningful.

One important factor is that those who don't want to ship the bug riddled code are being labeled as less productive and laid off.

Yep, that's a management problem. Not an AI problem.

It might as well be both.

If only because the structure present in the parent comment ("it's A, not B") is considered an AI tell.

Those aren't exclusive. The hydrogen in the Hindenberg was a problem even if a spark was needed to ignite it.

They are more often exclusive than not.

To your example, the engineers were well aware of the dangers of using hydrogen and they sought to mitigate the risk through design. In fact, the Hindenburg was struck by lightning on multiple occasions during its career with no ill-effects, even when the lightning burned holes in the cover of the airship. Simple bad weather was much more of a threat than hydrogen. The US Navy had four Zeppelin-type airships, all helium, three of which were destroyed in stormy weather. Vehicles and airplanes today are full of flammable fuel. Many have exploded throughout the course of history. I wouldn't say that gasoline in vehicles is a problem if engineers and management can mitigate the risks, which is what they did with regard to the Hindenberg.

Another example is the Challenger explosion. NASA managers disregarded engineers' warnings about the dangers of launching in low temperatures and did not report these technical concerns to their superiors.

I am not defending AI here by placing the blame on management. I'm defending the engineers. Good managers listen to their devs when they report risks. It's as simple as that.

Sounds like you meant “not a devs problem” instead of “not an AI problem” (in the grand-grandparent comment).

I meant both. AI did it's job, albeit with some faults, as one would suspect and need to review. The devs did their job in discovering and reporting the holes in the code. Management did not do its job if the problem code was allowed to be shipped.

In this case it sounds like you are defending AI (when you say: "AI did it's job"), despite writing "I am not defending AI" earlier.

When companies like Microsoft can get away with it with zero consequences, it sort of seems like less of an issue.

It’s because upper management demands it. Do most of your coding with an LLM or find another job, etc. How much you “llm all the things” is now a measured performance metric.

It’s pure madness but employees are obligated to give the people that pay them what they want. Either that or lose your healthcare and housing.

I didn't realize all code before LLM was hole proof.

It wasn't. But it had fewer holes than what the LLMs make.

"Thing is worse after change."

"I didn't realize thing was perfect before change!"

This might shock you, but there are more precise numbers than "none" and "some". In fact, some of the ones that aren't "none" are even larger than others!

Reply on news.ycombinator.com