Hacker News Reader

Comment by Gareth321 | Hacker News Reader

> meaning I am inspecting every single line it changes.

I think this is changing fast. Pressure is increasing on devs for output, and most devs I know are no longer inspecting lines. I have devs in my business unit who claim to not have looked at code for months, except on certain rare occasions. I am not a developer and this week I've been given access to the repo to build my own apps and extensions. There's a "review" between commit and deploy, but there's no chance the Tech Lead can manually review everything, so that's getting done by AI too.

I know this horrifies a lot of devs, but these tools are shockingly good and we are not seeing an increase in bugs. In fact our automated detections (also AI assisted) are reducing the number of customer reported critical bugs.

I really think the days of inspecting every line are over.

View on HN

This type of codebases will become a goldmine for cybersec in the near future. Except by then, only few people will be able to detect them or fix them. LLMs will leave the hardest problems and most difficult bugs plus and plethora of devs who either have skill atrophy or haven't learn these things in the first place.

Actually, cybersecurity seems to be the area that LLMs shine the most at; though it needs directing to actually look for the problem, LLMs have been able to find security faults in code that is well-vetted and written by experts.

I don’t think this is accurate. Our AI testing is finding exploits we’ve had in our codebase for 15 years. No human ever detected it. No deterministic tests. Our apps are getting more secure, not less.

It's an interesting philosophic question. LLMs tend to be overly verbose and defensive in coding. It's not a ton of extra complexity but it makes the code harder to follow for a human. But if a human is not writing the code how much does that matter?

I agree that they have gotten shockingly good. It's been a long time since I've seen them do something that is objectively wrong. Once we get closer to the "too cheap to meter" cost level things will change radically again.

Have you ever tried to use an LLM to add a feature to a really nice, pre 2023 codebase? It’s incredible how much easier it is to do, how much of a difference you instantly feel

I don't think I've ever had a job with a nice code base

> It's not a ton of extra complexity but it makes the code harder to follow for a human.

It is. I have seen Claude chasing after endless amount of edge cases that are just irrelevant in real usage especially for the kind of users we are supporting. At some point you need to stop reasoning about all those cases because it has zero benefit.

But it IS a ton of complexity and blows out context windows. Less code good. More code bad. Both for human and agent. Somehow, still only human can make code less.

Yes and….no. I have seen this play out at a large corp to spectacularly awful results. Talking 60k line react nextjs apps where every use effect has a linter silenced because the ai gave up on writing correct react code.

I have seen millions wasted because someone trusted an ai scripts calculation of a metric from the bottom of the org that led the top of the org to make a wrong decision only to laugh about ai. There is value but ffs read the god damn code. You can have the cake and eat it too. If the volume of code is so large you cannot read it, maybe it isnt worth shipping?

Or are you one of the ones pushing the real code reading on to others which seems to be common. Yes i can have agents vibe out 10 features and have my coworkers suffer fixing it in reviews.

What IS useful are the AI reviews. They catch bugs, not all are bugs but they do catch some. It is almost like they are better at finding logical issues across millions of tokens but not good at writing streamlined logic.

The number of times ai gives me a 800 line dif only to replace it with a 5 line dif after i read it and notice it grossly overcomplicated the ask and scoped in a bunch of nonsense from training data.

Reply on news.ycombinator.com