Hacker News Reader

Comment by thewizzardofnl | Hacker News Reader

There is a difference here. The "new coding analysis tool" that you use for the analogy here is getting better every few weeks with the release of new models.

It is plausible to assume that, for instance, a Linux kernel that was hardened for CVEs that Sonnet 3.5 could detect is not hardened for bugs that Sonnet 4.5, 5.5, Opus, Fable, and models in 2027 can and will be able to detect.

Hence, it is rather a constant catch-up game until the LLM improvements might hit a ceiling and won't get any better in this regard.

View on HN

You assume an infinite level of brokenness is legacy code bases. Granted, working on those things one can get the impression, but I think stable code converges to a low level of bugs after sufficient scrutiny and superhuman scrutiny doesn't reveal a never ending deluge of new problems. Not to mention that the bug-chains that you need for a successful exploit keep getting longer very quickly as more problems are discovered and the code is hardened.

I expect that the incremental model improvements will become smaller and smaller until they run into the same diminishing returns effect like all other new technologies (FWIW I've not beeen seeing a lot of difference between the latest Opus and Fable models for the stuff I'm doing, so I just stick to Opus for most things. There's a noticeable difference between Sonnet and Opus though).

I'm not ready to take any bets when exactly the curve will be flat though ;) (e.g. in a just couple of months or a couple of years)

You don't even have to assume. Mythos unveiled a ton of bugs across the ecosystem back in April, so this isn't the first iteration of the cycle.

Search for "Security in the LLM age" in this HN page for a reality check, apparently 80% of the security vulnerabilities that Mythos initially flagged in the Linux kernel turned out to be false positives. Better than nothing of course, but it really doesn't look like Mythos is quite the "wonder weapon" it was marketed as (and the situation by far isn't as dire as the initial flurry of Mythos news).

IIRC, in April, Mythos found 20x the monthly average? 20% of that is still a single system doing something that takes a team of engineers 4 months.

Like, "4x as powerful as a team of engineers" is still really quite impressive

The problem isn't the 20% actual bugs found (that's great), but the 80% false positive rate which are reported with high confidence and (most likely) misleading reproduction code. An experienced programmer familiar with the code base first needs to validate all reports and throw away 4 in 5. That's a massive waste of time. If a traditional static analyzer had an 80% false positive rate nobody would take it serious.

In my hobby projects I use LLMs in my development workflow mainly for passive bug scanning, reviewing and helping to maintain tests, they are definitely useful for catching some bugs early and noticing unhandled edge cases, but they're also definitely no silver bullet (they sometimes ignore quite obvious bugs, and the fewer 'obvious' bugs remain the more one has to be careful about false positives). E.g. the funny thing is that now I'm actually slower than before due to the intense 'rubber ducking' with LLMs and cross-checking their results, but I still want to pretend that the resulting code is more robust out of the door.

Eg everything that Greg KH says in the video sounds very familiar, and it's very disappointing that Mythos still suffers from the same issues (or maybe even worse) as older models.

> An experienced programmer familiar with the code base first needs to validate all reports and throw away 4 in 5. That's a massive waste of time.

Okay, but again, even with all that extra effort, they found 4x as many bugs, so it seems like the effort is clearly worth it.

And each model gets more reliable, we get better at building proper reproduction code, etc. - this was mostly a comment about the cycle, direction, and velocity we should expect from the future, given this has already happened twice.

Reply on news.ycombinator.com