Hacker News Reader

Comment by pixl97 | Hacker News Reader

>he mentions it is > 20% false positive rate, which wastes a lot of time.

If it's just 20% that is really low. Especially for complicated long chain potential bugs.

Most other detection tools have much higher rates of FP, or much higher rates of false negative.

Then you have humans that miss bugs for 20+ years. Or, they don't tell you about the things they thought were bugs they wasted hours on themselves. Because of this it's really hard to measure how bad/good the AI really is.

It would be interesting to know why the more SOTA models are getting the FPs. Is it from a lack of understanding of C? Is it complex code with deep branches? Is it code smell and convoluted logic?

View on HN

He makes the exact opposite claim in the video. He thinks 20% is far too high and no one will pay for it once these tools are no longer available for free. He specifically cites the company Coverity that also found many useful bugs with their code analyzer but had a much smaller false positive rate (something like 5% I think), and no one paid for that, and the founder had to write a post mortem. He thinks the same’s going to happen to these LLM tools if they stop providing it for free.

There are a number of successful companies around doing just that, it's this guy that wasn't successful at it, not everyone.

Maybe you should just go ahead and watch the video instead of throwing hundred of hypothesis. It'll take just as long and you'll be informed by then.

It's an ~hour-long video! I don't blame anyone for preferring the 2-minute TL;DR if it's available! ¯\_(ツ)_/¯

If there's something wrong or prone to misinterpretation in the TL;DR it would be better to call it out in response to that, rather than the users responding to the TL;DR, simply because it's likely that's what most people will respond to.

Yeah, if the only way to get content is video, I don’t consume it. It’s usually a complete waste of time, unless you speed it up until the speakers are chipmunks. The information density of video is poor compared to text.

Reply on news.ycombinator.com