Comment by Canada | Hacker News Reader

Comment by Canada | Hacker News Reader

Parent Full thread

> The article also claims, without evidence, that "Paid security audits help but won’t find all the bugs, as they tend to be broader than they are deep.". That's quite a claim made on behalf of "paid security audits" -- they're "broader than they are deep". I'm not entirely sure what that means, but there's nothing in the article to indicate where one can find that survey data needed to back up that claim.

The claim is the authors opinion. It’s a blog post not a study. I believe is he correct. Paid security audits are indeed broad and rarely deep.

The reason why is so obvious to anyone who’s engaged in them that what you’re asking for is like demanding a study that restaurants are more crowded around lunch time.

Paid security auditors must deliver vulnerability findings. The end. Deliver findings, get paid. Failing to deliver findings means not getting hired by that client again, or, in the case of bug bounties not getting paid at all. It really cannot be understated that if you are a professional auditor sitting at some client site you absolutely, positively need to be adding findings to the report. If you develop a pattern of not finding things then your colleagues will not respect you or want to work with you anymore.

Therefore, it is only rational to look broadly at the target for common classes of bug. There just isn’t time to deeply understand the target, and it’s foolish to try when everyone else just looking for common mistakes gets more findings and by virtue is more productive/valuable than you.

You're essentially giving us a tautology -- the claim is true because it's true -- about something non-obvious (based on what you say is an opinion) and not evidence-based. Plus you give us a tangent about what happens to those who don't do work they were hired to do which was never the point. You should be more careful about accepting opinion without evidence. There's nothing in "Paid security auditors must deliver vulnerability findings." that means auditors will go no further or are somehow structurally restricted from doing no more than you believe they do. Hence the need for evidence to back and clarify the assertion claimed.

People have been known to make serious errors in judgment before examining evidence because they felt so sure about their widely-shared opinion. Glenn Greenwald pointed this out in regards to Omar Mateen's killings in the Orlando, Florida nightclub ("Pulse") in https://theintercept.com/2018/03/05/as-the-trial-of-omar-mat...

"Mateen’s alleged motive in choosing Pulse — that he wanted to target and kill LGBTs due to some toxic mix of self-hatred over his own sexual orientation and his fealty to Islam — has been treated as unquestionably true in countless media accounts, statements from public officials, and ultimately in the public mind. But ample evidence now affirmatively casts serious doubt about whether there is any truth to this widely accepted belief about Mateen’s motives in attacking Pulse. While some of this conflicting evidence has been reported in the same media outlets that originally disseminated the narrative that Mateen sought to target the LGBT community, it has been downplayed to the point where few in the public are even aware that the original theories about Mateen’s motives have been undermined."

The point being that people formed their view before the evidence was in. After the evidence was examined people learned that their view was nowhere near as justified as they initially believed and told others. They should learn that it's not helpful to guess about why something happened without the evidence. Although what we're talking about in this thread could theoretically involve life and death matters (whereas the Pulse murders objectively do), I think there's something to learn in calling for evidence before formulating one's opinion. Something that blog post doesn't do and we'd be wise to not join the author in that framing of the issue.

I don't have strong opinions on what motivation that murderer may have had. I just don't have any experience shooting up gay nightclubs that I could draw on to help me offer insight into that one.

I do strongly support the claim that security audits are usually broader than deep because I do that kind of work and am personally familiar with the pressures and motivations described.

In my own experience, and in the experience of all of my friends who do this work, with whom I have discussed this exact topic at great length, the majority of the time we are in fact structurally restricted from going deeper. It doesn't mean this is all the time, but it is the case for most of the billable hours doing this stuff. It pays reliably going broad and not deep.

But you're totally right. That's just like, my opinion man.

Reply on news.ycombinator.com