Hacker News Reader

Comment by woodruffw | Hacker News Reader

They often do. The value of those kinds of blanket security audits is questionable, however.

(This is one of the reasons I'm generally pro-OSS for digital infrastructure: security quickly becomes a compliance game at the scale of government, meaning that it's more about diligently completing checklists and demonstrating that diligence than about critically evaluating a component's security. OSS doesn't make software secure, but it does make it easier for the interested public to catch things before they become crises.)

View on HN

Well, the value is ok, if considered seriously.

Also, any certificate bears a certificator company name. We can always say "company A was hacked despite having its security certified by company B". So that company B at least share some blame.

In practice, most commercial attestations/certifications contain enough weasel language that the certifier isn't responsible for anything missed (i.e. reasonable effort only).

But yes, there are many standards for this (e.g. SOC Type 2 reports).

In defense of their utility, the good ones tend to focus on (a) whether a control/policy for a sensitive operation exists at all in the product/company & (b) whether those controls implemented are effectively adhered to during an audited period.

We're talking about getting a judgement in the court of public opinion not a court of law, and no one is exempt from the former.

Many live in a special labelled class that cannot be criticized

That’s not really how they work. The auditor attests that they were provided with evidence that the systems/business units audited were compliant at the time of auditing. That doesn’t mean that the business didn’t intentionally fake the evidence, or that the business is compliant at any time subsequent to the assessment.

An auditor would certainly have some consequences if they were exposed for auditing negligently.

This is how the PCI SSC manages to claim that no compliant merchant/service provider has ever been breached, because they assume being breached means that the breached party was non-compliant at the time of the breach. Which is probably a technically true statement, but is a bit misleading about what they’re actually claiming that means.

Yes, certifiers are not responsible in legal sense, but nothing stops us from posting crap about them on internets.

> The value of those kinds of blanket security audits is questionable,

You're totally right. Why are people afraid to say that they're worthless? Why caveat or equivocate?

Adversaries in computer security do not mince words.

“Worthless” is quite a strong claim. There isn’t much work I’ve encountered that’s truly “worthless”, even though bad work can make me quite upset. Anyways, that’s why I would often caveat.

I'll say they are worthless because most of time they are dragging time away from things that could improve security. For example, $LastJob we spent a ton of time on SOC2 compliance and despite having applications with known vulnerabilities, we got hacked and ended up all over the news. Maybe of instead of spending all the time getting SOC2 compliance finished, we could have worked at upgrading those apps.

Actually, I doubt they would have upgraded the apps and pocketed the profits instead but SOC2 is providing cover instead of real change.

SOC2 covers a set of vectors (mostly social/separation of controls from what I’ve seen), and you were attacked on another vector.

Maybe the org prioritized poorly and sucks overall, but that doesn’t mean SOC2 or compliance generally is worthless.

>SOC2 covers a set of vectors (mostly social/separation of controls from what I’ve seen)

THAT WAS THE PROBLEM. My bad, I thought most hacks were due poor software management but I'm glad SOC2 truly addressed the real problem.

I don’t understand your hostility. Internet strangers are responding to your comments in good faith.

In this particular case it was worthless. If you have known vulnerabilities and you deprioritize that work to waste time on soc2, and get hacked because of it… soc2 was worthless. Because the whole point is security assurance. When you get hacked you’ve proved the opposite of security assurance.

But also you gotta have the balls to stand up to the guy pushing soc2 and say. No. There are known vulnerabilities. We are patching those first then we are doing soc2. The way I frame it is “we know we have critical vulnerabilities, we don’t need to go hunting for more till we fix them. Once we fix them we go looking for other ways to improve security posture” And if the ceo still insists (big client requires it so we’re doing soc2 simultaneously) you say fine, then hire a security consultant so we can go twice as fast. And if he refuses you quit because fuck that place.

That’s some bad prioritizing there Lou.

Mandatory audits by accredited auditors in order to participate in a market, inevitably create a market for accredited auditors that don't uncover too much but ensure all checkboxes are ticked. Much of the security industry is actually selling CYA and not actual security. The same dynamic at play means buyiong a home/boat/car you should get your own inspector, not blindly trust the seller's.

I’d rather understate a medium-confidence opinion than overstate it.

Because it's better than nothing when independent organizations are reviewing systems or other organizations. It's like saying that penetration tests are useless because you cannot prove security with testing.

Even if these govt. security audits are checkboxes, dont they require some nominal pentesting and black box testing, which test for things like SQL injection?

That shoudl have caught these types of exposures?

It may not apply to this specific incident, but pen-testing only ensures you meet a minimum standard at a specific point in time.

I almost feel I could write novels (if only I had time and could adequately structure my thoughts!) on this and adjacent topics but the simple fact is that the SDLC in a lot of enterprises/organizations is fundamentally broken, unfortunately a huge portion of what breaks it tends to occur long before a developer even starts bashing out some code.

Reply on news.ycombinator.com