Hacker News Reader

Comment by staticassertion | Hacker News Reader

I wish we could just all reject SOC2, it's such a grift. Bankers come in to read some docs (that they don't understand), look at screenshots (that they don't understand), take up 100s of thousands in cash and time, and then write a document (that they don't understand) that no one will read or care about (except to fuel their own SOC2).

The harm is so significant. Tons of these due diligence terms are driven not by any security engineer, but by a legal or compliance team - I've even had members of the security team outright apologize for having to send it over.

View on HN

Being a security person myself, I've found plenty of value in reading SOC2 reports. They're one of the few relatively standardized ways to address the laundry list of vendor due diligence questions.

Is it a good experience for small companies? No. Does it make it easy for your vendors to use cool new technologies? No.

Do I, someone advising on whether or not we should buy something, care about either of those in the moment? Also no. And I spend a rather distressingly large amount of my time trying to talk engineers out of using cool new technology for novelty's sake anyway.

You're absolutely right. SOC2 can, and I assume often does, go quite badly awry and waste literally everyone's time and money. I just know that I've found some value in it. And it helps provide a sound basis for making the vendor agree to assume liability for when they screw up due to grifting.

I think the fact that you don't care about challenges for small companies and new technology is a huge problem. Stopping new entrants into a space is pretty much anti-competition. Preventing new technologies is anti-innovation.

Competition and innovation are the most important things to a healthy economy and market. So essentially, SOC2 both stifles innovation and ruins economies.

You're absolutely right. New entrants, new technologies, and new people in a space are critical for the health of a market.

I just think it's possible that when procuring a tool for a given purpose, a company's chief concern might be about the safety of the tool and vendor rather than the health of the overall market. Your experience may well differ!

Also, I feel the need to clarify my remarks. I, someone advising on whether or not my employer should buy something, do not prioritize the health of the market or cool new technology or a good experience for the vendor over the safety of the tool and the vendor when advising on the purchasing decision. In other circumstances, I can and sometimes do make decisions different. I hope this is removes any misunderstanding that I may have engendered by failing to write clearly.

I don't think it's on people in your position to solve this issue, but I do think it's important for people in your role to understand it and try to help by accommodating security solutions that are not "pay exorbitant fees to known security vendors."

For instance, the industry desperately needs nearly free, open security tools, that are also going to be accepted by people in your role. Too often open source solutions are immediately dismissed by compliance people simply because they are unfamiliar, or because they don't believe open source can be as good, or in the worst case because of propaganda against open source by security tool vendors.

Similarly we need free starter packages and standard templates for processes that small companies can use to get SOC2 equivalent process in practice, without paying hundreds of thousands a year to expensive auditors.

Maybe there should also be a push on vendors not to use SOC2 related security features as an enterprise tier gate. E.g. SAML or SSO is often only available on "you can't afford it" enterprise tier.

There is a lot we can do to fix these problems, but we need people to care, including people in your role.

I am not someone who does not care. Please accept my deepest apologies for my repeated failures to communicate clearly. Please do not hesitate to ask for further clarification if it would be in any way helpful for you.

I fear you have mistaken me for someone who does not care about the health of the ecosystem. Merely because I am someone who advises on what's best for my employer's safety and risk management need not always mean this.

I am, for example, perfectly happy to make use of open source tooling. In many ways, I prefer it. I do not use price tag as a proxy for value. I also do not use cool new technology or vendor immaturity as a proxy for value. Your idea about making it easy for small vendors to understand what they need to do to attain compliance-equivalence is a wonderful one that should be broadly enacted immediately.

Again, please do not hesitate to ask if there is anything else I can clarify for you!

I'm not saying that no human being has ever derived value from a SOC2. I'm saying that the process around SOC2 is a grift.

The fact that it also causes harm to small companies, and detracts from resources for companies that actually care about security, just makes it a particularly harmful grift.

I agree.

I think if there was a better standardized process for getting all the same information, I would use it preferentially in a heartbeat. From experience, trying to replicate all that info-gathering process manually is not a good answer.

This is like saying Agile or DevOps is a mistake because the usual grifters sold a PowerPoint to the C-levels but didn’t change the culture. If you approach anything as a formality, you can definitely minimize the benefit you’ll get but that’s saying more about your organization.

That's the parent's point! The certification itself is completely meaningless. If a company has their SOC2 it basically tells you almost nothing about their security culture.

I've never been at a company that has taken SOC audits seriously -- plenty of companies that take security seriously but SOC compliance is an exercise in paperwork not security.

If that their point, it wasn't well expressed. It doesn't seem right to call something “a grift” because you chose to treat it like a formality when you could have used it as a good faith engagement to actually try to make it better. That's why I made the Agile comparison — organizations can have positive or pointless adoption without that saying more than “you get out of it what you put into it”.

I call it a grift because it really feels like it. It's this weird "you have to do this or else" thing that provides 0 additional value (SOC2 is not prescriptive, it is just a forcing function, you could have... just done it).

A bunch of unqualified people coming in to rank the unrankable in a vague, gameable, expensive way is my definition of a grift. Then, on top of that, you have a sort of ring of high status organizations that will audit you and charge a fortune, but they've convinced all of these other orgs that their audits are Definitely Better (companies audited by them get breached all the time too), and now you've got this whole racket on top of the grift.

Agile isn't forced on anyone. People won't skip over your product because you're "not really doing agile" or whatever. You don't get forced into yearly audits where you pay some outsider with an "agile certification" and 0 technical knowledge to take up multiple engineers for months to write reports.

This post explains more: https://news.ycombinator.com/item?id=24505894

I don't think you understand SOC2 at all.

The business itself defines what controls they want to be tested and included in the SOC2 report. This is why SOC2 reports are simultaneously the worst and best (in most cases, only way to get assurance) assurance you can get over a third party.

The problem is, as you say people who consume SOC2 reports don't understand what they are getting.

Well I think I do understand SOC2 :)

I am well aware of how controls are defined, it was only just an hour ago that I was reviewing our controls with the feedback from our auditor.

So how would you look to get assurance over third parties without SOC reports?

I am happy to concede that SOC reports are terrible, but at the moment they are one of the few effective ways to get assurance over third parties.

As I said in another post,

> This is a very hard problem. It's a regulation on a quality that is a very fast moving target with weak consensus.

Doing this in a regulatory way just isn't viable today.

> the few effective ways

I disagree that they're effective. I don't think they are. Lots of companies with a SOC2 are breached, no company that people consider 'very secure' has earned that reputation due to SOC2 (or any other compliance for that matter).

I found the SOC 2 very useful. Not because I make any Ops or Engineering changes based on the process. It made me think about risks to the business as a whole and the concept of controls to mitigate and monitor those risks.

For example, what is everyone doing to minimize 3rd party risks? How do you know that the whole team understands why PII data should be avoided when possible?

SOC2 is exclusively a forcing function, you could have adopted a NIST framework for risk assessment and controls and gotten that same value, and then had an extra 6 figures of budget to implement high value security work.

Same with the CIS controls which at least come with a tool to test servers against their host benchmarks -- CIS Level 2 is a fairly practical "Server Hardening for Beginners."

What do bankers have to do with SOC2? Do you mean consultants?

I think he means a CPA. Certified public accountant. They are the ones who actually write the report.

perhaps you don't understand the way a typical company operates. security is all but ignored. SOC2 is an absolute necessity. it's a forcing function for baseline processes and controls, that 100% would not exist by default if companies were left to their own devices.

for sure, i'm not talking about the 5% of competent companies.

Let's oversimplify things and say there are two types of companies:

1. Those who care about security

2. Those who do not care about security

For the (1), SOC2 provides no value, because any structure it lends (it lends none, but you'll end up choosing some NIST thing or whatever) is something you could have implemented for much less money. Remember, you'll spend ~1 full security engineer worth of money/ time, so you could hire a FTE to just do these things. Except you won't be constrained in nearly the same ways.

2. Companies that don't care will just grift the grifters. It's simple - there are lots of easy checkboxes, and most of it is just documenting processes. Anyone who's gone through a SOC2 should see how easy it is to "game" it. It's tedious, but a large company will just hire their way out of it, and have a compliance team that's almost certainly isolated from security.

Because it's gameable SOC2 is far easier for large companies to push off. They can hire a compliance team, call it 'security', and move on. Small companies, and/ or companies that care about security, are left having to dedicate their much more limited resources to compliance over implementing meaningful controls. A small company isn't going to know the many 'tricks' for doing minimal work to pass, which is a really important quality of SOC2 - you want the least policy to pass, otherwise you're setting yourself up for either stagnation or an even longer report next year, since changes between reports have to be documented and go through the process. Large companies can just get away with way more.

As one simple example, let's say you have 1 FTE seceng. For compliance, they could spend N% of their time setting up logging, documenting that logging, writing docs on their IR policy, etc. Or, without SOC2, they could spend N% of their time setting up logging, writing good detections, understanding and exploring their infrastructure, documenting in a much more natural way at a lower cost, etc. And then that budget could go towards improving infra, tooling, training, new hires, etc, to do that work even more effectively.

How many breaches has SOC2 stopped? Because clearly it hasn't been the deterrent in many cases - how many companies get owned, while being compliant, due to unpatched vulns (something any auditor is guaranteed to ask about)? What if they'd spent the few hundred thousand a year on a few more seceng? The way companies scale security puts ~10-500 employees to every seceng, meaning that even cutting a few would be a massive increase in risk.

In short, the companies that are already ignoring security will have no problem doing so when they're large, and smaller companies, or companies that do care, will only be drained by SOC2.

edit: I will also say that,

* I won't state that SOC2 is universally useless.

* This is a very hard problem. It's a regulation on a quality that is a very fast moving target with weak consensus.

Reply on news.ycombinator.com