The Infosec Apocalypse
blog.rickasaurus.com
blog.rickasaurus.com
Many vendors take a good long time to support new versions of languages, even mainstream ones like Java and the .net family. None of them are particularly helpful in getting this information to you. They have their marketing checklists and information deeper than this can be hard to come by from the salespeople. A few were scared of letting us have this information at all once they knew it was for a competitive analysis. That's a sign that they take a long time to support new language versions.
In many companies adoption of new language versions happen organically at the developer level, often within days of the new version being released. Even if the system admins try to press the brakes a bit on deploying the new version on production servers, the pressure is there for it to happen. SAST vendors typically are not going to be able to keep pace, which will make your developers unhappy or even give them an excuse for not using the expensive tool you purchased.
Many modern frameworks have one or more of: dynamic configuration, compile time annotations, reflection, IoC etc which make it very hard for a "first principles" scanner to make sense of what can actually happen at runtime.
You get much better (read: maybe useful) results if you happen to have a "rule pack" for the specific framework you're using which provides specific hints on sources + sinks, "gotchas" and how things are wired together. As a somewhat obsolete example, I would not expect a sast working only from "first principles" to be able to do anything useful with a Spring XML configuration file.
On the whole my experience is that these things work very well on certain types of codebases - e.g. naive PHP they can "go to town" because of the huge footgun surface and fairly direct control and data transfer.
Stuff with lots of "magical" framework features and indirection (where even a human reviewer can often have trouble finding the implementation from the interface being invoked) they often silently fail to do anything useful.
The part I quoted is called "tainting". Or if you are more academically inclined, "data flow analysis".
When it works right, it is an incredible force multiplier in security. You get detailed, actionable and above all helpful error messages directly from CI, because as a static analysis tool it's pretty fast and can be made part of the common linting pass. As you hinted, it does require a suitable config setup and/or code annotations to mark sources and sinks. And when it does work, it can eliminate an entire class of vulnerabilities - good taint analysis will prevent you from even accidentally using user-supplied data in anything that involves relaying, storing or displaying information.
The downside is, when it doesn't work, it's a source of unhappiness. Debugging a taint failure because the AST analysis gives a false positive can be infuriating.
This can be easy or hard depending on how bespoke your application is. If you're using something like Ruby on Rails, then there's a paved road that a scanner can preconfigure to understand your application. If you're using a homegrown authorization framework, then a scanner will likely have a hard time understanding your application and will need to be configured.
I ran bandit on the code base just for fun one day, and we had four hits and it took 5 minutes to run. It took a while, but I finally convinced the powers that be we were better using the tool we could verify works, rather than trusting the SAST vendor that it worked.
We have had to cancel engagements with suppliers who I have thoroughly enjoyed working with, and whose results were nothing short of spectacular.[ß] And all of this because certain regulators had their rules written by an established auditing firm who managed to make sure that only a select few (less than 5 globally) companies can ever perform their required assessments.
Regulatory capture through security auditing companies should deserve its own circle of Hell.
ß: Have you ever had a pentest report you could forward directly to your engineering teams and be certain that they understood the contents accurately? I have. But I can't use the supplier anymore, because they do not enjoy a royal charter.
The harm is so significant. Tons of these due diligence terms are driven not by any security engineer, but by a legal or compliance team - I've even had members of the security team outright apologize for having to send it over.
for sure, i'm not talking about the 5% of competent companies.
1. Those who care about security
2. Those who do not care about security
For the (1), SOC2 provides no value, because any structure it lends (it lends none, but you'll end up choosing some NIST thing or whatever) is something you could have implemented for much less money. Remember, you'll spend ~1 full security engineer worth of money/ time, so you could hire a FTE to just do these things. Except you won't be constrained in nearly the same ways.
2. Companies that don't care will just grift the grifters. It's simple - there are lots of easy checkboxes, and most of it is just documenting processes. Anyone who's gone through a SOC2 should see how easy it is to "game" it. It's tedious, but a large company will just hire their way out of it, and have a compliance team that's almost certainly isolated from security.
Because it's gameable SOC2 is far easier for large companies to push off. They can hire a compliance team, call it 'security', and move on. Small companies, and/ or companies that care about security, are left having to dedicate their much more limited resources to compliance over implementing meaningful controls. A small company isn't going to know the many 'tricks' for doing minimal work to pass, which is a really important quality of SOC2 - you want the least policy to pass, otherwise you're setting yourself up for either stagnation or an even longer report next year, since changes between reports have to be documented and go through the process. Large companies can just get away with way more.
As one simple example, let's say you have 1 FTE seceng. For compliance, they could spend N% of their time setting up logging, documenting that logging, writing docs on their IR policy, etc. Or, without SOC2, they could spend N% of their time setting up logging, writing good detections, understanding and exploring their infrastructure, documenting in a much more natural way at a lower cost, etc. And then that budget could go towards improving infra, tooling, training, new hires, etc, to do that work even more effectively.
How many breaches has SOC2 stopped? Because clearly it hasn't been the deterrent in many cases - how many companies get owned, while being compliant, due to unpatched vulns (something any auditor is guaranteed to ask about)? What if they'd spent the few hundred thousand a year on a few more seceng? The way companies scale security puts ~10-500 employees to every seceng, meaning that even cutting a few would be a massive increase in risk.
In short, the companies that are already ignoring security will have no problem doing so when they're large, and smaller companies, or companies that do care, will only be drained by SOC2.
edit: I will also say that,
* I won't state that SOC2 is universally useless.
* This is a very hard problem. It's a regulation on a quality that is a very fast moving target with weak consensus.
For example, what is everyone doing to minimize 3rd party risks? How do you know that the whole team understands why PII data should be avoided when possible?
Is it a good experience for small companies? No. Does it make it easy for your vendors to use cool new technologies? No.
Do I, someone advising on whether or not we should buy something, care about either of those in the moment? Also no. And I spend a rather distressingly large amount of my time trying to talk engineers out of using cool new technology for novelty's sake anyway.
You're absolutely right. SOC2 can, and I assume often does, go quite badly awry and waste literally everyone's time and money. I just know that I've found some value in it. And it helps provide a sound basis for making the vendor agree to assume liability for when they screw up due to grifting.
Competition and innovation are the most important things to a healthy economy and market. So essentially, SOC2 both stifles innovation and ruins economies.
I just think it's possible that when procuring a tool for a given purpose, a company's chief concern might be about the safety of the tool and vendor rather than the health of the overall market. Your experience may well differ!
Also, I feel the need to clarify my remarks. I, someone advising on whether or not my employer should buy something, do not prioritize the health of the market or cool new technology or a good experience for the vendor over the safety of the tool and the vendor when advising on the purchasing decision. In other circumstances, I can and sometimes do make decisions different. I hope this is removes any misunderstanding that I may have engendered by failing to write clearly.
For instance, the industry desperately needs nearly free, open security tools, that are also going to be accepted by people in your role. Too often open source solutions are immediately dismissed by compliance people simply because they are unfamiliar, or because they don't believe open source can be as good, or in the worst case because of propaganda against open source by security tool vendors.
Similarly we need free starter packages and standard templates for processes that small companies can use to get SOC2 equivalent process in practice, without paying hundreds of thousands a year to expensive auditors.
Maybe there should also be a push on vendors not to use SOC2 related security features as an enterprise tier gate. E.g. SAML or SSO is often only available on "you can't afford it" enterprise tier.
There is a lot we can do to fix these problems, but we need people to care, including people in your role.
I fear you have mistaken me for someone who does not care about the health of the ecosystem. Merely because I am someone who advises on what's best for my employer's safety and risk management need not always mean this.
I am, for example, perfectly happy to make use of open source tooling. In many ways, I prefer it. I do not use price tag as a proxy for value. I also do not use cool new technology or vendor immaturity as a proxy for value. Your idea about making it easy for small vendors to understand what they need to do to attain compliance-equivalence is a wonderful one that should be broadly enacted immediately.
Again, please do not hesitate to ask if there is anything else I can clarify for you!
The fact that it also causes harm to small companies, and detracts from resources for companies that actually care about security, just makes it a particularly harmful grift.
I think if there was a better standardized process for getting all the same information, I would use it preferentially in a heartbeat. From experience, trying to replicate all that info-gathering process manually is not a good answer.
I've never been at a company that has taken SOC audits seriously -- plenty of companies that take security seriously but SOC compliance is an exercise in paperwork not security.
A bunch of unqualified people coming in to rank the unrankable in a vague, gameable, expensive way is my definition of a grift. Then, on top of that, you have a sort of ring of high status organizations that will audit you and charge a fortune, but they've convinced all of these other orgs that their audits are Definitely Better (companies audited by them get breached all the time too), and now you've got this whole racket on top of the grift.
Agile isn't forced on anyone. People won't skip over your product because you're "not really doing agile" or whatever. You don't get forced into yearly audits where you pay some outsider with an "agile certification" and 0 technical knowledge to take up multiple engineers for months to write reports.
This post explains more: https://news.ycombinator.com/item?id=24505894
The business itself defines what controls they want to be tested and included in the SOC2 report. This is why SOC2 reports are simultaneously the worst and best (in most cases, only way to get assurance) assurance you can get over a third party.
The problem is, as you say people who consume SOC2 reports don't understand what they are getting.
I am well aware of how controls are defined, it was only just an hour ago that I was reviewing our controls with the feedback from our auditor.
I am happy to concede that SOC reports are terrible, but at the moment they are one of the few effective ways to get assurance over third parties.
> This is a very hard problem. It's a regulation on a quality that is a very fast moving target with weak consensus.
Doing this in a regulatory way just isn't viable today.
> the few effective ways
I disagree that they're effective. I don't think they are. Lots of companies with a SOC2 are breached, no company that people consider 'very secure' has earned that reputation due to SOC2 (or any other compliance for that matter).
To me this is a non-issue, because customers almost always ask for types of security checks, not for specific tooling (ie: asking for source code analysis vs asking for veracode). As a rule, compliance/government folks will be concerned about the types of security measures you have in place and not about the specific implementation. Commercial source code analysis tools have varying support depending on language (as others have mentioned: some languages are harder than others). A very valid alternative is to use a linter with security checks (and potential custom rules). The advantage will be that checking will go much faster so you can do it more often (every PR instead of nightly for example). Many security conscious companies have something like this in place.
In general when you're answering security due diligence, it's your job to convince the customer you're going to keep their data safe. They will ask about certain things you don't have and it's your job to explain how you're still solving the underlying problem. Typical example: customers asking for antivirus on all systems and you using (immutable) docker containers.
By the way, the interesting thing here is not the answers to the questions, but how you organise your company to quickly and effectively (as in: no follow up meetings or worse: action plans) answer them. My pet peeve here is "customer guided security": You start from what you think you need (baseline) and you add the security measures that take the longest to explain why you don't have them. That way, you're skating through most of the due diligences and sales velocity goes up, which will make your bosses very happy.
Client security teams have been very reasonable on deviations to their massive spreadsheet checklists.
On one hand, I think that if you, as a vendor, reply back with a few "well, we do X instead of Y in the same spirit" they will probably believe & trust your answers more than a spreadsheet returned in 2 hours with "yes/in compliance" for each question.
It doesn't surprise me in the least that you didn't get any feedback. The default option for these companies is to make you accept their specific blend of security requirements... Of course, you then have to support that forever...
I've had good luck setting up a meeting with both the due diligence person and the actual buyer/champion present. It's often easier to explain your stance in person and the buyer is going to stop the due diligence person when he's getting into the weeds.
If anything, in the long term that's probably a benefit to fancy functional programming languages with complex type systems - they provide much more info for static analysis tools to work with (static analysis is pretty highly related to type theory)
This feature is why small companies can compete with big enterprises. The big ones get the economies of scale, but they also get bogged down by being inagile.
I for one am happy companies ask about this type stuff, it's basic hygiene to keep control over your product's security, really, and the tooling really makes it a lot easier.
And at the same time, I have seen some terrible things that are picked up in code the first time they are scanned, that in theory should have been obvious but were missed for whatever reason.
It gets even worse when you're looking at included libraries.
Also, if you're using these tools, put in requests for new features and languages. This is how we know what customers want and where to focus resources.
It made me realize that we do not know how to make software well enough to regulate it safely, and no, I do not want to go work in a sector where prioirity one is complying with some privacy regulation when the top priority should be accuracy of diagnostic, reliability of a system or eliminating operator error.
Obviously you can't literally have 4 top priorities, but patient privacy isn't some dumb irrelevancy.
Putting a snare where two rabbits are active is just as good or better than putting it where one rabbit is active.
IMHO, this is one of the big excuses they use to push off interoperability. The lack of interoperability combined with the big vendors controlling the standards (i.e. HL7 or whatever) are freezing smaller vendors who might have dramatically better products out of the market.
Mark my words, the big consulting firms will use security compliance as another way to keep smaller and more agile companies (some with dramatically better products) out of the market.
The security community got exactly what they asked for.
Security people were selling fear of insecurity with limited actionable advice for security to come into products/systems bottom-up, so the business has to solve it with process. Breaking into computers is fun and all, but throw around words like "risk analysis" to sound like hot shit for too long and you end up with comprehensive risk analysis process that spans beyond the bits of tech you want to play with.
I work in a highly-regulated domain so software security is just another type of risk analysis we do. So shrug whatevs this doesn't calcify us more than we are already calcified. I just think it's cute that infosec people thought they were hackers, but didn't realize they're another flavor of boring business analyst telling the kids to turn down their music and develop software to their requirements.
As if enterprises were not responsible for not properly budgeting security concerns in their engineering teams. I guess it's easier to just buy a tool that will force a process overhaul, rather than doing a much more thorough process overhaul in the first place. The problem is that tools like vulnerability scanners address only one part of the problem; admittedly, it is a low hanging fruit.
Devs and SRE still have a very important role as SAST and DAST tools only catch a portion of security issues in code and are generally useless for gauging architectural/deployment/runtime issues.
Tooling helps; no question about that. But you can't assume you're engineering team can be oblivious to security concerns. You need to train, hire and equip your teams appropriately. You need to set the right incentives. Just limiting the kinds of software stacks you're allowed to use seems shortsighted.
How are you going to do that, if the internal tool is written in something that is not supported by SAST tooling?
I don't think I've ever seen a company where the security tooling team had that kind of authority or pull. I've seen plenty where the first time the security team hears about a new technology is after product development has started.
You only have to look at the rise of containerization in enterprise to see this in action. When it started tooling was way behind, and it's only catching up now, but that didn't seem to stop anyone.
I look at major enterprises quickly adopting what are still quite new technologies, a good example being the uptake of things that come under the cloud native banner and that doesn't tell me that things are becoming more centralised/controlled.
I've spoken to multiple security teams looking at container security who've said things along the lines of "this is getting deployed whether we want it or not, so we're doing our best to keep up"
Ofc my examples will just be anecdotal, but that's what I'm seeing.
Honestly, this was great, this prevented developers and new graduates from rewriting every goddamn thing in the language du jour.
I hope I never have to work in an organization with 100+ developers that doesn't enforce some standards. It's impossible to join a project and collaborate with other teams when every single developer/project decided to use its own language and make its own deployment system.
Yes.
> Seems like functional programming makes for BETTER security scanning all around.
Yes.
> unless the scanners are just surface level and are adopted as a matter of faith
Kinda. They are very useful, as they catch all those stuff that should have been designed out of the language/framework to start with. They are not something to get cracked, but they also won't discover any deep issue.
The usual FP vs IP argument is not relevant: the LLVM intermediate representation is FP.
Second, security scanning is just part of an overall strategy for increased software quality, which helps the product made out of the software, which is the entire point of writing software. Who cares if your stack calcifies if the user has a better experience because your app crashes less, needs to be emergency-patched less often, and doesn't leak personal data like a fire hose?
I am an SRE, and not a little bit of a security nerd, and I wouldn't trust myself with getting security right.
As you might guess, this drove me nuts. In both cases it eventually blew up in their faces and the events proved to be free of side effects. Their imperative colleagues did not have the same mindsets.
If those two shops are in any way representative, then it may perhaps be worth considering very carefully if keeping cool new technologies away from serious usage could in some scenarios be a win.
I don't think these tools are the answer though, they make it easy for CISOs to look good, driving down the number, but is it real security?
You really need experts thinking about the security of your app. Ideally someone who thinks like a hacker.
In my limited and less-than-universal experience as a security person thinking like a hacker, those scanners can and do enable real security enhancements. Keeping up on your patching is real security, as is having a system that can point out which inputs you didn't validate and what code paths they're on. Couple them with someone with the right experience and background, and you have the basis of a real application security program!
Which is to say that you're absolutely right. Having the right person in the right place is absolutely critical. I think it be possible that it might not always be sufficient.
Most of the time they end up making a 20 page report with 500 issues that nobody reads because 499 of the issues are stupid.
(However, they can work if highly tuned to specific environment and workflow)
Tell me, in this utopia, is there a world government carrying out these executions? Or is it just our great country who is purging it's security experts?
We learned from history and the Bloody Code, that when you have draconian sentences it means people will "upgrade" their crime since there is no difference between say "minor theft" and "major theft," but it also encourages horrible acts to avoid draconian punishments (murder is often involved in high risk crimes that have draconian sentences). But we also have most criminology and justice theory based on this modern principle that the punishment must fit the crime for law to be effective under the punitive system.
On a technical note, it is doing a favor to expose the exploits, as long as it's handled in an appropriate way, I mean that's literally why companies pay a ton of money to white hat hackers, and why there are bug bounties, etc. But even more that's why someone like Edward Snowden is a hero.
We try to keep a high standard of tooling to protect our customers and company data. It's really not about bogging you down, I know it sucks. It sucks hard. It's about ensuring when we upload data to your SaaS, we know it's in good hands that have been vetted.
The good news is, if you make it through it once other big companies start flooding to you as well and it becomes much easier to deal with them as you've been through the intensive process before.
I deal with a lot of deals and if you're building a startup up it's a lot easier to think about security at the start then retrofit and fix it all at the end.
Big FP shops will have solved this of course. I doubt StanChart or Jane Street are losing any sleep over it.
They are mostly famous in tech circles for one day one of their interns Yaron Minsky saying, hey let's rewrite everything in OCaml. And they did, and were wildly successful, and he's the CTO now. They bet big on FP and it happened to be an excellent fit for their problem domain.
Our devs use plenty of small open source projects. We [Security] like to recommend software that we are comfortable with, but any determination of "stacks" we leave to the actual software engineering teams. If something is pretty bad, not updated, constantly having problems, etc - we might ban it... but what's your case for using poorly engineer software given alternatives?
Not sure if mom and pop is supposed to mean commercial, but not OSS? OSS we can patch and modify if necessary. We can even PR patches back based on what our "scanners" and manual testing find.
Generally, we don't care about language, most issues are in implementation not the language, and less so in more modern languages where the creators have heard about security.
The basic type of code scanning needed for PCI and other compliance is a commodity offering and is manageable cost compared to marketing and relationship management costs needed to pursue big clients.
I am not sure the OP understands what a SOC2 report says/does. It talks about pretty high level controls and practices. You certify an app/service, not a stack. If you scan and fix your bugs and have a proactive security training, it doesn't care about how you do it. There is no golden stack that will help you pass a SOC2. You may be able to make your life easier with certain services/SaaS, but the issues come up in your practices and in the actual code implemented. If you have bugs in procedural or functional programming, its the same problem from this perspective.
Vendor due diligence? Some companies have their own questions, there are also agreed standards for these that some companies opt into. I am not sure why a big company should risk their bottom line on something unproven or that isn't ready for prime-time. It's like getting an inspection when you buy a house. In the same way, your org can improve and make improvements. This is no different then adding in features some customer wants in order to win your business.
I don't understand the ultimate point, people who build functional apps shouldn't have to care about security? It's just another non-functional requirement that helps you win a broad audience. It's the same argument that says government should regulate this or that, that financial advisers shouldn't need to act as fiduciaries. It's the cost of doing business.
Maybe the OP has some weird experiences where auditors jumped on functional programming as an issue to justify not doing more work or make their lives easier, but I don't think this is something that is a commonly held belief across audit and security (if people even know what functional programming is).