Hacker News Reader

Comment by slt2021 | Hacker News Reader

slt2021

Fully agree with the author, the often quoted CIA (Confidentiality Integrity Availability) actually belongs to the SRE platform's concern.

What I often see however, is SRE teams keep extremely understaffed (because "cost center") and often outsourced to offshore contractor in India - this basically means you can only keep the lights on via SRE team and cannot do anything else.

If you have a competent and fully staffed SRE Platform engineering team - you will NOT need a separate cybersecurity team.

But if company decided to cut costs on SRE folks, they then will have to invest heavily in cybersecurity team. At least because of mandated regulation (FedRAMP SOX HIPAA GDPR compliance disclosure etc) - it is easier to justify cyber budget, than the SRE budget

Plus recent laws mandate having CISO as a high level executive (chief incident scapegoat officer) with huge budget and disclosure of recent hacks etc

View on HN

> If you have a competent and fully staffed SRE Platform engineering team - you will NOT need a separate cybersecurity team.

Not all cybersecurity problems are platform problems, or even technical problems. Even if you decide to gather all AppSec/InfraSec engineers inside of SRE/SysAdmin, this creates two problems:

1 - They are now separate from the main cybersecurity team, which is solving other problems and that in turn creates all sorts of issues ranging from lack of coordination to managerial ones.

2 - This creates a HUGE lack of separation of duties (or a conflict of interest). The executor of a task should never be the auditor of a task. You cannot expect the SRE team to go out of their way to keep finding flaws in their own work and exposing them to the board so they can be forced to reprioritize and fix these flaws. The SRE can and should have security built-in as much as possible, but an external team is still required.

SRE is really a more familiar "sysadmin" from good ole days. You know, that bearded IT guru in a basement floor, will curse and shout at you, but will keep your company running when it comes to IT.

if you switch back to sysadmin model - having a single sysadmin do all the SRE and security work, divided per platform, then separate security is indeed feel like extra.

You can obviously hire independent third party security auditors and do pentest, but that thing could be done once a year. Often times your financial auditors (Big 4 accounting firms) will offer IT audit and pentest in a bundle and you can get a good deal.

>If you have a competent and fully staffed SRE Platform engineering team - you will NOT need a separate cybersecurity team.

So the SRE team is supposed to find vulnerabilities in the codebase and pentest the network?

I am gonna surprise you, but nobody from cybersecurity is finding vulnerabilities either.

ALL of cybersecurity is just operating and configuring a platform, whether it is Qualys or Snyk or Wiz or something else. "cyber" people are basically TOOL MONKEYS, and if SRE people can handle operating and configuring kubernetes - they sure as hell are capable of operating and configuring Qualys and Snyk and Wyz platforms.

Pentest is the same, I have not met any inhouse dedicated pentest folks, it is always third party contractor hired as needed for a short period of time. All inhouse fulltime cyber engineers are dedicated to blue team operations, rather than red team operations.

As part of the industry, I can tell you I've personally found dozens (hundreds?) of vulnerabilities per year for the more than decade running of my career. The same is true of my collegues.

I have met "cybersecurity teams" who are "tool monkeys" in the way you suggest. Usually these individuals are part of an extended compliance team. They are trying to check compliance boxes by having some level of automated scanning, because their parent organization/team can't/won't staff professionals capable of this or capable of scaling it or focusing attention on where it matters.

I don't want to oust you for where you work or your role within those companies - but really surprised to hear that there's no inhouse pentesting and no in house red teaming. I haven't worked at a company yet (on my fourth) that matches the experience you described.

a lot of your suggested vulnerabilities could be prevented from appearing if company just hired a person who can configure SAST and WAF and etc solutions (solving a class of security problems from showing up on a platform level, not on per each vulnerability instance level).

Why rely on an in-house redteamer who may find one or two vulns per year, or may not.

Eh... maybe.

I think "configuring SAST" goes back to what you had previously said about "tool monkeys". Now there's thousands of potential issues. Many are false positives. Who is going to tune that SAST for accuracy rate? What about vulnerabilities that are framework specific? Will the SAST even allow for this? Who is responsible for triaging all of these alerts? Are they capable of correctly addressing them?

If you just set up SAST and walk away, you end up with more noise than you do signal. And you created the need for security professionals to process the results.

I think the article had better suggestions for scalable security, for example #3 Build standardized patterns and #7 Provide isolation patterns. Better to systematically prevent SQLi with a platform/library than try to detect all variations of SQLi with SAST!

WAF? It's something I would put in front of a product I have confidence in as a defense-in-depth. But would I rely on it as the sole security investment? Absolutely not. The reason is not all vulnerabilities are web-based, and many do not have differentially detectable payloads (missing authz on an API isn't going to get caught by WAF).

Why Red Team? Because the leading way businesses get compromises is with phishing attacks.

A security team needs to ensure application security, network security AND operation security. A SAST or WAF aren't going to do that. Neither with Qualys, etc.

>>Now there's thousands of potential issues. Many are false positives.

Inspecting output/logs of Qualys is no different than inspecting logs of kubernetes (or other SRE platform). and both overlap.

If you have highly skilled SREs - task them with security. If you dont have good SREs, you have to keep IT architects (and call them infosec) who will be able to look at all your IT Zoo across all your on-prem datacenters and cloud accounts and can make a call to do X,Y, and Z to keep company secure.

and who can recover your infra from groun zero in case you got ransomwared

I disagree. I think inspecting the output from Qualys (and other tools, including SAST) are substantially and manifestly different from inspecting Kubernetes logs.

I would worry the argument about "highly skilled SREs" could become a "true Scotsman" argument. If a business has any persons who are skilled enough and plentiful enough to process all of the security output and take action on them, let it be so.

My experience is that in practice, there are not the resources to process all of the output that the tools generate. Do you have experience to the contrary where this has been done at a company scale or is your argument a theoretical one that you believe stands to reason?

If you task your fully staffed SRE division to have some people doing cybersecurity full-time, what's the difference between having that and an actual cyber security team? And if you have that, why wouldn't you want to hire experts in that field instead of cross-training or finding generalists?

hiring security experts is expensive and very few companies are able to afford and retain them.

ask yourself what is cheaper: hire and retain Cloud Operation admins in SRE org, hire and retain Cloud security experts in cybersecurity org -- vs hiring a cloud security guru and task him overseeing with maintaining and security $platform_name ?

very few companies are able to hire and retain SRE-Kubernetes operators and Kubernetes security architects, so it kinda makes sense to merge and hire one good expert

Now you're just haggling over price. Ask yourself, what's more expensive, a good security team, or getting hacked? For LastPass, it's existential.

It just depends on the size of the respective orgs. If engineering is 5 people, a dedicated security person doesn't make sense. At 500 you might be able to get away with one. At 5,000 engineers though, you real do need more than one good security expert.

I'm a full time red team. Depends on your org, but you're not ready for a red team until you get the basics of blue straight.

Most of our blue is threat and anomaly detection. They spend a lot of time in tools, sure but its usually a SIEM like Splunk or similar tools for event correlation and behavior detection like Crowdstrike. Qualys isn't going to find custom written malware using common utility as a C2 channel.

>Qualys isn't going to find custom written malware using common utility as a C2 channel.

yeah, but crowdstike or whatever antivirus you use can. Again, no need for red team, just an operator for Security Tool X/Y/Z

>yeah, but crowdstike or whatever antivirus you use can.

...No, it really can't. Not all of the time. It doesn't understand your business logic behind your architecture, it doesn't know how to separate normal from anomalous behaviour (yes, even if you have XtremeAI or whatever the vendor peddled to you), it cannot correlate alerts and activites in patterns not explicitly programmed and it can't generate alerts with 100% precision. In short, it's not intelligent.

thats not the job of red team, this is blue team work (Detection Engineering or Incident Response or Security Operations whatever they are called) - a SIEM tool monkey basically.

again, no need for a fulltime red teamer

I don't think you know what the point of a red team is, if you did you'd understand how oxymoronic your statement is.

A part time red teamer is just a pen tester. That's not what a red team does, and adequate threat emulation isn't someone who flies in for two weeks does somethings that you can only accomplish in two weeks, then fly out to the next company to do the same things. You actually need to dig deep and research a companies individual weaknesses and that takes time. Threat actors have that kind of time to sit and observe and probe, a part time noisy pen test isn't going to get you more than the basics.

Also a blue team has to be right all the time, the attacker only has to get things right once. That's why you constantly test your assumptions with a red team and give blue someone to "train" with and improve with that isn't an actual threat actor. You don't want the first time you fight to be in the ring, you need to spar.

A lot of pen testing consultancies brand themselves as red teams, but a lot of them are just rebranding pen testing services to the ignorant.

yeah nice word salad, except that it doesnt happen in real world. what you described is probably just one threat intelligence engineer that also does bunch of other stuff like IR

This is literally what I do, but okay.

Threat Intel informs the red team, and we'll model our ops or threat emulation exercises off of that information, we also have a few members that came over from threat intel. Last I checked, Threat Intel engineers don't write malware to simulate malware real threat actors that target a companies industry use.

I think you're speaking from your limited experience at your own company. Again, you have to have a mature blue team before you're ready for a red team. Most orgs aren't ready for that, especially if you think Qualys covers your threat models. That's just vulnerability management and might catch some apps/hosts that missed the patch cycle.

If it's word salad, maybe its because you're out of your depth when and ignorant of how these programs are supposed to work.

WTF are you talking about, we regularly evade these tools?

> "cyber" people are basically TOOL MONKEYS

That was also my short experience in the area, and it was boring AF, almost soulcrushing. Maybe I just got unlucky, but it feels that a lot of security engineering positions are like that.

Would anyone happen to know of what search terms should one use to find open positions that are focused on security tool building rather than deploying a purchased solution?

Security jobs are tiered:

Tier0: Pure security research shops, like Google Project Zero and alike. YOu can only get this job after doing PhD in Computer Security at top-tier university lab, and have publications on stuff like linux kernel fuzzing, have 10+ linux CVEs under your belt, and listed as contributor to a well-known open-source security tool.

Tier1: top security vendors: crowdstrike zs qualys pan cisco and etc. R&D (or Business Unit) departments responsible for developing security product features and maintaining security content.

Tier2: FAANG and adjacent cloud-native companies, job title "security software engineer" or "security engineer" or "$platform_name engineer" or "cloudsec/appsec engineer"

Tier3: pre-IPO unicorns which are hiring for Product Security and AppSecurity positions

... Tier10+: all other public companies, regular S&P500 infosec department.

the general rule is the higher the salary the more demanding the job will be and the tougher is competition to get that job

> I am gonna surprise you, but nobody from cybersecurity is finding vulnerabilities either.

My recent interview as a security engineer had distinct sections for software review and for architecture analysis. My last job had me running Burp Suite once a month (because it was fun and I didn't mind doing it), but that was roughly 30 minutes every 4 weeks.

Sounds like you've worked with some quite junior security teams.

interview vs real work is different. FAANG engineers are not flipping a binary tree either, yet this is a common question they ask.

as for Burp Suite - why would a company keep engineer full-time only for him to run burp suite for 30 minutes once a month? Sounds like poor investment on a company's side.

> interview vs real work is different.

...or, and hear me out, maybe some companies are more proactive about their security. Craziness, I know, but maybe!

> why would a company keep engineer full-time only for him to run burp suite for 30 minutes once a month?

Uh, I was doing other things the other 159.5 hours a month.

so you agree that it is cheaper and better to ask SRE team to run BurpSuite for 30 minutes once a week, rather than hiring FTE to do that?

I honestly have no idea what you're trying to say.

>I am gonna surprise you, but nobody from cybersecurity is finding vulnerabilities either.

Well, no. I work in "cybersecurity" and my day job is literally finding vulnerabilities.

Reply on news.ycombinator.com