Hacker News Reader

Comment by watermelon0 | Hacker News Reader

Windows is not the issue here. If all of the businesses used Linux, a similar software product, deployed as widely as Crowdstrike, with auto-update, could result in the same issue.

Same goes for the OS; if let's say majority of businesses used RHEL with auto updates, RedHat could in theory push an update, that would result bring down all machines.

View on HN

Agree. The monoculture simply accelerates the infection because there are no sizable natural barriers to stop it.

Windows and even Intel must take some blame, because in this day and age of vPro on the board and rollbacks built into the OS it's incredible that there is no "last known good" procedure to boot into the most recent successfully booted environment (didnt NT have this 30 years ago?), or remotely recover the system. I pity the IT staff that are going to have to talk Bob in Accounting through bitlocker and some sys file, times 1000s.

IT get some blame, because this notion that an update from a third party can reach past the logical gatekeeping function that IT provides, directly into their estate, and change things, is unconscionable. Why dont the PCs update from a local mirror that IT has that has been through canary testing? Do we trust vendors that much now?

Poor Crowdstrike. This might be the end for them.

https://access.redhat.com/solutions/7068083

Just last month there were issues between RHEL's kernel update and Crowdstrike.

I'd like to read more about it, but that link is... paywalled I think? It's not even clear.

I keep a RedHat developer account active for their documentation. I didn't notice.

I did find these related forum/reddit threads:

https://forums.rockylinux.org/t/crowdstrike-freezing-rockyli...

https://www.reddit.com/r/crowdstrike/comments/1cluxzz/crowds...

Good to see Rocky keeping their promise of bug for bug compatibility.

I would posit that RedHat have a slightly longer and more proven track record than Crowdstrike, and more transparent process with how they release updates.

No entity is infallible but letting one closed source opaque corporation have the keys to break everything isn’t resilient.

His example actually had 2 parts. One RH bricking the OS, the other one with a commercial vendor creating software with separate auto update.

You've only addressed the RH OS angle.

Yes but the problem here was bricking the OS

>> Windows is not the issue here.

Yes it is. Windows was created for the "Personal Computer" with zero thought initially put in to security. It has been fighting that heritage for 30 years. The reason Crowdstrike exists at all is due to shortcomings (real or perceived) in Windows security.

Unix (and hence Linux and MacOS) was designed as a multi-user system from the start, so access controls and permissions were there from the start. It may have been a flawed security model and has been updated over time, but at least it started some notion of security. These ideas had already expanded to networks before Microsoft ever heard the word Netscape.

> was designed as a multi-user system from the start, so access controls and permissions were there from the start.

Right and Windows NT wasn't? Obviously it supported all of those things from the very beginning (possibly even in a superior way to Unix in some cases considering it's a significantly more modern OS)...

The fact that MS developed another OS called Windows (3.1 -> 95 -> 98) prior to that which was to some extent binary compatible with NT seems somewhat tangential. Otherwise the same arguments would surely apply to MacOS as well?

> These ideas had already expanded to networks before Microsoft ever heard the word Netscape.

Does not seem like a good thing on its own to me. Just solidifies the fact the it's an inherently less modern OS than Windows(NT) (which still might have various design flaws obviously, that might be worth discussing, it just has nothing to do whatsoever with what you're claiming here...)

We have Crowdstrike on our Linux fleet. It is not merely a malware scanner but is capable of identifying and stopping zero-day attacks that attempt local privilege escalation. It can, for example, detect and block attempts to exploit CVE-2024-3094 - the xz backdoor.

Perhaps we need to move to an even more restrictive design like Fuschia, or standardize on an open source eBPF based utility that's built, tested, and shipped with a distribution's specific kernel, but Windows is not the issue here.

Security is a complex and deeply evolved field. Many modern required security practices are quite recent from a historical perspective because we simply didn't know we would need them.

A safe security first OS from 20 years ago would most likely be horribly insecure now.

That's assuming in this alternate universe we'd also be using kernel antivirus software to counter malware. It's far from obvious.

yes, staggered software update is the way to go. there was reply in this thread why Crowdstrike did not do it -- don't want extra cost of Engineering for that

having 1/3 of Airlines computers Windows, RHEL, Ubuntu .. all unlikely to hit same problems at same time.

But you're more likely to encounter problems. That's likely a good thing as it improves your DR documentation and processes but could be a harder sell to the suits.

The update here is relevant to catch 0day exploit.

Without the update, your system is "naked" for the duration.

But then it'd be putting all eggs in the Linux pc basket, wouldn't it? I think they point was that more heterogeneity would make this not be a problem. If all your potatoes are the same potato it only takes one bad blight epidemic to kill off all farmed potatoes in a country. If there's more heterogeneity things like that doesn't happen.

The difference being that RHEL has a QA process which crowd strike apparently does not. The quality practices for open source involved companies is apparently much higher than for large closed source "security" firms.

I guess getting whined at because obscure things break in beta or rc releases has a good effect for the people using LTS.

Maybe this is pie-in-the-sky thinking, but if all the businesses used some sort of desktop variant of Android, the Crowdstrike app (to the extent that such a thing would even be necessary in the first place) would be sandboxed and wouldn't have the necessary permissions to bring down the whole operating system.

More secure OSes would consider an application being able to take down the entire OS as a security issue and would make that impossible.

When notepad hits an unhandled exception and the OS decides it's in an unpredictable state, the OS shuts down notepad's process. When there's an unhandled exception in kernel mode, the OS shuts down the entire computer. That's a BSOD in Windows or a kernel panic in Linux. The problem isn't that CrowdStrike is a normal user mode application that is taking down Windows because Windows just lets that happen, it's that CrowdStrike has faulty code that runs in kernel mode. This isn't unique to Windows or Linux.

The main reason they need to run in kernel mode is you can't do behavior monitoring hooks in user mode without making your security tool open to detection and evasion. For example, if your security tool wants to detect whenever a process calls ShellExecute, you can inject a DLL into the process that hooks the ShellExecute API, but malware can just check for that in its own process and either work around it or refuse to run. That means the hook needs to be in kernel mode, or the OS needs to provide instrumentation that allows third party code to monitor calls like that without running in kernel mode.

IMO, Windows (and probably any OS you're likely to encounter in the wild) could do better providing that kind of instrumentation. Windows and Office have made progress in the last several years with things like enabling monitoring of PowerShell and VBA script block execution, but it's not enough that solutions like CrowdStrike can do their thing without going low level.

Beyond that, there's also going to be a huge latency between when a security researcher finds a new technique for creating processes, doing persistence, or whatever and when the engineering team for an OS can update their instrumentation to support detecting it, so there's always going to be some need for a presence in kernel mode if you want up to date protection.

I mean, to me that's just a convincing argument against using kernel-mode spywa-, err, endpoint protection, with OTA updates that give you no way to stage or test them yourself cannot be secure.

How are those arguments against kernel level detection from a security perspective? His arguments show that without kernel level, you either can't catch all bad actors as they can evade detection, or that the latency is too big that an attacker basically has free reign for some time after detection.

Easy: plenty people in this forum aren't entrenched in the security field.

That's why there are so many misinformed assumptions

Reply on news.ycombinator.com