I can tell you they’re quite unhappy about it. Have a friend working there who frustratedly says it wasn’t their fault every-time it comes up. Which is quite often and at every social occasion since.
They didn’t have much of a choice - it is very hard to get adequate performance with real-time filesystem filtering without doing it in kernel mode. Not aware of any other mainstream OS which succeeds at that.
And they kind of had to provide this feature, since they’ve supported it since forever (antivirus vendors were already doing it back in the days of MS-DOS and Windows 3.x/9x/Me), and there is a lot of market demand for it. It is easy for Linux to say “no” when it never has had support for it (in official kernels)
But, as the blog post points out, it sounds like CrowdStrike is doing a lot of stuff in kernel mode that could be done in user mode instead - whether due to laziness or lack of investment or lack of sophistication of their product architects
> they at least _allowed_ it to happen every time
Microsoft, in allowing third party code to be loaded into their kernel, is no different from other major OS kernels, such as Linux or Apple XNU.
Apple is (increasingly) the most restrictive about this, and a lot of people criticise them for it.
Even Linux imposes some restrictions-which kernel symbols to export (at all or as GPL-only)—although of course being open source, you can circumvent all restrictions by changing the code and recompiling
Whereas this Windows outage rather obviously was.
eBPF being able to crash the kernel is usually sign of a kernel bug. And it sounds like in this case it was even a bug specific to Red Hat kernels, introduced by a Red Hat patch.
That said, even if they are triggering a Red Hat kernel bug, CrowdStrike should be testing their software adequately enough to pick up that issue before customers do – and it sounds like they haven't been
Whereas, from what I hear, L4 and its derivatives have solved this problem in a way that Mach/Minix/etc could not. Yet still, it makes me wonder, if L4 has really solved it, why aren’t we all running L4? L4 has had some success in embedded applications (such as mobile basebands, Apple Secure Enclave); but as a general purpose operating system has never really taken off.
All of these parties could do better (stupid tree!). But the real problem is the valet.
We can say that it is obvious that the electronics-heavy cars of today should anticipate rogue valets and build in protections. But we shouldn't let rogue valets off the hook for damages.
As a consumer, you could choose to only purchase cars that have "valet mode". So should we blame consumers who don't? If so, we should blame the airlines, hospitals, etc.--not Microsoft.
How about we prosecute valets unless they refuse to park cars that don't have "valet mode"?
No, the operating system is supposed to provide secure access to hardware and isolate independent subsystems so they can't interfere with each other. That's its whole purpose for existing. The fact that people feel they need to deploy CS is a Microsoft failure. Windows is just not a secure OS.
They don't need to deploy shit. Only reason it's deployed because it's a whole racket.
I get that you hate Microsoft, but not everything is their fault and it’s disingenuous to pretend otherwise.
> ing. The fact that people feel they need to deploy CS is a Microsoft failure.
CS is also available and widely deployed on Mac and Linux. Is that a failure of Apple and all the distros? It literally took down Debian and Red Hat systems earlier this year, is that also not CS’s fault?
I don't.
> CS is also available and widely deployed on Mac and Linux. Is that a failure of Apple and all the distros
Yes. All widely deployed commodity operating systems have terrible security designs. None of them have access control systems that enable the principle least privilege, let alone encourage or prioritize it, and none of them are written in robust languages that make verification of safety or security properties possible. Microsoft has made some headway on partial verification, but it's a far cry from what's needed.
What, exactly, is your solution then? To never use a computer again? Because that's certainly what it sounds like.
Not the best analogy. The organization who deploys said software is responsible for the uptime of their systems. They didn't have to use CrowdStrike and if they do they should have a plan in the event of failure.
Microsoft took no steps to force-eject them from their ecosystem, despite their long history of issues.
If a Caesars Entertainment property in Macao has enough incidents, should GM update the firmware on their automobiles to force-eject valets at Caesars Entertainment properties in Las Vegas?
Now imagine that GM actually operates valet services in Macao and Las Vegas. Should they be allowed to force-eject valets from competing services?
I am not a Microsoft apologist. I think they should do better. I think Linux and FreeBSD should do better. I personally avoid Microsoft products. But I place more blame on people who use MS products than I do on MS. After all, I never intend to hand my beat up old Corolla over to a valet so why should I have to pay for a "valet mode" feature that Toyota is forced to build into all their cars? Isn't it reasonable that motorcycles, 18-passenger vans, and scooters don't need "valet mode"?
In my book, the auto manufacturer is lower on the list of culprits than the valet, "the establishment that keeps a valet with an abominable record on staff", and the vehicle owner. But some place like Car and Driver could definitely prioritize encouraging GM or Toyota to develop valet modes over berating owners; so I don't mind a place like HN shooting a few arrows at MS. Unless the general public follows their lead and lets bad guys off the hook by shifting too much focus to somebody lower on the list.
Not OP, but I think the analogy here is the hotel "fore-ejecting" (firing) the valet with a history of doing joy rides. That seems very reasonable.
Also the restaurant has their own entrance without security and questionable people are entering regularly, and they are sneaking into the hotel rooms and stealing some items, breaking the elevator.
At the same time, the hotel is in a litigation process with the restaurants association, because in the past they did not allow any restaurant on their premises. The guests, naturally, do not care about this, since their valuables have been stolen, and they have food poisoning. The reputation of the hotel is tarnished.
I don't think this works since Microsoft isn't the hotel. The hotel in your example chooses which restaurants are inside, but Microsoft doesn't. In this example, Microsoft is the builder who built the hotel building for a 3rd party. That 3rd party decides which restaurants it wants to partner with, as well as any other rules about what goes on in the building.
If the builder came around and made changes to ban the 3rd party's restaurant partner, that would cause a ton of issues and maybe get the builder sued.
Microsoft can't decide what can and can't run on their platform - the most they can do is offer certification which can't catch everything, as we just saw with Crowdstrike since they decided to take a shortcut with how they ship updates. Microsoft also had to allow for equal API access so they don't get sued by the EU.
Again the reasoning of allowing equal API access to avoid getting sued is a false dichotomy: Microsoft could choose to make an OS that would not need such mechanisms to be simply usable.
They could also remove their own crowdstrike-alike offering, so that it would not be considered anti-competitive. They could also choose not to operate in EU. Of course, that would lower their profits, which is the real motive here.
Once you sum it up the reasoning goes: hospitals/flights can stop working because a company cannot lower its profits, and said company is not to blame at all. It is clearly false, the rest is sophism, and back-bending arguments IMO.
So the hotel can have an infinite number of restaurants which can move and move out as they please with not input from the hotel itself?
Please note, that in my analogy the hotel has input in which restaurant is allowed (opposite of your scenario). There are also not infinite Crowdstrike-like offerings, only a few. Same thing applies to the hotel, yes, only limited by the surface of the building and cultural norms.
I any case, the analogy cannot please everyone, and I can see how there are some errors with it in some aspects. In others, I consider it accurate. Using an analogy is an invitation to nitpick on it, so it is my fault really, but I could not resist.
There are other points in the analogy that I feel reflect very well how ridiculous it is to claim Microsoft has no responsibility whatsoever. IMO they do have at least partial responsibility. One cannot simply excuse them "because EU".
In reality this doesn’t seem to be the case at all.
Or in Microsoft's case, via regulatory, social, or software, prevent Crowdstrike from causing harm to their customers.
I'm aware it's a sticky regulatory situation, but CS has a history of these failings and the potential damage could be severe. Despite this, no effort (that I am aware of) was made by Microsoft to inform customers that Crowdstrike introduced potential risks, nor to inform regulators, nor to remove the APIs CS depends on.
I don't believe Microsoft is solely responsible, but I do believe that throwing all of the blame for the very real harm that was caused onto CS alone is missing a piece of the puzzle.
Last aside, every large corp has team(s) focused on risk. There's approximately zero chance they didn't discuss CS at some point. The only way this would not have happened is negligence.
wonderful world.
I’m pretty sure anti trust law doesn’t allow Microsoft to go anywhere near that kind of action, even if they wanted to be more Apple like.
Isn't the point of userland that you can (try to) do anything from there?
It seems like MacOS and Linux provide substantially safer alternatives that are still performant?
> As for restricting kernelland, EU competition regulators would not be happy
I keep seeing people say this. Is there a basis for that assertion, or is that mere speculation? Again, hasn't MacOS already deprecated kexts?
Via Google: https://www.techtarget.com/searchsecurity/news/450420491/Mic...
(Also via myself, as I was at MS when we wanted to make this change and the EU said no.)
I think that would have been a proactive approach. That said, I'm not entirely convinced that the EU was right to place the restriction in the first place.
I think a proactive approach might have been for Microsoft to provide safer interfaces with the kernel, and then use those in its own security offerings.
That said, it does sound like EU competition regulation was a contributing factor here, and I think the EU is wrong on this one and that an OS vendor should not be required to provide unrestricted kernel access to allow security software vendors to compete.
Mostly unrelated, it seems somewhat interesting that this was Kaspersky insisting on kernel access... The US government seems convinced they are compromised.
[0]: https://www.ft.com/content/60dde560-194a-40d1-8c98-1d96d6d01...
Kind of weird that anyone is blaming Microsoft for any part of this, imo
More like:
If you install a security product that then prevents your car from starting; are they entirely blameless for letting you install it?
If you pull the hood up, tear off the “voids warranty” seal, ignore the “don’t open this” labels, crack the seals open and shove something into the engine… sure.
…but if you just slap a widget with the “vendor approved” sticker on your dash and it bricks your car; that’s a bit sucky right?
I do feel Microsoft is not entirely blameless in this.
It should be easier to recover from this kind of thing.
They should have been paying attention and made a fuss that one of the biggest security vendors has been doing this literally since they started.
I would bet money that until two weeks ago Microsoft was high-5ing them for best security practices.
It’s not “their fault” but they can’t just go “wasn’t us!”.
It was them.
It wasn’t macOS. It wasn’t *nix.
Suck it up. They should’ve done better.
Before Microsoft comes into the picture the issues is crowdstrike pushing updates without proper testing, selling a product on which customers cannot control the update schedule, and customers for being so naives and not checking what the product they install on critical stuff do.
It got there because a user or administrator approved and installed it. It didn't just appear there, Microsoft didn't install it there. The user ran it.
Basically IOS which is so locked you can't even run apps not expressively approved by Apple.
Pick one. If I build a bike and you remove the breaks to save weight don't get mad at me when you crash.
or at least provided some level of protection against crashes in third party kernel code.
drivers and kernel binaries are typically installed and maintained by user space programs that run with some sort of elevated privileges.
"kernel space" is just a runtime context, what gets loaded into there typically comes ordinary (protected) files on the disk.
The OS loads file A into the kernel. It crashes. It reboots. It decides not to load file A this time.
Wow, it's a rollback of kernel-space code.
Unless your argument is that you can't guarantee a rollback of every possible kernel driver, because it might have installed a rootkit while it had full control? Okay, cool, but this isn't a malware removal idea. It's an idea for normal drivers.