Microsoft says EU to blame for the worst IT outage
euronews.com
euronews.com
I suggest Microsoft follows Britain's example and leaves. The main difference is that we Europeans actually miss the Brits, whereas nobody would miss Microsoft and its shoddy products and business practices.
On a more serious note, I fully understand that the Digital Markets Act is causing Microsoft headaches. But I think this headache is well deserved. Big Tech has been building moats where they should have built bridges, and now our computing landscape resembles medieval Germany where everything was at the mercy of a few feudal lords. It is time to drive out those lords and reshape software in a way that empowers, not enslaves.
Or, an alternative interpretation: Microsoft had 15 years to fix any issues.
However, doing so is no easy feat. The first version of eBPF was released over 10 years ago.
Apple did not have to sign a settlement with the EU, which Microsoft did in 2009.
The terms of the settlement state:
"Microsoft shall make available to interested undertakings Interoperability Information that enables non-Microsoft server Software Products to interoperate with Windows Server Operating System on an equal footing with other Microsoft Server Software Products." [0]
"Microsoft shall ensure on an ongoing basis and in a Timely Manner that the APIs in the Windows Client PC Operating System and the Windows Server Operating System that are called on by Microsoft Security Software Products are documented and available for use by third-party security software products that run on the Windows Client PC Operating System and/or the Windows Server Operating System. These APIs will be documented on the Microsoft Developer Network, unless open publication would create security risks. In such circumstances, Microsoft will provide third-party security vendors with access to such APIs pursuant to a royalty-free license and on fair, reasonable and non-discriminatory terms." [0]
This means that by offering Microsoft Defender for Endpoint, Microsoft needs to give similar access to the underlying kernel to competing vendors like CRWD and S1.
> At the very least, it means there are safer ways to load third-party code in the kernel without allowing them to crash your entire system by mistake
An eBPF or a similar technology wouldn't necessarily enhance stability when probing kernel-land from user-land, as any interaction with a kernel can cause kernel panics.
Plenty of endpoint vendors have had issues with PTrace (MacOS), kprobes (Linux), eBPF (Linux, K8s), etc.
The reality is that $))&#( happens, and a lot of comments on HN about this bug are really dumb.
[0] - https://news.microsoft.com/download/archived/presskits/eu-ms...
Nothing is perfect, latest generation of Intel CPUs are unstable, bugs are part of all kernels, and so on and so forth. But eBPF does allow for less crashes than a kernel module and these crashes are usually sandboxing error rather than an error made by the third-party provider.
As a matter of fact, the eBPF Linux version of Crowdstrike did create kernel panics on some distros.
> This means that by offering Microsoft Defender for Endpoint, Microsoft needs to give similar access to the underlying kernel to competing vendors like CRWD and S1.
Absolutely, and that's a good thing. This doesn't seem to prevent them from moving Defender to another set of security APIs or from creating another set of security APIs for anyone to hook on.
Apple is also irrelevant in the corporate world, where the Crowdstrike shitshow happened, as they produce mostly consumer devices.
MacOS endpoints are a prominent market as well. There's a reason CRWD, S1, Tanium, and others offer MacOS protection as well.
And anyhow, it seems you are ignoring my point - the settlement with the EU tied MS's hands.
Everything is an API. This means they cannot restrict access to the underlying kernel.
> They had the choice to give Windows APIs that allow to implement such software in a safer way
An indirect method would still inevitably have a potential issue. If a third-party vendor made a mistake, it's the third-party vendor's fault - in this case CRWD.
Sure, it's safer if only a single vendor is allowed to do this but it's also unfair.
> that are called on by Microsoft Security Software Products
Nothing says that Microsofts Security Software needs to be implemented in a way that runs in the kernel, which is inherently more vulnerable. Other platforms have APIs for security software to hook into kernel actions without directly running in kernel space (Windows does too, but to my understanding more limited to logging, not intercepting activity, and hence limited in usefulness).
Hence the claim that it was impossible for Microsoft to prevent this is false. That doesn't mean they take all the blame for someone elses mistake, but it still means that they made a choice leading to it: Deciding that security software, whoever made it, running in kernel modules was the way to do it.
No it doesn't. It just means they should restrict their own products from doing so, too.
They are not saying the EU is the root cause of the failure, just that they cannot close the hole currently due to the EU.
What they leave out is that they could choose to integrate Defender into the OS for free, thereby removing it as a product to compete against. They could also move Defender to not require kernel hooks either. Neither are options they want to consider currently.
No comment about being able to move Defender to not require kernel hooks (I don't know).
Two reasons:
1 - Few people understand anything about how their computer (/car/stove/phone/medicine/...) works -- they spend their time on other things.
Without any model of how their device works its easy to misassign responsibility (see how many people think that Safari is Google or vice versa). So it's in MS's interest to try to get the message out. Of course people do this when they are at fault as well.
2 - EU is in a wave of beating up* on certain large companies. This can also be an opportunistic way to push back.
* I am not implying whether I think the EU is correct or not.
If MS had blocked these type of things people would be in here complaining about antitrust and MS is evil.
While the hate is valid in many cases, I've observed that the cribbing about it has also been unwarranted or unjustified a lot of the time (also no other corp is held to the same standard) - and this is a prime example.
MS cannot legally restrict third party kernel. Apple can, bc they didn't get struck down like MS did.
MS has an option to not bundle Defender with their OS, which would let them lock the kernel to avoid the anti-trust restrictions, but that would be an insane decision to make.
Damned if they do, Damned if they don't indeed
Our EU friends really enjoy having all the regs on everything... but then demand to be treated as-if the regs don't exist. It's amazing to see...
I work in big tech, and unfortunately we frequently need to have conversations about the smallest features because we have evidence about us giving users an inch and they taking a mile.
Windows kernel signing does not work like Apple's Big Brother approach, it uses a set of certificate authorities.
Microsoft does have a program for verifying drivers: WHQL, which you may recognise from the slower driver that Windows Update installs for your GPU before you download the faster one that didn't pass Microsoft's verification from the manufacturer's website. CrowdStrike doesn't seem to be WHQL-certified.
People interested in running CrowdStrike would be forced to use the EU version anyway, and people uninterested wouldn't be affected by the Crowdstrike bug whether they used the EU version or not, so I don't see exactly how having two versions of Windows would help here.
The rest of the world would have come to a different solution not based on kernel-level access.
I.e Taking Microsoft's argument to it's literal conclusion.
More discussion: https://news.ycombinator.com/item?id=41029590
Euronews Next has contacted Microsoft for comment
High quality stuff.
Sure, if you access the kernel you can break it. So we should make easier to do the right thing.
EU complained that Microsoft was the only one that could access the kernel because a lot of Microsoft applications needed kernel access.
(Edit for the downvoters - nothing I said above was an opinion. https://news.ycombinator.com/item?id=41049312)
Ha, no. It was caught because an engineer noticed his SSH was taking slightly longer than normal, a few hundred milliseconds of difference. There was nothing in the code to suggest an anomaly; so he began fully reverse-engineering the binary as though it was proprietary software. The open-source communities meanwhile had approved and were even distributing that code on testing branches. And to top it all off, that engineer worked for Microsoft; so it's pretty ironic to complain about Microsoft's behavior with Windows, considering they just saved Linux from catastrophe.
I mean open source idea is that even MS engineer can find their problems.
And then MS engineer found their problems.
So yeah, does seem like a win for open source ideas which is not even something I particular care about...
Cloudstrike was not in Microsoft's code; so yell at Cloudstrike.
> And then MS engineer found their problems.
The Microsoft engineer found the malicious code from the binary first - the same way he would have investigated proprietary software. The fact that it was open source didn't help discover the vulnerability in any way. The open source nature only helped with explaining how it got in there afterwards.
> So yeah, does seem like a win for open source ideas which is not even something I particular care about...
According to many security researchers, the `xz-utils` thing was deeply underrated and shows how, let's just say, not necessarily more secure open source software is. The fact that every Linux computer was nearly backdoored, globally, and it was found by accident, by a Microsoft engineer nonetheless, without any use of the code to find the bug, after that code was approved by both Debian and Red Hat, looks terrible to the open source community.
I take that back. It doesn't just look bad - it is bad. The ideology and security assumptions are in question, bad. If Microsoft's engineer had not found that bug, Linux and open-source as a whole could have sustained a mortal wound and a crushing blow to the theory of open-source being more secure.
Yes of course it was; it ran in kernel space. Microsoft let it in, ao it was in there.
> so yell at Cloudstrike
Sure. And Microsoft.
Also Linux fanboys will usually blame the system admin for not configuring things properly if things break: "it's not the operating system, it's <something stolen from OpenBSD>".
End of the day Linux is only popular because of the inertia UNIX had on mini-computers/servers. For standard end users GNU Linux is lightyears behind Windows and macOS in terms of usability and stability.