A Basic Timeline of the Exchange Mass-Hack | Hacker News Reader

A Basic Timeline of the Exchange Mass-Hack

krebsonsecurity.com

96 pointsparsecs24 comments

Microsoft was aware of the vulns for 2 months before issuing a patch.

Some of the vulns existed in the Exchange codebase for 10 years.

Microsoft faces perverse incentives. When their customers get compromised, Microsoft benefits from accelerated upgrades and cloud subscriptions.

Yet their customers blame foreign threat actors and not Microsoft, so Microsoft suffers no reputational damage.

With these incentives, why would any rational corporation spend resources hardening their software or responding rapidly to new disclosures?

I guess I have 2 questions for you.

First: what if the fix really took 2 months?

Second: I'm not well versed in this kind of software; is this on-prem stuff that clients have to manage (update) themselves, or is this SaaS cloud stuff that MS can immediately update without clients in the loop?

This is Exchange Server, entirely on-premise that must be updated by clients.

If the fix took two months I would say that is not confidence-inspiring. Consider the enormous financial reserves at their disposal.

In Microsoft’s defense, most organizations have security vulnerabilities in their code that all may or may not be aware of.

Just because Microsoft has software powering very important things globally doesn’t exclude them.

Some of the “foreign threat actors” are also major Microsoft customers.

If somone commented "Stop picking on Microsoft", how would you respond, if you chose to respond.

With these incentives, why would any rational government spend taxpayer money on this company's software.

Would you switch your vote to the opposite party because your party used microsoft software or because the other party promised they wouldn't? If not then what incentive does a rational government have to avoid using this software?

If a party candidate pledged to address quality control problems with software or "tech" companies such as Microsoft, then yes, that could potentially influence my vote.

The canned response for every problem with Microsoft today seems to be "Well, nothing more can be done, except what Microsoft (and other "experts") tells us to do." The company that creates the problems is deemed to be the only one who could ever attempt to solve them. This is pure nonsense. With few exceptions, this goes virtually unquestioned,

The "experts" have perverse incentives. Business is good just the way it is. If problems are actuallly solved, business would likely decline.

Publicly available software suffers from the fact that millions and millions of people can poke and prod at the software in ways that the manufacture can't due to resource and time constraints driven by the need to generate revenue.

Microsoft runs their software through multiple code scanners looking for weakneses. Developers do unit testing, and then there is acceptance testing. Microsoft conducts internal penetration tests on their software. Microsoft hires 3rd parties to conduct penetration tests on their software. Large corporations conduct internal penetration tests on Exchange, and hire 3rd party companies to conduct penetration tests on Exchange, just to be sure. Governments conduct penetration tests on Exchange.

A lot of people have been poking at Exchange for years, and years, and years, and this bug was just discovered, and it's been present in the code base for at least 7 years, I'd say that's pretty damn good, it seems like a hardened product to me, not a slipshod product as you suggest.

You are so funny. First you complain that Microsoft has no incentive to deliver quality software, then you complain that they can't delivery quality software quickly.

I guess they would have been better off not bother to conduct all the testing necessary to ensure that they didn't fix one problem and create two more.

It's kind of funny how Microsoft akways gets away with it.

Everyone asks why it takes 2 months to develop, test and deploy a patch. Nobody ever asks why it took over 10 years for criminals to find and exploit this bug. What were they doing all this time? Why did it take them so long? Do criminals have perverse incentives to be law-abiding or are they just plain lazy?

Something interesting I learned when looking into all of this is that if you have a large environment (2000+ mailboxes) and transition to Exchange Online, Microsoft still (since 2010) has no idea on how to fully decommission your Exchange Server environment, since you need at least 1 to facilitate on-prem AD connectivity (which isn’t true if you didn’t have a hybrid environment). So even if you transitioned to the cloud, you may not have been safe.

https://docs.microsoft.com/en-us/exchange/decommission-on-pr...

The whole AD security model is outdated.

When something is wrong with any of a domain's services, your organization is breached.

There is variation in details like in how to exploit vulnerabilities or how to move laterally...

But AD is a SPOF, cloud or not.

This is a major unresolved issue. Not only is Exchange, the service, a vector for attack, but AD-related permissions granted to the Exchange computer account and admins can leave your entire directory vulnerable.

This is true, although if I look at my old company - they've migrated to Exchange Online and run onprem Exchange servers to support the hybrid environment. But they don't need to publish those servers to the Internet.

Many of these orgs should have learned this after [CVE-2020-0688](https://www.thezdi.com/blog/2020/2/24/cve-2020-0688-remote-c...)

If it's a hybrid environment, it doesn't need to be exposed.

That doesn't solve MS problem of coming up with a better solution though

So like, was the vuln more or less made widely known at some point? This feels like the scope grew so large because many groups obtained the 0day before Microsoft expected it to go wide, which is not what folks seem to have expected.

It'd be interesting to see more info in the timeline about when that might have happened. Just feels like this info is entirely based on what the research community was seeing, not based on any info from the adversary side of this event (not that collecting that kind of data is easy, so fair enough).

Not my field but from observation: Other security researchers watch when a CVE/vuln is announced and many have enough experience/knowledge to then know how to reproduce that vulnerability.

I suspect that the researchers involved all saw how bad it was and didn't feel that it was safe to let Microsoft wait, as (although I don't think it's particularly common) they may drag their feet on a patch. Leaking this vulnerable forces MS's hand sooner. So it's not really necessarily totally off from what I've seen in the past. Just not very common.

This tends to happen when researchers discover the zero day is ALREADY pervasive in the wild.

>many groups obtained the 0day before Microsoft expected it to go wide,

There were multiple groups conducting attacks? Is there evidence of that?

I don't understand how anyone thinks Exchange can still be used ... just setting it up without obviously choosing any obviously insecure settings somewhere in the stack while also trying to support the actual needs of a diverse set of users (without even considering the presence of unpatched vulnerabilities and required patching spedds) probably exceeds the IT capabilities of 99% of corporations.

this article is a tire fire and even links to the exploitation of a different exchange bug

i don't see an issue here

microsoft patched a bug within a 90-day disclosure timeline and even released patched before the agreed date when it learned they were exploited

why is krebs making a big deal out of it

Krebsonsecurity.com needs to update it's website to make it mobile responsive.

HN likes to nitpick on everything but whenever someone brings this particular point up it's nothing but crickets. It's 2021 and for someone who has the audacity to selfhost ads, you would think he is also capable of adding a couple of CSS rules.

“Please don't complain about website formatting, back-button breakage, and similar annoyances. They're too common to be interesting. Exception: when the author is present. Then friendly feedback might be helpful.”

https://news.ycombinator.com/newsguidelines.html

Reply on news.ycombinator.com