CrowdStrike unhappy with Delta litigation threat, says airline refused free help | Hacker News Reader

CrowdStrike unhappy with Delta litigation threat, says airline refused free help

theregister.com

12 pointsrntn18 comments

I think it is possible that someone will make a convincing argument about how CrowdStrike was negligent in testing and rolling out updates. I have no idea whether this will result in significant court wins, but this case does seem a bit different than many other previous large-scale IT outages.

Delta also considering to sue Microsoft is kinda ridiculous though, and does not give me a good impression. I can see blaming Microsoft for not providing better APIs for this kind of security software to be run in a safer manner, but that is more abstract and long term criticism. It is not something you can use to sue directly for damages.

We all know that mistakes happen, even big ones. But the difference here is that Crowdstrike is running in the most privileged position possible in the OS, is very widely deployed in somewhat critical systems and receives frequent updates on very short notice. There has to be a much higher expectation and burden in cases like this to make this process as safe as possible, ensure a high level of testing and take all possible precautions when rolling out updates.

Sure, Delta is responsible for its IT decisions... But the IT decision it made is putting CrowdStrike on the critical path for most of its public-facing hardware, which is what CrowdStrike's sales and marketing says you should do.

This argument coming from them is not exactly a glorious self-opinion. Were I considering compliance software purchases, I would take this as a signal that CrowdStrike is saying the product can't be trusted.

Crowdstrike is going to talk a big game in pre-trial because they know that once the jury cuts through the PR spin, Crowdstrike is still responsible for screwing up the lives of hundreds of thousands of people by their negligence.

Delta's tech team will be forgiven for not instantly wanting support from the group responsible for the largest IT blunder in history.

Hundreds of thousands is a pretty conservative estimate.

For sure. I just meant with respect to the Delta piece.

Crowdstrike certainly comes out looking terrible. The fact that Delta might have also made mistakes is weak sauce.

But the response from CrowdStrike is pretty solid. Other airlines recovered much more quickly than Delta. Why was that?

Delta being shit at IT doesn't excuse Crowdstrike for their obscene level of negligence. Delta had an eggshell skull but Crowdstrike is the one who broke it by pushing an untested update to the world.

The fact someone releases some software doesn't make someone inherently responsible for any and all business loss related to an event caused by it. I suspect CrowdStrike's EULA has some significant disclaimer of liability, for whatever worth that ends up in court.

Depending on how hard they push this in the Courts, one possible consequence might end up being the legal weakening of all such EULAs.

I wonder if a judge is going to look for someone to blame when a software glitch grounds more than half the planes in the US and conclude that "the vendor said no take-backsies" just isn't good enough?

If they do, could have interesting consequences for the various software licenses with indemnification clauses.

I mean, I assume CrowdStrike will end up settling to get this out of the news, and I assume that's all Delta is looking for, the handout to shut up.

But let's look at how ridiculous this could go: Imagine Delta's house of cards IT stack was so fragile that this incident ended the company. Not another Delta flight ever took off, the company went bankrupt, etc. Would CrowdStrike be responsible for the full business value of Delta? Of course not, that would be ridiculous. The fact that Delta says they lost a half a billion dollars does not automatically mean CrowdStrike is responsible for a half a billion dollar loss. CrowdStrike has likely some responsibility, and Delta has the rest for not having factored this possibility into their resiliency strategy.

That's undisputed. The lawsuit will be over how much the damages are and the balance of liability.

Their response may well have been stellar, that just doesn't matter all that much after you've rendered a whole fleet of PC's incapable of starting up.

Deflection and distraction from Crowdstrike apparently not doing any testing whatsoever on their update.

Still, they can't do much else otherwise they'd go bust if they admitted liability so they have to play this game.

Even if Delta doesn't win this is going to lead to more and more customers wanting to negotiate terms when they buy software like this to either get money back if the vendor causes them problems or hold the vendor liable.

Seems ridiculous but big customers seem to be able to negotiate this stuff. One place I worked Verizon Wireless was a customer and they were able to negotiate money back over bugs taking too long to fix and/or outages caused by the software. That was a long time ago.

I don't think it's ridiculous that if a company violates their SLA they should compensate their customers. A lot of hosting and cloud providers do this regardless of customer/contract size. The good ones will refund you automatically, and for others it's like pulling teeth.

Where does the responsibility of testing OS patches, AV updates and other automatic type installs fall? Systems deemed critical to your primary business must have some level of internal testing and validation in place from automatic external vendor updates. Does that responsibility not lie with Delta's IT management policies, regardless of CrowdStrike's Sales suggestions?

It's complicated.

The short answer is "yes. Much like a pilot's first job is 'fly the plane,' it's Delta IT's responsibility to run the infrastructure. Their house and their responsibility."

But the challenge here is the balance point on how to optimize that responsibility. Zero-days can go from dark web disclosure to one actor surreptitiously plugging a USB stick into a flight-information display kiosk that exfiltrates your ticket sale database in hours. To guard against that, Delta IT gave a vendor the ability to mass-distribute zero-day protection into all machines in drop-everything emergency mode. The fact they did so to dump an empty file that crashed every machine that read it is on them.

If a plane crashes because the propeller falls off, we blame the pilot for not keeping enough altitude to land safely in event of failure but we also absolutely blame the mechanic who's job it is to certify that propeller.

Reply on news.ycombinator.com