CrowdStrike's impact on aviation
heavymeta.org
heavymeta.org
I read somewhere that their crew tracking software was hit hard and took time to recover. Will look for source on that.
(Edited) source: https://news.delta.com/update-delta-customers-ceo-ed-bastian
“… and in particular one of our crew tracking-related tools was affected and unable to effectively process the unprecedented number of changes triggered by the system shutdown…”
[0] https://en.wikipedia.org/wiki/2022_Southwest_Airlines_schedu...
Keep in mind there was no way to opt out or delay CS Channel updates.
It would be like claiming raid1 is a backup.
Any DR system will have to accept some risks, and those don’t necessarily invalidate it in general, just make it insufficient for some scenarios.
Conversely, if they ran the main system on windows with crowdstrike and the DR one on poorly configured linux with no security software, they probably would have needed more sysadmins, had more trouble maintaining software for both, and been vulnerable to risk from both linux and windows bugs, so I feel like they made the right tradeoff in general.
I’m sure you, who can deride this DR system, have devised your own system such that it is resilient to a meteor destroying the earth.
That reminds me one of Corey Quinn's comfortable AWS truths.
https://x.com/QuinnyPig/status/1173371749808783360
> If your DR plan assumes us-east-1 dies unrecoverably, what you're really planning for is 100 square miles of Northern Virginia no longer existing. Good luck with that ad farm in a nuclear wasteland, buddy!
Do CS updates somehow work over airgaps? You know, the kind that production systems have to prevent any access to or from external networks? Well... some production systems anyway.
It looks like it wasn't a good security product after all...
https://www.tomshardware.com/software/windows/windows-31-sav...
https://www.forbes.com/sites/tedreed/2024/07/20/meltdown-wha...
> A story on the website govtech.com on Friday asked the question, “Why isn’t Southwest affected by the CrowdStrike/Microsoft outage?
> “That’s because major portions of the airline’s computer systems are still using Windows 3.1, a 32-year-old version of Microsoft’s computer operating software,” the website said. “It’s so old that the CrowdStrike issue doesn’t affect it so Southwest is still operating as normal. It’s typically not a good idea to wait so long to update, but in this one instance Southwest has done itself a favor.”
The govetech.com article is https://www.govtech.com/question-of-the-day/why-isnt-southwe...
which linked to https://www.digitaltrends.com/computing/southwest-cloudstrik...
which linked to an earlier Forbes article - https://www.forbes.com/sites/hershshefrin/2022/12/31/can-sou...
> The December 2022 scheduling fiasco was the result of skimping on information technology. I am old enough to remember when Microsoft introduced a new operating system called Windows 95, to replace its predecessor operating system Windows 3.1. The 95 in Windows 95 refers to the year of its introduction: 1995. By some accounts, major portions of Southwest’s scheduling system for pilots and flight attendants is built on the Windows 95 platform. That platform is now more than 25 years old.
“That’s it. That’s where all these stories can trace their origin to. These few paragraphs do not say that Southwest is still using ancient Windows versions; it just states that the systems they developed internally, SkySolver and Crew Web Access, look ‘historic like they were designed on Windows 95’.”
https://www.osnews.com/story/140301/no-southwest-airlines-is...
I am not claiming that they run Windows 3.1 or Windows 95 ... but rather "this is where that story was sourced from" because everyone kept linking to somewhere else. The relevant XKCD is https://xkcd.com/978/
Essentially modern versions of Soviet-style disinformation campaigns, but augmented with new technology (social media), and without the ideological hindrances of a Communist government (e.g. sell hard to both Right and Left).
RAND Corp calls it "the Russian Firehose" model: https://www.rand.org/pubs/perspectives/PE198.html
Similar approaches are also used by NK, Indian, Chinese, and other national-tier disinfo campaigns. This contrasts with models used by the West, which are often less about creating a disinformation clusterfuck, and more of a "watch our Disney / BBC / Scandinavian TV & movies and their implied messages about freedom and human rights and shit".
Obiously Russian, Chinese, and all other governments engage in information campaigns, and obviously the US government knows a lot about what they are. But we the public does not necessarily have the same information. It's not really possible to distinguish the "well known and documented (by the military and espionage industrial complex)" operation of foreign countries from domestic propaganda developed by those corporations and agencies to influence their own citizens.
It's not so much a severity as "hard"; but with the hub and spoke model that Delta uses, scheduling being down (at all on Friday), combined with FAA hour limits. It becomes exponentially difficult to reschedule flights.
Put more plainly, on Friday, your scheduling software is down for 4 hours in the morning, so you "borrow" any replacements you need for employees that are late or sick. This ruins the availability for the next flights, at which time you hope the system is up again; but if it's not, you borrow from the evening flights. Combine this with each flight that was late/cancelled as you were hoping to fill now affects the hours available for the employees that were available. Finally, as you've cascaded this, you head into a weekend trying to catalog how many hours each crew member did or did not log, and you're not sure how to get them back in time.
Otoh, Delta seemed to have recovered after about a week, and canceled about 1,000 out of about 4,000 flights for several days. It's way better to fly 75% of the daily flights than not. There's less wiggle room in a summer schedule for weather, but there's still some wiggle room.
I read somewhere that their crew tracking software was hit hard and took time to recover. Will look for source on that.
I heard on the radio (maybe NPR, not sure) it wasn't about the computers, it was about Delta's response.
According to the report, the other airlines delayed flights, while Delta cancelled them outright. That left Delta with more people and planes in the wrong places, making it harder to recover.
As someone else pointed out, they probably weren't ready by the time they needed their systems for the morning rush so they went to their business continuity strategy (manual). This has a throughput and recovery time penalty and obviously it compounds the longer they are in that mode.
I think what we're finding with the Southwest meltdown and now the Delta meltdown is that the big airlines just don't have the manpower or scheduling slack to accommodate going into business continuity. I do think this should be investigated. Hopefully financial penalties incentivize action but time will tell.
> big airlines just don't have the manpower or scheduling slack to accommodate going into business continuity
Which one profits the CEO more? Stock buy-backs or robust IT? Robust IT is only good for the company in the long term; however, with stock buy-backs or other skimping on IT, if disaster like this happens, the CEO just takes his golden parachute and leaves, but if no disaster happens, he gets a huge bonus to buy another private yacht.
Delta already took a huge financial hit for this.
So, yeah, lack of DR is why Delta was so screwed.
Unfortunately some of our outsourced suppliers didn't have such attitudes.
Obviously some selection bias there, but I'd love to hear some success stories.
definitely saw a blip and stuff had issues for 10 minutes, e.g. pages timed out or had to restart a process, but generally sites failed over and were able to keep limping on while we did triage.
got something like a $19 ($21?) service credit and an apology from the data center. our CEO shouted a lot and threatened lawsuits but it never went anywhere. Director of IT Infra quietly thanked all of us for having failover that mostly worked.
https://www.marketwatch.com/story/delta-hires-law-firm-seeki...
> To give you an idea of just how outdated this operating system is, Windows 3.1 was originally launched in 1992, and Microsoft ended support for it on December 31, 2001, except for the embedded version, which was officially retired in 2008.
I keep hearing the Windows 3.1 story repeated. I mean here it comes from TechRadar and even has the "Pro" in the name, they can't possibly make stuff up, right? But still don't quite believe it.
Can anyone working at Southwest confirm that their main scheduling system is running on Windows 3.1?
It’s wrong [1] and serves as a litmus test for whether an outlet independently verifies its claims.
(“The systems [Southwest] developed internally, SkySolver and Crew Web Access, look ‘historic like they were designed on Windows 95’.” That got mangled into they run 3.1.)
[1] https://www.osnews.com/story/140301/no-southwest-airlines-is...
I knew this story was false immediately though because no company ever even in 1993 had production server systems which ran a desktop OS like Win 3.1. It just wasn’t up to the task. They would have used NT if anything.
/s
The systems don’t run on W95, they look like W95
But that doesn’t mean this is the only modern design system that meets those requirements. And conflating all modern UI with consumer design trends is an equally frustratingly broad statement.
Very standard looking legacy Win32 looking app. Which, admittedly, would have probably look very similar had it been on Windows 3, but is probably running on LTSC Windows 10 or something in reality.
The button shapes, minimize/close window buttons, the titlebar are all looking wrong for Windows 95.
It looks significantly more like Swing, but then the buttons don't match that either.
It's not old-fashioned, it is _timeless_ B)
[1] https://www.cnn.com/2024/07/25/investing/southwest-airlines-...
https://kotaku.com/southwest-airlines-windows-3-1-blue-scree...
Not a tweet from Southwest, mind you. Not even a tweet from someone who says that they used to work for Southwest. Just... a tweet.
Good starting point is if the news is free. A shocking fraction of people get their news from solely free sources.
The core metric for subscription news sites is minimizing churn. A mistake will cost a subscription site subscribers who have a massive lifetime value. These sites are heavily incentivized to report high quality, accurate news even if they're not the first to break the story.
Yes, in this context.
> Why do you think an outlet that works for you will necessarily deliver better quality news that the one that works for advertisers?
I can’t explain the mechanics precisely. But it’s pretty clear when I compare my subscription and non-subscription sources where the quality lies.
If we held food safety to the same standards as paid news sources are held, people would get salmonella once a week.
Just like with anyone you meet: you are the judge if they are trustworthy, nice, mean, funny, etc.
That said, I think tech journalism is the bottom of the barrel. I just feel like they focus more on tech than journalism.
They don't.
I question the ethics and standards of the New York Times at least a little at this point so it's not like great journalism is common.
There are bad journalists (if they can be called journalists at all) and good journalists. At this point in history, our only hope lies with diligent reporters from reputable publishers.
Newspapers got the bulk of their funding from advertising back then, just as they do now.
Generally there is a chain of trust in news publishing that goes nowhere and there's nothing we can do about it, as more often than not, someone credible repeats the hearsay nonsense down the line, at which point they count as a primary source.
So much of news publishing I would describe as not even wrong.
Wikipedia sources an article from a semi-legit source. That semi-legit source either just says "sources" or points to something less-legit, like a Tweet.
You can bring new "facts" into existence by just laundering them from lower- and lower-quality sources.
I can't confirm that, but I can certainly confirm lots of hospital equipment is still running Windows XP and lots of hospital personnel browse the internet with Internet Explorer.
https://sfstandard.com/2023/02/02/sfs-market-street-subway-r...
That article links to an (only slightly older) article about British Airways loading navigation updates every month off of the fancy new 3.5-inch floppy disks.
At least they immediately mentioned it on their website, as a banner right at the top. The immigration office's appointment system has been down for over a month, and it took them 3 weeks to just acknowledge it.
https://finance.yahoo.com/news/delta-air-lines-seek-compensa...
Going after Microsoft seems like a misguided move here. What does Microsoft have to do with a third party driver installed by your own IT department?
Equally reporting on this whole issue seems to be by journalists, not techies. It's been framed (a lot) as a Windows issue not a Crowd Strike issue.
(8.5 million machines were affected, out of 1.4+ billion windows machines [1])
I have one affected customer (10k machines) who assumed I'd suffered like he did, and was surprised when I said we weren't affected. The reporting was consistently that it was a Windows issue, caused by an MS update.
Even this article leans into this narrative...
"as the New York Times put it, “It is more apt to ask what was not affected.” The answer is Linux, Macs, and phones."
Let me add "not to mention 99.4% of windows".
So the journalists don't know what happened, or who was affected, and felt "some computers have a problem" was a weak headline. The lawyers get that narrative and run with it.
And yes, it's easy to squint and claim the "OS should cope with this", but there's realistically limits on what an OS can do once you install a kernel-level driver on the machine. Should we go after Intel for making the chips?
[1] https://www.pcworld.com/article/608447/microsoft-delighted-b...
If I run bullshit on a Linux or MacOS box it can also be unstable and brought to its knees. Or is that poster really trying to argue there's no way you can get a Linux box to lock up?
> Third party vendors are forced into writing unsafe kernel drivers because Microsoft does not provide sufficient user mode APIs.
AFAIK it's different on Linux, and the reliability is higher. Is this not the case?
https://access.redhat.com/solutions/7068083
https://lists.debian.org/debian-kernel/2024/04/msg00202.html
You can install buggy kernel modules in Linux as well. I can't even count how many times an apt upgrade/yum update made my system unbootable when using nvidia GPU drivers.
And besides, if you're really wanting that AV system to deeply know about everything the operating system is doing and hook into tons of syscalls, you pretty much can't be running exclusively in usermode. If someone compromised the root of the system you then can't trust the info the kernel is giving your usermode application. eBPF isn't usermode.
And in the end, the poster literally said Windows is unsuitable because third party updates can kill it. That was the key takeaway from their post. Well, third party updates can kill Linux, it can kill MacOS, it can kill darn near everything.
...And neither were any of my Windows 7 systems affected.
"And yes, it's easy to squint and claim the "OS should cope with this", but there's realistically limits on what an OS can do once you install a kernel-level driver on the machine."
What are those limits, and why are they limits? Are there solutions? Yes there are. For instance, Microsoft doesn't takes snapshots of the working system state before loading a kernel patch, which on crash it would automatically reload without patch, nor does it employ various other techniques that would solve the problem.
I've discussed these issues in other posts so I won't repeat them here.
> Southwest wasn’t affected because they don’t use CrowdStrike
This could happen for anything that supports this type of automatic mass deployment. Just in this case that thing was popular enough and happened on one of the most popular platforms.
https://news.ycombinator.com/item?id=28750894
> infrastructure as compared to an OS
By the way, I don't think quality of Microsoft's infrastructure is relevant here.
In the former, you are paid well and have some sort of prestige and political capital. In the latter, you are underpaid and your prestige/political capital is often equivalent to the janitor's.
Blaming Windows for this outage is like blaming Linux for Apache bugs. The two systems are distinct.
It just so happens that Crowd Strike was very successful at selling to large corporates. That includes some airlines.
99.4% of Windows machines were unaffected. Including those of airlines using Windows, but not Crowd Strike.
This gets to be a problem when IT wants to get their hooks into OT networks. The PLC is meant to be left alone, and will happily send its Ethernet packet to that servo drive or digital IO card every 10ms for literal decades. There is no reason to update its firmware ever, just don't expose it to the Internet. But corporate wants everything on the Internet.
The PLC will reliably run its sequence when you close the contacts on the physical "Cycle Start" pushbutton. But if corporate is down, you can't know what part number you're supposed to make or how many of them, or get a serial number from and report test results to the traceability database.
If you want to monitor, you need to sit in the lab and watch, or if you're lucky, leave a PC with a webcam pointed at the tool and remote into that machine from your desk or your laptop at home.
This works, but long cycle times kill productivity, and engineering twiddling their thumbs costs money. It's easy to end up spending multiple hours a week just walking back and forth doing this dumb dance. One would expect that with 40+ years of networking experience, we would have come up with a way to securely perform these tasks without simultaneously exposing our tooling to cyberattack. Perhaps some kind of segregated network that can't access the internet, but gets pull-only access to the file share? Or vice versa - a screencap feed that gets sent through a data diode so an engineer can monitor the tool from their phone or laptop without being able to affect it?
Perhaps such solutions exist and are just beyond the IT skills, budget, or complexity appetite of the sorts of production tooling shops that I'm familiar with.
Caveat tho - I don't work in this space, I'm just friends with people who do.
The article quotes https://www.reddit.com/r/delta/comments/1edtfbh/why_did_delt... (with improper attribution)
topgun966Platinum wrote on Reddit """ These "experts" are completely wrong. The core issue was Delta did NOT have a proper DR plan ready and did NOT have a proper IT business continuity plan ready. UA, AA, and F9 recovered so fast because they had plans on stand-by and engaged them immediately. After the SWA IT problem, UA and AA put in robust DR plans staged everywhere from the server farms, to cloud solutions, to end-user stations at airports. They had plans on how to recover systems. DL outsources a lot of their IT. UA and AA engaged those plans quickly. They did not hold back paying OT for staff. UA and AA have just as much reliance on Windows as Delta. AA was recovered by end of data Friday and resumed normal operations Saturday. UA was about 12 hours behind them having it resolved by Saturday morning resuming normal schedules Saturday afternoon. The ONUS is 100% on DL C+ level in their IT decisions. The problem is that the lower level IT staff is going to get the brunt of the blame and the consequences. """
crowdstrike. n.
1. A set of major disruptions caused by an update that was not tested enough, pushed to many devices across the globe.
2. The name of such an update.
3. (by extension) a joke so bad it causes major disruptions.
For instance:
- Congrats for your crowdstrike! Now my weekend is ruined as I'll be the one who'll be asked to fix this mess.
crowdstrike. v. (simple past crowdstruck or crowdstriked¹, past participle crowdstricken, or crowdstruck, or (obsolete, regionalism) crowdstroke²)
1. Action of pushing an update to many devices that causes a global outage or major disruptions in various sectors.
For instance:
- We've been crowdstruck. Again.
crowdstrike. adj.
1. Qualifies an update that, when pushed to many devices across the world, causes major disruptions across the globe.
2. Qualifies such a (set of) event(s).
For instance:
- We are sorry for the crowdstrike event we caused. We gently remind our kind customers and their end users that per our ToS, we will issue no refund, and that no liability can be held against us. Customers who don't try to contact us in the following month will get a discount for their next contract renewal. You will hear us speak before the Congress, who nicely invited us for some comedy in the hope it will appease you all. Make sure you like the related videos on the various online platforms. We wish you a nice end of the week and nice, relaxing summer holidays.
² some people have tried to push crowdstricken, which first caught on in some areas or particular contexts. The idea that this form likens the qualified subject to the bearer of some sickness has eventually seduced a critical mass of people after some initial push back. Please also see the usage notes for strike for other, rarer, alternative forms [*].
[*] https://en.wiktionary.org/wiki/strike#Usage%20notes
(Thanks to the contributors in this thread)
"The update wasn't tested, so the servers are all crowdstricken."
I've already bought some of their stock, i'm pretty sure it's bottomed. I bet i make 30% a year from now. This always happens some "ohnoes!" event cuts a stock price off at the knees but then everyone forgets and in a year or so it's back to where it was before the event.
Said, "Yeah, it's all right
We're doing fine"
Yeah, it's all right
We're doing fine, so fine
Crowdstruck
Yeah, yeah, yeah, crowdstruck
Crowdstruck (crowdstruck)
Whoa, baby, baby (crowdstruck)
You've been crowdstruck
They struck a very big crowd real bad.
New investing strategy is to look for companies whose name also fits this pattern but who have not yet caused the disaster and short the stock.
From crowdstrike terms and services [1]: […] THERE IS NO WARRANTY THAT THE OFFERINGS OR CROWDSTRIKE TOOLS WILL BE ERROR FREE, OR THAT THEY WILL OPERATE WITHOUT INTERRUPTION OR WILL FULFILL ANY OF CUSTOMER’S PARTICULAR PURPOSES OR NEEDS. THE OFFERINGS AND CROWDSTRIKE TOOLS ARE NOT FAULT-TOLERANT AND ARE NOT DESIGNED OR INTENDED FOR USE IN ANY HAZARDOUS ENVIRONMENT REQUIRING FAIL-SAFE PERFORMANCE OR OPERATION. NEITHER THE OFFERINGS NOR CROWDSTRIKE TOOLS ARE FOR USE IN THE OPERATION OF AIRCRAFT NAVIGATION, NUCLEAR FACILITIES, COMMUNICATION SYSTEMS, WEAPONS SYSTEMS, DIRECT OR INDIRECT LIFE-SUPPORT SYSTEMS, AIR TRAFFIC CONTROL, OR ANY APPLICATION OR INSTALLATION WHERE FAILURE COULD RESULT IN DEATH, SEVERE PHYSICAL INJURY, OR PROPERTY DAMAGE. Customer agrees that it is Customer’s responsibility to ensure safe use of an Offering and the CrowdStrike Tools in such applications and installations. CROWDSTRIKE DOES NOT WARRANT ANY THIRD PARTY PRODUCTS OR SERVICES.
[1] section 8.6 of https://www.crowdstrike.com/terms-conditions/
This is pretty standard. There is almost identical language in the Windows and macOS EULAs, for example.
In the PSAP I support we have three dedicated PCs at each workstation to run the CAD, phones, and radio. Each of those has a dedicated VLAN, separate physical servers and storage, separate Active Directory forest for CAD (no AD for radios or phones-- standalone PCs), and default-deny ACLs for inbound and outbound traffic on the hosts and at the borders.
A fourth dedicated PC (VLAN, ACLs, physical servers, AD environment) does email, web browsing, etc. (All of it is shackled together with a nice KVM that supports a single keyboard and mouse controlling up to 5 PCs.)
Not every PSAP does this and I think that's insane. The law and fire agencies we interface with absolutely do put a single PC on a desk (or in a cruiser) and use it for everything (and we filter and monitor the traffic coming in from them over our VPN heavily and block access at the first sign of anomalous traffic). Often their budgets don't support the notion of using dedicated computers for task-oriented work. The marketers have pushed general purpose devices for this kind of application.
In the last 5 years all three "hardened" systems we use (all companies acquired by Motorola) have started requiring Internet access for various APIs they use, and for integration with third-party vendors (mapping, public information databases, and task instructions for telecommunications). I think it's ridiculous, but I don't get to decide the direction of the product roadmaps or what the business stakeholders want from a feature perspective.
Motorola (who makes the CAD software used by some of the largest US municipalities) is pushing for hosted CAD and integrating hosted features into on-prem systems. (Of course, they have a managed security product offering that they want to sell along side it.)
(If such a thing did exist, it would cost a lot more!)
Example: As of right now I am still required to expire passwords every 90 days. My state is considering the current guidance from NIST but FBI CJIS policy still mandates the expirations.
If that's also the interpretation of the courts, then each company would be invidivually liable, at least towards the government.
There are often 3 opinions between any 2 lawyers so we have a chance to learn the outcome many months and millions of dollars later.
At this point using windows for these tasks seems like using legacy software because training people to use an iPad or a web browser seems too complicated or because no one wants to move their age old systems to a more modern web based system because of costs. Native apps work great, but I think the world is moving to the cloud and that means web based everything should be the norm. Yes AWS AZURE outages can still happen but those can be fixed by spinning up a VM in different clouds.
This is also why software jobs aren’t going anywhere thanks for a while. Many systems need to be changed to more modern and robust clouds. It might take decades for this transformation across the globe.
The world is absolutely full of things that have worked for decades to centuries without the Internet, are eventually more or less consistent (remember carbon paper credit card machines?), and did an amazing job of keeping the world running despite, wars, network partitions (the “network” would basically always be partitioned), mistakes, entire branches offline, etc.
Sure, a lot of things are easier when centralized, and “the cloud” is incredibly powerful. But it’s not necessarily more robust. Also, depending on any sort of cloud means you’re also depending on the network, and networks are far from infallable. There’s a reason that a lot of stored-value transit systems still track balances on the card and will let people in even if a fare gate cannot connect to a cloud service.
And CrowdStrike took out plenty of cloud instances, and recovering them can be worse than recovering physical hardware, as the “robust cloud” has an absolutely terrible ability to do anything outside the happy path of booting an instance normally.
I guess dishwasher years are like dog years. At least it definitely behaves like a teenager at 2 years old, finishing when it wants to finish. Estimates be damned.
It actually is. Fixed length cycles haven't been a thing for many years now - modern washing machines adjust the washing cycle length by the weight of the laundry and its behavior during spin-drying, both its vibration behavior aka weight distribution (that can have multiple adjustment cycles to achieve reasonably even distribution) and how much water it loses - when no more water comes out during spinning, it will cut the cycle short to save energy.
If it says (e.g.) 43 minutes, but sometimes it takes 40 and sometimes 49 or 53, set your timer for 60 minutes and get on with life. Your laundry sitting for 17 or 7 minutes isn't the end of the world. If your timer goes off and it's still not done, set it for another 20 and do something else.
Of all the things to fill your head with worry and annoyance with, laundry is near the bottom of the list for me.
If I buy something new like this and have a few choices, I intentionally pick the one with as few smart features as possible.
I'm already mentally replacing "cloud" with "clown" anyway, to the point I have to stop myself from accidentally saying "clown computing" out loud.
1. Check back in an hour (like my (grand)mother did—and she managed to do laundry without Wifi).
2. Or: have a washer that beeps.
3. Or: set a countdown kitchen timer (or a timer on my phone) that will beep if my washer does not have a washer.
There are complicated situations in life: doing laundry is not one of them.
In all seriousness, I think never has there been a better time to educate people on the fundamental philosophy of computing freedom, and I usually start with Eben Moglen and RMS's talks with people.
I don't know how much of this is generational, or how much of this is corporate sell out, or maybe even sockpuppetry for consensus cracking and other psyop techniques, but relearning the lessons of early computing (such as being able to do things offline, locally, as a core part of a functioning decentralized system), seems highly in order.
Where I went to college, our dorms had (free) shared washing machines. This was "pre cloud", but wifi was throughout. One student rugged up a hall-effect sensor and attached it to each power cable. It could detect if the washers and driers were on. It sent this info to a specific website that the students could monitor to see if there were any available washers or driers.
If it is serious, you could always set a timer.
I'm not sure we're quite at the stage where a check-in agent using their personal un-managed devices to handle passenger data via a web-app is a great idea.
iPads aren't designed to be turned into kiosks or airport departure displays and web browsers aren't operating systems (except maybe ChromeOS). So this advice boils down to don't run Windows, but CrowdStrike has caused outages of Linux as well.
Resilience comes from diversity, in computing and in biology. Whether that's having critical workloads on multiple cloud providers or having one user interface on windows on network A (Arista) with crowdstrike and one on a mac on network B (cisco) with Sentinal one
Sometimes perhaps you can't eliminate a single point of failure, but you can sure reduce them to a minimum.
Or you can choose to increase next years bottom line and thus your bonus by not having a robust DR plan or system. You can also skip on boring things like raid and backups.
The trick for a CxO is to ensure that when failure happens, it's massive and widespread. Then it's not your fault. The CxOs in a given industry won't be fired because their DR plans didn't work because they believed Gartner and all their CxO chums in competitors did the same thing.
Nobody got fired for choosing IBM/Microsoft/Cisco/Crowdstrike/Azure, even if it's worse than the alternatives. People do get fired for bucking the trend even when it's measurably more reliable.
Monoculture means a successful attack will take out all of your operation.
Cyber attacks rarely take down stuff directly. Rather attackers will establish a bridge head into your organization first and inspect the network and gather data for further (phishing) attacks.
Diversity only means more opportunities to install bridge heads.
[1] https://blogs.microsoft.com/blog/2024/07/20/helping-our-cust...
native desktop apps are absolutely necessary for most professional / serious work and native desktop apps need offline support too.
with cloud - your risk factor goes up massively.
the risk here is that most of these companies are reliant on windows and of course snake-oil salesman of antivirus tools.
if you have a proper native desktop app, that runs in a sandboxed environment then you simply wouldn't need crowdstrike and the likes.
unikernels / bsd jails are things that have been well known and will easily mitigate "security" issues.
even windows these days has sandbox mode.
but incentives rule the world.
- Latency
- Security
- Legal obligations
- Offline work
- Managing the different sources of locking.
- Avoiding a single point of failure (I get the irony).
People need to stop believing everything they read on the Internet and have a little bit of skepticism.
With all of the angry customers, lots of incoming lawsuits, and the fact that their "protection" is provably more costly than no protection at all now - I can't imagine why investors aren't dumping it like mad.
Companies already trust Microsoft, they buy Windows, Office, Azure.
Why would they bother with a 3rd party here when the low effort low risk solution is to pick the tool made by the OS vendor. I.e. windows defender
It should be a nobody gets fired for picking IBM situation. How did this random place get so much credibility that people trust them over the manufacturer?
2. Companies react slowly. When has a vendor paid a high price for failure? Boeing can kill people and fail time after time still sell planes.
Catastrophe always changes less than anticipated.
Huh? Once you get the control plane/backend of a new AV vendor configured, you just uninstall AppA and deploy AppB on your nodes.
It's not like Crowdstrike is deeply integrated with other systems: it's an agent.
Having not done this just cost them billions. Now imagine the US at war or another nation state that just wants to cause havoc.
You're suggesting either that Crowdstrike itself will get used as a vessel for an attack, or that banks and airlines have firewall rules open for enemies.
The major airlines may not be "small time operations" but none of them are even in the top 100 US corporations by market cap. They simply don't have the level of IT resources or competence that we see at major tech companies.
That's really only a secondary problem though, because disconnecting a network from the Internet isn't a replacement for security software or software updates, so you wouldn't even avoid the root cause of the issue here. I'm not saying CrowdStrike is essential software for Internet connected computers either, but if your business thinks it is, you should probably be running it on your "airgapped" computers too. And you should definitely be installing updates, so you can still fall victim to a bad updates regardless of which software you run. At best you perhaps increase the likelihood of hearing about a problem with an update before you deploy it on disconnected computers, but you can get a similar effect by delayed deployment of updates even on Internet connected networks.
My experience from the airline industry is that the vast majority of systems classified as flight safety critical are not connected to the internet, or large networks at all. Which is good.
But unless we want to drastically change how airlines operate, the rest need to be online.
Today, you can purchase a ticket (or rebook an existing one) on your phone really close to the departure time. When that happens, a gazillion interconnected systems, across legal entity borders, need to cooperate to take you (and your luggage) to the destination.
To put all of this in a large non-internet network seems pretty pointless.
If we wanna go down that route, the only real "security improvement" I can think of is to dismantle the digital systems and go back to paper. Like Ryanair did during this incident. Handwritten boarding passess, verified against print-outs of passenger manifests.
But those couldn't have gone down due to Crowdstrike.
Saying "run your own network" isn't exactly practical. Even imagining the very small airlines (that partner with big ones for the last leg) that only service a handful of rural airports, this doesn't seem practical.
The days of point to point updates over modems are overish with the amount of data that needs to be consistent transmitted and available.
I can imagine a modem at each airport and a phone bank of about 700 modems that are each getting or sending updates. The long distance calls to distant countries for that data could get expensive. Woe to the power outage in Texas that takes down the phone bank for a day or two or three or four.
Alternatively, there's a system that was developed to do this and it works pretty well most of the time. Combine this with having redundant systems there that are geographically separated. It isn't turnkey, but its probably better than other options that would involve home grown solutions.
Problem once again is humans. Humans need to interact with systems, either receive information from them or give it to them or use the systems to process it. And for efficiency in general it nowadays happens online. It could be offline, but that would be slow. Or it could be segregated networks but that would get really expensive. Imagine having own fiber line for your instant messaging and email? With different terminal...
In the end most of these affected computers are on Internet for very good reasons. And this model really is working vast majority of time generating lot of efficiency.
In the case of CrowdStrike, they would have been able to deduct this event from their emissions for decades.
this is pretty damning both ways
on the one hand, it's insane, unfathomable and inconceivable that anyone can run anything critical on windows 3.1 (!!!)
on the other hand, it's equally insane, unfathomable and inconceivable that those who do are actually better off - 30 years of "progress" is actually just bs? what are we as an industry "even doing here"???? is computing actually a solved problem and we're really just mostly reinventing the wheel and enshittifying perfectly already working systems?
Edit: https://kotaku.com/southwest-airlines-windows-3-1-blue-scree...
80% of the work is json bureaucracy
The other 80% is adapting to new requirements
And if you’re lucky maybe 0.1% of the time you get to build something new.
Fear not, a lot of this stuff was perfectly solved with pen and paper long before us computer nerds came to play in the big boy sandbox
We're talking about a problem on the scale of 4,000 flights per day. Assuming you avoid O^2 complexity computations that's the sort of thing even 90s computers could handle easily.
Two of my first contractor roles I had as a developer really opened my eyes to a lot of this.
We were building an inventory management system for a large company that built farm equipment. We started building it and once we got to the browser and mobile requirements, one of the VPs spoke up and asked if it would run on IE6 since they had not one, but THREE of their inventory legacy systems that still ran on Win98. This was in 2014, a full 6 years after many companies had stopped supporting it. And another 4 years since websites stopped supporting it.
The other one was for a very large, regional construction company. Same thing, we were building a web app for them and in one of the conference calls, one of the VP's was asking how this would run on Windows95 for the same reason. They had several legacy ERP systems that were running on Win95 and had specific requirements for stuff to run on that OS.
As a developer who was used to working with somewhat current tech - it was a real eye opener. It was crazy to think how many massive companies just didn't have the constitution to upgrade their stuff, and then by not doing so, had now dug themselves into an even deeper hole.
Once I started hearing stories about the details of this scenario, it made perfect sense to me since I had seen it multiple times. And not from little companies who didn't have the money or resources to upgrade, but massive Fortune 500 companies who just neglected their stuff until was too late.
A lot of software doesn't need that additional complexity. Having thirty year old software, if properly sequestered (since there are security holes large enough to fly a 737 through), means that this is software that has been working for three decades. It has issues (as the mess they had previously showed), but Southwest appears to be able to be able to manage that to some degree without needing to incur the additional complexity of managing a modern software stack for application software that doesn't need it.
The ability to play minesweeper on critical computing equipment without impacting it isn't necessarily a desirable feature. Having the computer boot in five seconds and run the desired application is.
And there are a number of ways to handle that ... running old operating systems is one of the ways. Space Force S02E07 is not a desirable situation https://youtu.be/xDLvUqhwHZc . You could also have a kuberentes cluster with multiple replicas and load balancing and all of that additional complexity that takes more people to be able to manage without any real gains in what the application itself is doing.
There are certainly more modern options that allow that and it's highly doubtful that specifically is particularly relevant for Southwest.
Not that there is any evidence that they're actually using 3.1 for anything?
> that this is software that has been working for three decades
Or it's so buggy or designed (or more likely updated) so poorly that everyone is afraid to touch it. e.g. I doubt there are many (even any?) practical reasons for airlines to use GDS besides the cost and complexity involved in designing an entirely new system and somehow forcing all other airlines to switch to it?
Windows 3.1? No. I'd even say there's no evidence that windows 95 is being used but rather that they've got what appears to be some old software with older design.
https://www.dallasnews.com/business/airlines/2022/12/30/what...
> 2. The crew scheduling system is the main culprit.
> Southwest uses internally built and maintained systems called SkySolver and Crew Web Access for pilots and flight attendants. They can sign on to those systems to pick flights and then make changes when flights are canceled or delayed or when there is an illness.
> “Southwest has generated systems internally themselves instead of using more standard programs that others have used,” Montgomery said. “Some systems even look historic like they were designed on Windows 95.”
Screen shots of this are in http://www3.alpa.org/LinkClick.aspx?fileticket=IO7kd%2Bfm2Do...
Unfortunately, I don't know the nuances of Microsoft Windows UI well enough to be able to pick out which OS version is running the software in those screen shots.
---
> Or it's so buggy or designed (or more likely updated) so poorly that everyone is afraid to touch it.
That is a very common occurrence (I'm dealing with that now ... a .jar file that hasn't been rebuilt in 15 years). The big rewrite is something that comes with one part excitement (I can do it right this time!) and dread (oh my, that's how much code that I need to retest?!).
I was involved in the tail end of a 3 year project at one company with some software that replaced previously running DOS (and yes, it was DOS - they had an C and assembly guru employed who's job it was to remove / optimize code in the binary to get it to fit into 640k) to a Java Web Start (which was a neat technology) and the millions of lines of software that monstrosity had and needed to be debugged and fixed.
While they're in a better spot now (can use modern hardware), and its something that they can build in house (a major part of the reason to do it was to drop the external contractor who didn't like maintaining the C code) ... but that also came with the added complexity of the software that they licensed and the maintenance and deployment of that software. Before they could put the software on a floppy and have it shipped to each location ... now its a big bigger and more complex of a deployment (that was built with duct tape and chewing gum one night to do diff deployments of specific class files rather than trying to push the entirety down the pipe for each location).
My rambling point is that we are moving forward with complexity - and that allows us to manage more complex situations, but it comes at the cost of managing that additional complexity of the infrastructure and software it needs and that cost is ongoing and not always taken into account.
They had json apis. Each one had some variation on parsing http from a raw tcp connection with IBM RPG. I had to do some unspeakable things to a ruby library so I could control the order of the headers.
Computers only started showing up in nationwide productivity figures in the 80s to 90s.
That's a stretch.
I guess it increased the productivity (measured in amount of "work" done, not necessarily something useful) expectations for most workers which effectively did nothing for them because they still need to work as much even if modern software allows them to accomplish much more in the same amount of time. So t might make sense in that regard.
> workers tapped at light speed due to muscle memory
That's great if we're mainly talking about robotic tasks than can be mostly automated to only require a fraction of those clicks but wasn't for some unclear reasons.
Don't get me wrong, my Macbook is an absolute beast for all of my work tasks, and my gaming PC is an utter joy to use for my recreation time, but at the end of the day, for a ton of applications, a computer doesn't need to do shit beyond sending a lot of signals out of a parallel/RS232 port to control systems to operate... I mean Christ, anything. CNC mills, building lighting/security systems, packing machines, or to do things like issue tickets to people parking in a ramp. Like... a lot of this stuff just does not benefit at all from a modern software stack. Stick a crappy PC inside instead, load it up with the same image it had before which includes firewall rules that shut down every last port and connection apart from whatever needs to manage it, and you're done.
Don't fix what ain't broke.
On a whim I tried playing Solitaire on windows the other day. You know, that game that’s shipped with windows since forever. Well, it’s horrible now. When I tried firing it up, it first spent several minutes downloading software updates. Then it loaded in some horrible “casual games bundle” app which felt laggy like a web app - complete with Xbox cloud sync for my progress, and daily achievements and other junk.
The game used to run flawlessly on my old 486. My computer now is orders of magnitude more powerful - but solitaire feels laggy. I bet the entirety of windows XP is smaller than the “update” it performed to install solitaire.
I have a personal theory that there’s always something that gets the attention of the best engineers. Decades ago it was human interface guidelines and UI toolkits. Today it’s LLMs and AAA game engines. Most of the rest of the software in the world is worked on by the B team. And they don’t blink an eye at the idea of rewriting solitaire for windows on top of electron. If JavaScript is all their team knows, so be it. Heaven forbid we have to learn how to properly build software for windows.
They already are properly building software for Windows, and your Solitaire app is a good example of it. It's much, much better than it used to be: it's laggy and slow, and downloads a bunch of extra crap, which has the potential of getting you to spend more money. In short, changing from the old and simple Solitaire to this bloated mess makes Microsoft more profitable, and customers like you keep using Windows regardless, so why shouldn't they do this?
Meanwhile, on my Linux box, my simpler games haven't changed substantially in years, if not decades (perhaps updates for the newer GUI toolkits being used), and still work great. But people prefer to stick with Windows and then complain about things being too bloated.
I don't think any of the extra stuff is monetised. I don't know; I didn't look closely at it.
> customers like you keep using Windows regardless ... But people prefer to stick with Windows and then complain about things being too bloated.
"Stick with windows"? I run windows, macos, linux and freebsd at home. I want all of them to be better. It pains me to see any of the great operating systems of our time fall slowly toward mediocrity.
Of course I complain about it. Windows should be better than this.
If they weren't making money out those changes somehow, they likely wouldn't be dedicating resources to making those changes.
>Of course I complain about it. Windows should be better than this.
Why should it? It's not yours, it's Microsoft's, and it exists solely so their shareholders can extract money from customers. It's not there to be whatever you think an OS should be. They're doing things to it that they believe will improve their profits, and that's all Windows needs. As long as MS makes changes to Windows that improve profits, that by definition makes Windows "better".
Microsoft makes more money when the broader community wants to use their products. Users are stakeholders.
You’re right in part - Microsoft’s incentives aren’t perfectly aligned with mine. But they aren’t in opposition either. We’re both better off when Microsoft makes their products better for me such that I give them more money. Microsoft will try things on - of course! But if the broad community hates what they’re doing, they will stop doing it. They must, if they want to remain in business.
Which is a round about way of saying, yes of course windows isn’t “mine”. But as I’m a paying customer, my opinion still matters - or at least, the opinion of their customers in aggregate.
When you’re a customer in a situation like this, you have at least 2 choices: Voice and Exit. In this case, exit means dumping windows. And voice - at least how I’m exercising it - means making a bit of a stink about it in a forum like HN where microsoft engineers sometimes visit.
If you don’t think this is the forum for that, feel free to downvote my comments. But in the meantime, it seems mighty strange of you to try and convince me that my opinions don’t matter and I should… what exactly? Be quiet and take it instead? Provide no feedback and quietly migrate to another equally imperfect ecosystem?
I’m not sure how I would be helped by being smaller in the world.
>Microsoft makes more money when the broader community wants to use their products. Users are stakeholders.
>But if the broad community hates what they’re doing, they will stop doing it. They must, if they want to remain in business.
This mostly isn't true. MS customers are going to use Windows no matter what, unless MS somehow makes it so completely unusable they're forced to abandon it. So they can make whatever annoying changes they want, and their customers aren't going anywhere.
However, this isn't true of everything MS makes, so you have to be careful not to conflate Windows with anything else. If people hate Teams enough, they might be convinced to switch to Zoom or whatever. This just isn't the case with Windows. They're free to piss people off as much as they want here.
>But as I’m a paying customer, my opinion still matters - or at least, the opinion of their customers in aggregate.
No, it really doesn't. You aren't paying for a Windows subscription, you paid for a Windows license. You probably paid it when you bought your PC. Or if you're a big company, you probably have some sort of site license, because you made the decision to use MS products across your organization. And if you're BigCo, you probably don't have Solitaire installed on employee computers anyway. So MS making the solitaire game slow and shitty isn't going to change the amount of money they get from you: they already got your money when you bought your PC. Even worse, you probably didn't have much choice: there aren't a lot of no-OS computers unless you assembled yours from parts, and those that exist (or come with Linux pre-installed) usually cost more, because MS isn't making money from you so much, but from kickbacks from all the crapware that's pre-installed.
>...a forum like HN where microsoft engineers sometimes visit. >it seems mighty strange of you to try and convince me that my opinions don’t matter and I should… what exactly?
What makes you think MS engineers have any sway at all over the user experience? That stuff is decided by upper management, product managers, etc. And they don't care about you; they only care about the company's profitability. Adding more crapware and ads into Windows makes it more profitable, so that's what makes sense for them to do. Making a bloat-free, high-performance OS without any annoying ads or other crap doesn't make them more money, because you and everyone else are going to keep using Windows (and buying new computers with it pre-installed) no matter what.
>it seems mighty strange of you to try and convince me that my opinions don’t matter and I should… what exactly? Be quiet and take it instead?
Basically yes: you're wasting your keystrokes complaining about Windows enshittification, because your opinion really doesn't matter. It's not like MacOS, where someone has to actively want to use a Mac to go buy one, or Linux where you not only have to actively be willing to buck the trend, but also choose which of dozens of distros you want to use. With Windows, it's just the default choice for 90% of users, and they're not going to switch to something else, which MS has found out over the course of decades now.
>Provide no feedback and quietly migrate to another equally imperfect ecosystem?
Other ecosystems are far more likely to take your complaints seriously and to worry about your user experience.
Thats not what happens.
Windows is worked on constantly. And most of the changes are clearly done with the intent of making the product better. Things like IO completion ports. C# and the .NET ecosystem. Support for big/little (P/E) cores in the kernel. Edge replacing IE. ARM support. And so on. Of course like any big company, they make plenty of bad choices - like telemetry, the "watch everything you do" AI assistant, and so on. But its too simple to paint any large company with the "evil megacorp" paintbrush. Microsoft has 200 000 employees. I'm sure some of them are exactly who you think they are. But some - I suspect most - of them really want to do a good job and make products their customers want to use.
Just like any big company.
> Other ecosystems are far more likely to take your complaints seriously and to worry about your user experience.
Are they though?
Parts of Apple are great. And other parts are obviously horrible, extractive and greedy. Did you know companies can't use the NFC chip in Apple phones - in my phone, which I've already paid for - without being paid millions of dollars for the privilege? I'd love it if I could pay for public transit in my city using my phone, which I own, but my government doesn't want to pay the extortionate price apple is charging to bless them with that capability. Horrible.
Companies aren't good guys or bad guys. They're just big groups of people. And people are complex, and they have a wide variety of incentives and drives. Calling companies out for bad behaviour and sloppy work is important - even though it often has no effect. "Killed by google" is infamous inside google. Microsoft rolled back their AI assistant thing after the bad press.
If you're looking for great products made by companies who are ethically spotless, well, there aren't a lot options. Linux on the desktop is pretty decent these days. But I still use windows for gaming. And I'm typing this on a mac.
Now this isn't what I think either. It's the top executives mainly that drive this stuff, since they set the direction for the company. Middle managers are just pawns, except for the ones who are empire-builders trying to work their way up. Also, I'm not claiming that all companies are this bad, but I think MS is a unique company, and Windows a unique product, because of its monopoly status. Other companies really can't afford to piss off their customers too much because they'll just jump ship. MS doesn't have this problem with Windows. Remember how paranoid they were about Linux back in the early 00s? Now they don't seem to care at all, and I think it's because they figured out they had nothing to be worried about after all: almost no one was going to stop buying or using Windows (or even if they did stop using it, they still bought a license with their PC anyway; the addition of pre-loaded crapware also changed the economics here so they weren't worried about actual license fees from individuals).
>If the world was really as dismal as you think it is, microsoft would destaff the entire windows team and just let the ecosystem rot while reaping in their locked in revenue.
Of course, they can't just go to this extreme. They'd keep taking in money, but things would go down eventually, and they wouldn't grow revenues either, and remember, Wall Street wants never-ending growth. And as you point out, the ecosystem is important (as Apple has shown); they make a lot of money from all that other stuff too. Windows itself might not even be that much of a money-maker these days.
>Of course like any big company, they make plenty of bad choices - like telemetry, the "watch everything you do" AI assistant, and so on.
MS can afford to do a lot more bad stuff like this, because Windows users aren't going anywhere. It's why they can bake ads into the OS.
>I'm sure some of them are exactly who you think they are. But some - I suspect most - of them really want to do a good job and make products their customers want to use.
I'm sure there's some of them too, but they're not pulling the strings. They do get to work on cool stuff now and then of course, like the interesting kernel features you listed, but those are in support of their long-term goals (like being prepared for a post-x86 world).
>Did you know companies can't use the NFC chip in Apple phones - in my phone, which I've already paid for - without being paid millions of dollars for the privilege? I'd love it if I could pay for public transit in my city using my phone, which I own, but my government doesn't want to pay the extortionate price apple is charging
No, I didn't know that. It doesn't surprise me though; Apple was like this with Firewire too, and it's why Firewire is dead now. But that's weird too, because here in Japan Apple users routinely use their phones for the public transit, and I kinda doubt the IC card companies paid that much, though I guess it's possible. (Android users use their phones too, but only if their phones are Japanese models. iPhones however all have the Felica NFC chip needed for Japanese transit systems.)
It will support a huge number of new chips, new peripherals, more memory, and so on.
If I'm running Southwest's crew scheduling software, how much of that do I care about? Do I care that it will now support the latest Bluetooth? Do I care that it now has the same UI as tablets? Do I care that it has better ads to display on the start menu? No, no, and no.
The only thing might be more memory. (I mean, the UI might not look like it belonged in the Stone Age, so that's something, I guess...)
There hasn't been a real fundamental improvement in the functionality of OSes since Windows 3.1. It's all been device support (including new classes of devices), new CPU support, and new UI styles. (The security improvements in Windows were a legitimately big deal, but those were fixing what was broken, not adding new functionality.)
And I'm sure that, having said this, someone is going to point out something really important that I forgot...
Not a lot of if you can't/don't want to upgrade or replace that software to make sure it runs on modern OSes. I'm sure that for the most part that software is causing various unnecessary issues and decreasing potential productivity at least to some extent. Just look at GDS, they love to get rid of that, but that would require a coordinated effort and extensive collaboration between all major airlines which is somewhat tricky.
> There hasn't been a real fundamental improvement in the functionality of OSes since Windows 3.1.
Multitasking? A massive amount of other important features that matter if you want to build new software or significantly improve what you're using now.
Also you seem to be downplaying security a but too much? Those devices would need to be carefully isolated from everything else (not that as I understand there is any evidence that Southwest Airlines is actually using 3.1?).
> but those were fixing what was broken, not adding new functionality
It's like saying that every new feature in introduced in any type of software that wasn't entirely novel was actually fixing stuff that was broken and wasn't "new". I guess some would apply to the claim GUI/desktop wasn't something new but it was just "fixing" (inherently "broken") command line interfaces?
Being able to design significantly objectively better (based on how much it could increase productivity) is I guess is not strictly tied to the OS at least in some cases. But it certainly make it a lot cheaper/easier.
I'm presuming that Southwest's internal software backends are not internet exposed, which is why I'm downplaying security.
OMFG, does this mean we need to be prepared for a (juicy) “IT failure” that brings down Southwest at some point?
https://en.wikipedia.org/wiki/2022_Southwest_Airlines_schedu...
[1] https://www.cnbc.com/2021/10/12/southwest-airlines-reduces-c...
[2] https://www.npr.org/2022/12/26/1145536902/southwest-flight-c...
[3] https://www.npr.org/2023/11/09/1211064462/southwest-airlines...
A lot of code lives on much longer than you think. The general attitude we took was that most of the code we were writing would be running for at least 30 years. And that was the attitude at an R&D branch, arguably a side of that industry where we were working on the new tech.
Edit: Win 3.1 or something else, the point still stands. There is a lot of old software running out there that will continue to run our core services. Legacy software doesn't just mean v1 versus v2, it can mean v1 versus v41.
Southwest isn't running Windows 3.1, though. That's some rather lame, but predictable, truth-through-repeated-assertion thing on social media.
Not everyone uses CrowdStrike, and in this case SW was the lucky one that didn't.
Instead I saw a lot of MML and (happily) TCL.
Southwest wasn't affected because they don't use Crowdstrike. That's it.
Notably, crowdstrike won't run on 3.1, and thus you're kinda right.
Relatedly, it is nice to provide a source for your claims. I did see this [0] which would have been an appropriate thing to link
[0] https://kotaku.com/southwest-airlines-windows-3-1-blue-scree...
> For everyone flabbergasted by Southwest running ~Windows 3.1~ old software
You said it; own it. You could have said "For everyone who was tricked by the joke that Southwest runs ~Windows 3.1~ old software" but didn't.
The really damning bit is Windows 3.1 did not have preemptive multitasking. It barely had networking. You couldn’t run a server with it if you wanted to.
Or, at the time, OS/2
The “ingenious” strategy saved them from a weeks worth of downtime this year. But that same “ingenious” strategy was the primary reason for their meltdown in 2022
[1] https://www.npr.org/2022/12/30/1146377342/5-things-to-know-a...
[2] https://www.nytimes.com/2022/12/28/travel/southwest-airlines...