See some of the comments from affected medical staff on the main outage story, but the tldr is, tightly coupled systems.
Fail safe is the only acceptable failure mode for any critical system. Crowdstrike failed here, but they're not the only thing that can go wrong with computers. Where is the redundancy?
Also, air-gapping helps only so much when network dies and hospitals can't exchange patient information or send images from MRIs and X-rays to radiologists.
My dentist literally took a photo of my x-ray with his phone and sent it to to my orthodontist via Whatsapp and everything went quick and smooth, much faster than the official channels. Solutions to get a job done quickly and efficiently in case of emergency always exist, they're just not "by the book".
Hippa doesn't apply in Europe but GDPR, and I don't see how that would be in violation since my information was exchanged only between the two parties with my consent, on an encrypted channel.
They would only get into trouble if that info would leak in an identifiable way to unauthorized third parties and would cause damages (here there's no punitive damages like in the US). And people here tend to guard their WhatsApp chats pretty well since it's what everyone uses and it also contains their private chats so in a sense it can even be more secure than the official medical channels which are just more burocratic but offer no actual guarantee of more data security.
Say WhatsApp is found to have a security hole that has been leaking data to 3rd parties. What may be the fate of dentists / doctors that decided to use it an "encrypted channel" for medical records? Are doctors / dentists not fat targets for lawsuits? What might the guidance be from their lawsuit insurance policy?
I understand they would just postpone whatever can be postponed to save the headache, I don't get the stories about life/health threatening situations.
Note that I am not a doctor and have absolutely no specific knowledge beyond what is in the original article, but I am guessing at potential explanations.
Additionally, the article states that there is some "wiffle [sic] room" around the timing of the infusions. So it may be that the delay is not quite as serious as the title makes it sound.
Again, I understand that restoring a complex net of servers is hard and takes time. But they surely have local hospital IT admins for these absolutely critical computers who are always available on site and can do it individually - it's not like there will be more than a hundred of these at a particular hospital? Hack it a little if you have to, disable the SSO etc - all that can be fixed later.
I think it's always a mistake to outsource control of a mission-critical system, but that is exactly what large tech companies have been encouraging every organization that will listen to them to do for decades now
The culture of organizational IT is broken because a lot of powerful companies found it profitable to break it and leave something inadequate in its place
Again, we had all of this for a forest logging operation - is it too much to expect at a hospital?
Why did we get here? If you're installing kernel-level software you might as well run a kiosk that only runs presigned code and runs off a read-only system image. And a lot of the machines in question DO APPEAR to be kiosk settings (like hospital data entry terminals).
It's easy to sit back and armchair, I'm sure there will be many cybersecurity experts who would figuratively jump at my throat for suggesting that trusting a vendor to run a rootkit on your computers is a bit incompetent. LOL. :D
[0] AppleMobileFileIntegrity, the daemon and kext on iOS that enforces very strict code signing.
Restore from backup or reimaging fresh often means you need a working backup or image server, which at a lot of these places is also a Windows server and is likely also running the same endpoint protection, and is likely also boot looping.
Restore from zero isn't something any IT wants to do, and many of them aren't prepared to do it either.
Like it or not, hospital care revolves around the electronic medical records systems, and while Kaiser Southern California in the 90s was using amber screens and some sort of mainframe, afaik, almost everyone is on EPIC now, which is a windows application with all the baggage that contains. Even before EPIC took over Kaiser, they were running terminal emulators on Windows.
IMHO, it would be better for them to put together a ground up desktop distribution with exactly what they need, but that has user training costs and development costs.
It's preferable, from the corporate perspective, to have everything fail temporarily than to relinquish this level of workforce management.
If this is hard to imagine, just think of a Lyft driver from the perspective of Lyft Inc.
what you're saying is, if the less important service fails, of course the more important one will fail too.
With regard to this case, I don't know any specifics, but I can imagine tools require digital calibration, inventories not tracked outside digital systems, certain meds behind digital access control, and emergency response striained to the point where complicated non emergency procedures would be more risk than benefit.
And we were managing forests and waterways, not hospitals and human lives.
I'm happy nothing significant was hit over here in Poland; reading the main HN thread on the outage feels like reading war reports.
That's literally what we did to restart our forest logging machinery. Are human lives less critical than that?
Ad medicine - hence my question, I'd really like to know what's the blocker. So far it seems the blocker is bad IT management, regulation and liability, not impossibility to perform the treatment.
Again, from what I've seen, infusions are not just "throw it in an IV bag and wait".
Failed hardware is different, but hospitals likely have very few computers just 'lying around'. Especially the highly regulated machines, such as those which are attached to MRIs and the like.
CFR 21 Part 11 was the bane of my existence. Software that can be installed and configured in a matter of minutes? That's a six month project, at least. Sure, backups are great, but then you've got a significant process to get it back up and running.
These aren't early-2000 logging operations.
I see you'll never be convinced, but this is how modern operations work. Being a hospital (or other industry with heavy government regulations) make operations that much worse.
Hospitals also have limited resources in terms of IT staff. It's not a Azure army of operations staff that can rush out to every endpoint and click buttons.
When I was in helpdesk eons ago, I was "responsible" for roughly 300 - 400 endpoints, plus a handful of servers. As were all of the other helldesk techs. If something like this happened, there's simply not enough hands to go around as fast as everyone would like.
And due to CFR 21 Part 11, what you ask would be a non-starter.