Vulnerabilities found in GE anesthesia machines
zdnet.com
zdnet.com
GE Healthcare Security Alert issued July 8, 2019 https://www.gehealthcare.com/support/security-information
DHS Security Alert issued July 9, 2019 https://www.us-cert.gov/ics/advisories/icsma-19-190-01
However, for what it's worth, these sorts of safety systems (and similarly SAR monitors in MRI, etc.) tend to be well validated as part of the overall hazard and risk analysis, and people spend the time here, they tend to have interlocks and other often redundant safety subsystems that work.
I guess what I'm saying is that I wouldn't expect bugs to trip you up in the primary-yet-dangerous function, as this is where the obvious problem areas are.
This article describes how the attack surfaces on medical devices aren't good. This is definitely true. Especially with older designs that have been updated over the years but were designed with no network or private network in mind.
> The Therac-25 software errors that cause radiation overexposures can be reduced down to interface errors. The first of these errors involved the entering of treatment data by the machine operator. Once an operator enters treatment information at the terminal outside of treatment room, the magnets used to filter and control radiation levels are set. There are several magnets, and the process takes about 8 seconds. If the operator makes a very, very quick change of the treatment information, within 1 second, the change is registered. Or, if the operator is rather slow about it, takes more than 8 seconds, the change is also registered. However, if the change occurs within the eight seconds it takes to set the magnets, the change is not detected and the magnets continue to be set up improperly, and thus the level of radiation is set up improperly.
> The last of the accidents occurred at the Yakima Valley Memorial Hospital. On January 17, 1987 an operator placed a patient on the turntable in the field-light position for small position verification doses. After attempting to administer the treatment dose, the machine shut down with a quick malfunction message and a treatment pause. The operator pushed the "P" button, and the machine paused again. The machine indicated that the patient had received his prescribed 7 rad of treatment. The patient, however, complained of a "burning sensation" and died three months later from complications related to the overdose (Leveson and Turner, 1993, p. 33) .
http://users.csc.calpoly.edu/~jdalbey/SWE/Papers/THERAC25.ht...
And partially because the industry learned from Therac-25 (and other issues), collectively it got much better at avoiding this sort of failure mode.
I’m not saying it’s perfect, but it is not a high risk scenario for the poster I responded to.
It does not mean that medical device SW is perfect or bug-free, but it means that the manufacturer should demonstrate some level of risk management, verification and validation to the regulatory bodies before being allowed to sell a new product. It is not perfect but you should not get sick because someone pushed some untested bug fix on Friday night and you have an exam on Monday morning on an untested SW release, the process would not allow it.
What IEC62304 adds to the mix is specific guidance on your SDLC process. If I recall correctly you are still not required to audit to in the US but new projects should follow it or demonstrate why they are not.
I encourage concerned customers to ask for their dose report. IMO it's mildly interesting on its own. It's more interesting if your physician or surgeon prescribes follow up tracking to see if tumors return/grow. In that case you have repeated scans over a period of many months to track lesions/tumors and the accumulated dose at the same body location becomes worth more concern.
The prescribing physician should balance the harm of dose with the diagnostic concern/risk being evaluated, but there's nothing wrong with advocating for yourself. In general I think the cases where you have repeated scans it's because something terribly serious has been diagnosed that far outweighs the impact of the dose.
Also if you're concerned about dose you might be able to use CT scanner dose features as a tool when evaluating different outpatient imaging options. Feel free to debate this with your physician and not necessarily go to the one that they get a kickback from.
[1] For a stunning counter-example to my claim here, you can look at the case where the CT tech completed MANY repeated cranial CT scans of a toddler (scanner was mfd by Picker IIRC) -- https://www.diagnosticimaging.com/articles/california-blames...
It's kind of like how I've seen you comment multiple times about how software shouldn't be written in C. You even said it during the DNS fuzzing thread earlier today.
You saying that is just as pointless. You would be very surprised at how much new stuff is written in C!
The opposite is the case for network connectivity: the trend is demonstrably and decisively towards increased connectivity, and "air-gapping" is not taken seriously in the industry. There is no indication of that changing.
The comparison you're making is invalid. I'm not trying to score points; I'm observing that the litany of "this should be air-gapped" complaints isn't productive, because things are not going to be air-gapped.
(My full-time job is assessing software security and my background is in C software vulnerability research, so it's less likely that I'd be "surprised" by a new C package than that I'd warn my clients to avoid it, and flunk it in vendorsec assessments.)
So, how would one be productive about it?
Your post seems to say, and please correct me if I am wrong, that because things aren't currently happening and there are some barriers to making it happen, we should all give up on pushing for it? To me, that seems like a rather fatalist attitude to have. Do we apply this line of thinking to everything? Or just air-gapping?
I am not, by the way, happy about this, but I've also spent essentially a lifetime (minus maybe 13-14 years at the beginning) having all the surprise on this particular issue knocked out of me.
I guess I have a little bit of surprise left in me on this issue.
It's good to raise the bar from drive-by internet strangers to people and organizations willing to take mild physical risks, but it's not a panacea.
I don't quite know how my comment led you to believe that I think airgapping is a pancea which solves all the existing computer woes in the world.
I certainly don't think, and didn't intend to imply, that airgapping removes the risk from contractors or a reason to not keep up on patches. Again, I'm confused how you reached that conclusion based on my comment.
"And even if there weren't, if you've been thinking of the airgap as a "solution" and not keeping up with patches, ..."
Nobody here is going to say airgap and done, but out in the wild they will certainly deprioritize updates on airgapped equipment.
Perhaps the 'you' was intended to be generalized. I interpreted as directed at me, since the entirety of the comment is directed at me. Maybe I'm mistaken.
The joys of trying to have meaningful conversations over text.
One might start by observing that there are reasons these things aren't air-gapped. A person could go on to note which of these reasons continue to apply and are considered compelling by those who make the relevant decisions, and thus that air-gapping is likely to persist.
So maybe the system could be connected to the network except for when it is treating a patient. A big red slider with 'On Air/Isolated' could be present which would lock out the patient treating options as soon as the machine is networked. Now, this would still leave some gaps: an update could be faulty, a malicious actor could install something that triggers only after a while or when the machine is used to treat a patient. But it would remove a lot of the concern I have with equipment like this being online all the time.
The risks of compromise of an anesthesia machine are scary. It’s also scary that without EMR integration, a dosage might be misreported or an allergy missed.
It’s possible to securely segment a network to defend against these types of risks. The bigger problem here is that the professional practice of IT is such a garbage fire, it’s assumed that the LAN is compromised and airgapping is the responsible choice.
When will we get an ANSI standard for one of these new fangled languages?
Go is captive to Google, and Rust is more akin to C++ than C -- neither have an actual standard with competing compilers.
Until that changes, I think plenty of new software will be written in C, especially freestanding and embedded software.
C++ doesn't have an actual standard? Are you sure about that?
Yes both C and C++ have standards, and I consider that to be a good thing, though apparently tptacek says I shouldn't care.
C is on the way out. It's not gone, but it's going. There will not be a revitalization of the language. It'll be good to know, because some things (OS kernels and device drivers) will continue to be written in C for the foreseeable future. But that's a very small niche relative to the industry as a whole.
Nobody cares, and nobody should care, about ANSI standards.
There are, no doubt, a number of niche systems that require specific toolchains. There are, in our fallen world, systems that require Ada or even particular variants of C. If you want to tell me that aviation flight control systems are such a niche, I will believe you --- I've never had to assess one.
But it is not the case that industrial computing or medical device software are locked into memory-unsafe languages due to industry-wide certificational requirements; in fact, that's something I know not to be true from specific experience. And virtually all of the embedded systems I've had to assess over the years would have benefited, commercially, from a memory-safe implementation language.
This is a major barrier to entry for new programming languages in these markets. Note that I am not saying that improved memory safety wouldn't be useful in embedded software. But the market is so conservative in parts that real uptake is at least a decade or two away.
Since many people are interested in using Rust for such applications, there are efforts underway to stabilize the compiler and do the necessary paperwork so that not every company needs to do it themselves.
What test suite are you talking about? I am really curious because this would completely upset the whole industry if what you ssid was true for the toolchain.
I've been a contractor for a medical devices company for one year, and the process where really lightyears away from what you are describing. Nothing even forces you to write and run unit tests…
Yes, that's scary.
The next company I worked at was industry, not medical, and this one did it right. So while there's black sheep out there, not all of them are.
Side note: whether a device can fall back to becoming inert on a failure depends on the type of device and the specific risks involved with that failure mode. It may be that stopping to do anything is the wrong thing to do, e.g. in a blood pump.
Borrow checking is unrelated to whether you are on an embedded device or not.
Guarding against mutable aliasing with respect to hardware state could be enormously valuable.
I don't quite understand what you are saying in your second paragraph.
let mut x: Result<i32,f32> = Ok(1);
let y = x.as_ref().unwrap();
x = Err(1.0);
println!("{}", y); // unsafe cast of float to int
The borrow checker is great for all sorts of things, like ensuring that you only have a single mutable reference or an arbitrary number of immutable references to objects (to prevent corruption) and for the pseudo-threaded model interrupts entail. Memory safety matters in embedded systems, too. You can encode state cleanly with ADT enums [2] where the language can statically detect invalid state transitions.
It's also a much more expressive language, providing you with great abstractions with no added cost. The ergonomics are pretty great compared to dealing with C once your peripherals / register accesses are wrapped, as often are already. For instance, check out the stm32 package [1].
Being able to rely on the compiler more means you get correct code faster, and debugging on embedded systems can be absolute hell. Anything that helps me avoid it is a massive win in my books.
[1] https://github.com/stm32-rs/stm32-rs
[2] https://hoverbear.org/2016/10/12/rust-state-machine-pattern/
Specifically speaking to medical devices, there is a bit of systemic bias. Early hospital networks were isolated, and this led to some naive thinking about security (e.g. DICOM protocol has no auth provisions). Many early machines weren't connected to anything at all, then they were but only "safe" networks so not much care taken. Development lifecycles on complicated devices is incremental over sometimes even decades. But now hospitals etc. have lots of pressure to connect all the things, for all kinds of good and productive reasons, and air-gapping just isn't a realistic solution.
It's not an easy problem to solve, especially quickly.
You are starting to see some partial hardening of devices, which will probably be the practical solution.
So even if you can't normally access certain records or data, if you claim it is an emergency you suddenly can!
There are sometimes calibration modes or research keys that let you do odd things to, say, imaging machines. But there are also processes in place to disallow clinical use.
Because from what I can tell, it would require formal proof that the connected parts are safe (logic bugs can also be exploited, so just using a safe language is insufficient).
If you can't get manufacturers to do the cheap, easy air-gapping, how will you get them to do the very expensive and time-consuming formal verification, that only a tiny, tiny fraction of developers is proficient in? Intractable doesn't even begin to describe it!
Nobody is going to air-gap anything and I think there is ample evidence over the last 20 years that the trend goes in the opposite direction. Given the reality of the world we actually live in, productive conversations about securing embedded and IoT systems can't involve air gaps.
Slightly later
Here's another, simpler way to say the same thing: you might just as productively argue that these products shouldn't be built at all.
I am generally opposed to liabilities for software defects, but system integration seems a more reasonable place to apply rules.
The second reason why stuff is not airgapped is that it almost certainly connects to the hospital's patient records system. They have to keep a record of everything that happened until all potential lawsuits time out. Just in case there are complications from the surgery, they need all the records from the anesthesia machine uploaded after every use. So the anesthesia machine has to be on the same network as the patient records system - and so does every other medical device in the entire hospital. That network should be completely isolated from outside, but to do so, you have to airgap the network, not just one machine. Yes, it should be done, but that's harder to do, and harder to maintain.
For that matter, I've wondered, when you visit someone in the hospital, if you plug your laptop into the ethernet jack in their room and start looking around, what do you see?
What now?
Like every non-networked digital camera I've ever owned: put some fwupdate.bin file on a SD-card, plug it in and run some procedure on the device.
If a DVD are used, the techs can locally burn an .iso image onto a blank as an alternative to getting it in the mail.
Logs can also be gathered from the machine on removable media.
Medical records can be available via some non-airgapped laptop. Maybe something can be integrated into the machine, but the control of actual dangerous parameters for therapy or anaesthesia should be air-gapped. Manual entry only.
Operators can still be duped into entering malicious values manually, but at least there is a fighting chance for some oversight.
>[...] the techs can locally burn an .iso image onto a blank [...]
>Operators can still be duped into entering malicious values manually, but at least there is a fighting chance for some oversight.
or something like stuxnet.
Unless the IT team consists of Santa’s elves, that’s just not going to happen.
That sounds like an easy solution to me. God forbid people need to learn something new for their job.
More importantly, these records often have clinical significance.
By eliminating updates. Why isn't software that runs on such machines proven correct? Yes, it's expensive, but medical equipment commands a premium price since it's supposed to meet stringent standards regardless. There's no reason to not up the standards for the software they run.
First, what are you going to prove? That it does what it's supposed to? No, because you have to have some formal definition of "what it's supposed to", and that can actually be wrong (see MCAS for an example - the software worked, the problem is that the spec was insane).
Second, as somebody (Donald Knuth?) said, it's amazing how many bugs there can be in proven-correct (or formally verified) software. Does your proof prove against all possible bugs? No. Against all possible security issues? Still no.
Third, even with those limitations, proven correct software is much harder (slower, and more expensive) to write, because the proof is really long. You want medical devices to have proven correct software? That would be nice. But we'd have maybe 2% of the medical devices that we have now, just because the proofs are so hard. That would be less nice. In particular, more people would die in that scenario from the absence of medical devices, than die in the existing scenario from bugs in them. That's less than helpful.
and that is doesn't do what is not supposed to do.
As for your second point, if you could provide some background reading for that, I would appreciate it. I know of Knuth jokingly stating "Beware of bugs in the above code; I have only proved it correct, not tried it", but if he or someone else wrote a more serious paper on the occurrence of software errors in proved software, I would appreciate a link. A proof should prove against all software errors--if it doesn't I'm eager to learn more.
As for your third point: software errors are a solved problem, but as you noted, it's a hard (expensive) solution. The solutions also currently may limit functionality, so it's unfortunately not applicable everywhere, but some classes of software should warrant it.
In my view medical devices that can directly interact with a patient's physiology is one of them.
So the proof that software is secure may be invalidated by new attacks. And this is (part of) why "proof against all software errors" is... let us say overly optimistic.
Even when it's not security issues... do you have a list of all possible software errors, so that you can prove that none of them occur? I don't. I'm not sure that anyone does. I suspect that there's always some kind of error that isn't covered.
It's not about security. It's about the software functioning correctly so no functional updates are necessary, diminishing the need for remote updates and all the security problems that bring along.
Even if remote access is needed to conveniently store various data in a different location there is no need to run everything on the same level. Smartphones are mass-produced computers that separate concerns with having a baseband and a main cpu. There is no reason a medical device could not have such a separation. One OS to keep the device working correctly under all circumstances, one OS to slap networking and interfacing on.
I don't expect the medical technology industry to change quickly, but as things are they are adopting bad practices of software development and moving in the wrong direction.
"If problems pop up, we'll update it later remotely" should not be an acceptable mindset for some classes of computer equipment.
There should be zero way to do anything remotely to these machines without someone at the local console pressing a button.
In fact, if I had to design such a system, this is how I would do it--network server, GUI server, and anesthesia administration would be 3 very separate "instances" with hard communcation links in-between. The GUI->apparatus would be a bidirectional link (probably CAN for reliability) and the GUI->network server would be a separate, unidirectional link only from the GUI to the network server (probably CAN again).
It would simply be far too difficult to verify the reliability of a system that had the network, the console UI and the anesthesia administration apparatus commingled.
A data diode would be appropriate, though.
Anaesthesia machine control should be air-gapped.
Unfortunately, in the modern world of electronic medical records, we do need access to a stream of measurement data. Historically this has been obtained using one-way serial port data streams, which is probably safer than having a network stack accessible to the web.
The users (anaesthesiologists and hospital administrators) don’t understand these problems. (I’m an anaesthesiologist)
Would be perfectly reasonable to pass monitoring data from the control unit via a tx only channel to a networked monitoring device. For instance using a digital opto-coupler to galvanically isolate the two units.
Hardware tx only would only make sense if you avoid side channels as well.