Your pacemaker should be running open source software
theregister.com
theregister.com
Open source is orthogonal to quality at that level. It may be a desirable, additional quality to ensure openness and prevent obsolescence, but is no substitute for actual verification efforts by trained professionals specialized in the field actively tasked with analysis.
All we have to do is look to critical software like OpenSSL and the Heartbleed vulnerability to see that “many eyes” is insufficient to achieve the necessary quality in even widely used, critical software. There is no guarantee of “any eyes”, and no guarantee they are competent or have sufficient domain knowledge (think of how long it takes to get up to speed on a new software project) to meaningfully assess the systems.
We must have known, active, unbiased, competent third party verification for critical software just like we demand of all other components and fields. And yes, I am aware of the certification process for Class 3 medical device software, and no it is demonstrably insufficient for our current use cases as seen time and time again.
Running reproducible builds of open-source code certified by unbiased, competent third party experts, so that the semantics and runtime behavior of the code can be continuously evaluated by multiple unbiased, competent third party experts, since the business definition of "third party" can change over time.
Where binary blobs are needed, the blobs can use reproducible builds and the certified source code can be escrowed so that the binary blobs can be reproducibly created by a third party.
That would be shockingly stupid. There is exactly zero plausible deniability if they did that. That is “slam-dunk go directly to jail and get fined billions” stupid. There is exactly zero reason to worry about that at all at the device manufacturer level (component suppliers might lie which transitively affects the final product, but that should be handled when identifying the target of evaluation). If they want to be evil, there are plenty of ways that do not require them to put their evil in legally binding writing.
If you are concerned about a hack or accidental deployment, that is a real concern, but image integrity, secure boot, and secure update are really part of the certified process/product.
Only that certified source code and software binaries can be continuously evaluated after production shipment, as original developers and certifiers move onto other projects. See the "Right to Repair" movement.
Over timeframes of multiple years, we learn more about stacks of hardware, software and threats, as can be observed in the constant stream of CVEs for CPU microcode, firmware, operating systems and application code.
Note that failure to remediate known defects in components or covering it up is a very serious offense in basically every regulatory regime. That is basically “the thing” that gets you in trouble. Those standards should probably be made stricter, but that is for another time. Just getting software to “industry standard” is a major win.
What do you think of the EU CRA requirements for critical software?
Liability should lay with the provider of the final product to use components fit for purpose, not with the component providers to make their component fit for every possible purpose. This is how it works on every other industry. Airplane manufacturers must use airplane-grade screws; they do not get to demand that every screw in Home Depot be fit for airplanes.
If a company wants to use a FOSS library then they must make sure it fits their standards, or they must fund development and verification to make it meet their required standards. They do not get to unilaterally demand the library authors work harder to line their own pockets.
"Quality" has many meanings. The article gives the example of Sandler, who was unable to read data from her pacemaker, because the correctly functioning, quality, high-assurance software was kept secret from her, and only the manufacturer could decipher it.
Regardless, quality is a red herring - the real issue is sovereignty over our own bodies. Which is impossible if we have devices with secret functionality embedded in us. We have a right to know how they work, the same as we have a right to know what medicine doctors administer to us.
For that matter, even if you received those only a vanishingly small number of individuals would even be qualified to evaluate that information.
No, what we as a society do is rely on a certification, verification, and validation process done by third party experts that guarantees the medicine fits a specification and is fit for purpose. This does not always get it right, but it is pretty good and works in literally every other industry.
Competent unbiased third party verification is sufficient for safety-critical deployments. Openness is neither sufficient, nor necessary. A closed device with no safety defects is usable, a open device with safety defects is unacceptable.
Openness may constitute other goods, such as freedom, agency, and such, but it does not provide safety, efficacy, and fitness for purpose which are the most important elements of medical devices (i.e it is orthogonal to effective operation).
Individual humans and their medical advisors choose what to ingest, from a marketplace of substances on a spectrum of transparency in content and consequences.
Openness is not and has not been sufficient, necessary, or even expected for useful medicine or basically anything else.
Openness is orthogonal to efficacy and safety. It just looks correlated in software because software is totally unverified as a general rule, so “unverified, but ability to verify” is much better safety and efficacy wise than “unverified, no ability to verify”. But “unverified” of any form is worthless garbage and unfit for critical purposes. What you need is “verified” at a bare minimum, anything less is unacceptable.
Yes, most countries currently rely on centralized regulation and testing.
Hopefully, decentralized verification will continue to improve with science education and sensor technology, as humans learn that "supply chain security" applies to all inputs to human bodies.
I was not aware of this. Predictably, this causes problems [1], just as the closed nature of the pacemaker in the article caused problems. Something which you ignored, and continued claiming, counterfactually, that closed-source doesn't hamper patient safety.
[1] https://theconversation.com/whats-in-your-medicine-may-surpr...
As to the formulation of your medicine, your average person is unqualified to evaluate it for efficacy. They would generally rely on a competent trusted third-party (their doctor) to evaluate it for them. The evaluator needs openness, but general openness is not really required even if it may be desirable for other reasons.
A common benefit to openness is not restricting it to the designated evaluator allowing more freeform evaluation. This can have benefits in medicine if you are able to navigate medical analysis and do your own research. However, this still has limits as even a highly skilled layperson is unlikely to understand the specific pharmacokinetics so general openness at that level of detail is unlikely to be particularly useful. You will almost certainly rely on a competent, trusted third party to confirm and verify conformance for you.
To sum up, what matters is verification by trusted, competent parties. Verification demands openness to the evaluator, but does not need to extend further than that to see significant benefits. There may be additional benefits to openness, but efficacy and safety are more effectively achieved in other well-known, tried and true ways.
? Don't you get close enough by reading the relevant FDA approval documents, the list of patents of the active ingredients of the medicine you ingest? Nobody needs to be a biologist but many independent experts can verify what's in a pill. When it comes to software in our bodies, it's supposed to be secret?
I’m thinking specifically about “artificial pancreas” aka “looping” insulin pumps, which automatically adjust how much insulin they give you based on a continuous glucose monitor (CGM) providing blood sugar readings.
My kids use these devices, and I’ve watched their devices dose insulin (a potentially lethal drug) erratically when the CGM data itself occasionally goes haywire.
I wish I could review the algorithm for myself. There’s no excuse for a piece of software which is automatically injecting drugs into my children to be a black box.
FDA > Medical Devices > Cybersecurity: https://www.fda.gov/medical-devices/digital-health-center-ex...
"FDA Fact Sheet: THE FDA'S ROLE IN MEDICAL DEVICE CYBERSECURITY: Dispelling Myths and Understanding" https://www.fda.gov/media/123052/download?attachment
oss-fuzz: https://github.com/google/oss-fuzz :
> We support the libFuzzer, AFL++, and Honggfuzz fuzzing engines in combination with Sanitizers, as well as ClusterFuzz, a distributed fuzzer execution environment and reporting tool.
> Currently, OSS-Fuzz supports C/C++, Rust, Go, Python, Java/JVM, and JavaScript code. Other languages supported by LLVM may work too. OSS-Fuzz supports fuzzing x86_64 and i386 builds.
Thing is, get the corner cases wrong and you're shocking the hell out of people every 10 seconds or so. Or worse.