I remember hearing about this case in college. My professor's point was don't trust software alone. Always include hardware safe guards when it comes to safety.
Design the security/safety system up front, for the task of the system being designed, for all potential failure modes - and then implement the first, and test the hell out of it, and document it thoroughly.
Unfortunately, this tends to get shot down, until it is needed, and then it gets bandaid-patched in place, because usually at the point it is needed, a whole refactor of the code base is usually called for, and we can't have that, so "just fix it" becomes the call of duty.
And it never works out properly. Things get missed, security is effectively out the window, until something happens, with the refrain of "why did this occur" and of course nobody wants to take the blame for "I told you so".
Unfortunately, even implementing such a security system first will only get you so far, and it is still possible that something might be missed, and potentially be the cause of a safety critical issue. But hopefully, most if not all of those extreme events can be caught first in the system, by some part of it, and what's left will be relatively minor.
What does that actually mean? Maybe in the Therac-25 case the maximum dosage could somehow have been limited by a hardware dial in addition to software. But what form would hardware safeguards take for the millions of decisions software makes every second on a starting rocket? And considering hardware has far higher failure rates than no-moving-parts software, wouldn't any system that lets hardware dominate software decrease safety?
From Wikipedia:
"One, when the operator incorrectly selected X-ray mode before quickly changing to electron mode, which allowed the electron beam to be set for X-ray mode without the X-ray target being in place. A second fault allowed the electron beam to activate during field-light mode, during which no beam scanner was active or target was in place.
Previous models had hardware interlocks to prevent such faults, but the Therac-25 had removed them, depending instead on software checks for safety."
A dose integrator would be a detector + an integrator independent of the therapeutic delivery software. If it goes over limit, it shuts the system down.