As well as, consider that liability should be part of the industry as it is in other industries.
Sadly the drive for profit speaks otherwise.
When I studied the Therac case, there was also a study in the "Killer Robot" - see here : http://www.onlineethics.org/cms/5122.aspx. Well worth a read to understand how software can become dangerous.
Sadly nowadays, a search for Killer Robot turns up all sorts of stories of drones being used to kill people. When I first looked up killer robot a number of years ago, the results were pretty much all about the ethics scenario and not real life people being killed. How society has moved on...
That is undoubtedly true. However, the commonest classes of bugs can be ruled out by languages with more expressive static verification tools—type systems, effect systems, and so on—leaving us with a stronger baseline from which to work.
This is true, however there are languages which allow for more programming errors than others.
I just mentioned C as one possible example.
The goal should be to have programming languages that reduce programming errors, not that make it easier to shot yourself.
Of course this is only a small step towards better software, as whole system design also plays a big role.
The software was written in assembly language that might require more attention for testing and good design. However the choice of language by itself is not listed as a primary cause in the report.
Wrong on that one. I do have a good background in compiler design.
> C is not just a portable macro assembler.
Given what it offers when compared with safer systems programming languages, it is one according to my own definition. :)
> The defect was as follows: a one-byte counter in a testing routine frequently overflowed; if an operator provided manual input to the machine at the precise moment that this counter overflowed, the interlock would fail.[3]
Can you describe how exactly a language other than C would have helped with this?
Unless the programmer has explicitly disabled them, that is.
Crashing is better that corrupt data.
http://en.wikibooks.org/wiki/Ada_Programming/Pragmas/Suppres...
Modula-2, a possible documentation link
http://www.excelsior-usa.com/doc/xds/isom203.html#15
Just to cite two examples.
* Raise an exception? If so, it's up to the programmer to treat it correctly. There are various ways to detect overflows outside Ada; it's still up to the programmer to treat them correctly. Runtime detection of the overflow (in a hopelessly untested firmware of the Therac-25!) would have brought the matter no closer to resolution.
* Abort execution? If so, is the behaviour of the program defined if execution is aborted? Because undefined behaviour from a control system is quite worse.
SPARK Ada allows the use of source-code annotations to denote information flow in a program, and can vastly reduce the chances of these sorts of problems (at the expense of more programmer time).
Standard Ada compilers can do a reasonable job of detecting some problems at compile-time, but SPARK seems to be the favoured method for safety-critical stuff.
It definitely is! However, it still assumes that the exception is safely handled. The flaw in Therac 25's development process was that testing did not reveal this bug. Consequently, the error recovery process could have itself been incorrect and remain uncaught. It's tempting to think that, if error recovery is similar each time, testing it once is enough, but error recovery is as susceptible to being impeded by race conditions as any other part of the code. No uncertainty is removed this way.
It's also worth noting that a value of 0 (to which the counter overflowed) was actually a legitimate value in the Therac-25 program.
If you are going to say we shouldn't use C because of overflow handling, at least suggest a language that actually does it. Otherwise you haven't really made your case.
Chances are, when you travel by train or take a plane, the control system was developed in Ada.
When human lives are at risk, C is becoming less and less an option.
And when the company still goes C, many countries have certifications in place like MISRA, that make C look like Ada with C syntax.
It's less the language than the ecosystem and the infrastructure. It's eminently possible to develop 'C' that's exhaustively tested - I've done it. I am not sure that is true of C++ or Java.
If the crashed program is controlling a device that is directing radiation at a person's body, then crashing is likely to be at least as undesirable as data corruption.
I understand your point about programming language choice, and your comments elsewhere about the undesirability of the C language. But the root causes of the Therac-25 malfunction wasn't technical, they were organisational. Specifically poor code review and code reuse practices. Changing the programming language wouldn't fix them.
In embedded control software of medical systems that are directly affecting the lives of the patients involved crashing is definitely not an option and it is not necessarily better than corrupt data.
What is an option is to very carefully model the permitted states of the machine and then to exhaustively test those states and their transitions to make sure there are no undefined states the machine can be in.
One way to do this is to have a watchdog process (or even an entirely different system) that monitors the state of the machine and that initiates a safe shutdown if any departure of the defined operation is detected.
In general, the more DRY the code, the more mind-space for review / testing.
This can also be said of choosing simpler design, organizing the code in a better manner, or could have been avoided through more comprehensive testing -- all of which were pointed out by the official report.
The fact that the programmers might have had less stuff to worry about and, therefore, saved their concentration for more critical parts of the code, is hardly direct evidence that the bug could not have been introduced. It can equally well be argued that a higher-level language would have attracted programmers with less experience in mission-critical code, who would have failed to grasp even the possibility of bugs more subtle than this one.
It's something that would be considered firmware today, written in assembler.
Trying to be DRY in assembly is a really great way of introducing subtle bugs.
If you look at the reports of serious failures like this once (and other famous ones), none of them are related to the use of C.