You would think that equipment valued at $12m would have safety features to prevent damage from rogue commands. Interesting.
If the hardware can be destructive under the control of software then you must have hardware interlocks, not just software safety features.
Also on that list (from memory).
* AT&T's switch cascade failure * Mars Climate Orbiter * Ariane 5
He also took us through some bugs in code he'd written when he was doing massive telecomm's systems.
I came away with a few lessons.
* If it can't fail, test it twice. * If it's designed so it can't possibly fail, test it three times. * All code is buggy.
When faced with this kind of requirements, I always lift my eyes right above my screen, where it is written:
> The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair.
And for emphasis, next to it is a poster of a nuclear island cut-out.
Sure enough, we once had a motor controller break where the problem was interconnecting wiring...
For those who haven't read it, http://sunnyday.mit.edu/papers/therac.pdf
1. Those who read about Therac-25 and decide never to work on any systems where bugs could threaten lives (your chosen course, and mine).
2. Those who read about Therac-25, get frightened, and work very carefully on such systems.
3. Those who read about Therac-25 and think, "Those idiots! Good thing I'd never make a mistake like that."
The prevalence of (3) in all professions and walks of life is kind of frightening.
If you dig into the details of why its programmers and their managers killed people, it's very easy to say "I'd never be that irresponsible", but still screw up with much more subtle bugs.
:)
But it can be stressful to write software that has a very direct impact on peoples lifes. While I was rolling out an updated version of an inhouse application a serious bug was discovered. The application was used as part of chemotherapy follow-up/planning and on some occasions it gave the wrong answer for a test. I discovered that the bug was old and that wrong answers had been given for a long period. Personally I was relieved that I had not caused the bug, but from a medical perspective it would have been better if the bug was new and had only affected very few patients.
My takeaway from that experience is that care should be taken when writing critical software, but also that we will make mistakes and it is very important to always focus on how we can make fewer mistakes instead of pointing fingers.
Even if obligation to publish source code were to harm companies (which might even not be the case) the benefit it would bring to the consumers and whole ecosystem is more important.
Companies don't need special protection. The most important advantage of a company is that it can die without killing all people who work there and destroying all its assets. This allows companies to take risks no sane person would take if he were to be responsible with all his money (both savings and potential debt).
He's no doubt learned some important lessons, but I can't find destroying $12 million in equipment in any way funny. [c0deporn has updated his original to the more accurate and appropriate "interesting".]
If someone broke a $12m machine, surely their career would be over.
Who would you rather have working on your $12m machine, someone with no real experience with that sort of hardware or someone with plenty of experience and some painfully learned lessons about exactly how careful you have to be when working with brittle $12m machines.
There are two philosophies of justice: retributive and utilitarian. Retributive justice is about punishing the guilty. If you destroy a $12 million machine, you deserve to have $12 million taken away from you. Most people don't have $12 million, so the retributive boss will take away as much as he can: fire you and file suit if he can and badmouth you to other companies so you never work again.
Utilitarian justice is about preventing future harm. It's not about punishing the guy who pushed the button on destroying $12 million of hardware, it's about preventing future hardware losses. You fire the guy if you think he's likely to destroy more hardware in the future, because maybe he was negligent and careless or even malicious. You keep the guy if he was just in the wrong place at the wrong time, and any other competent employee would have made the same call and inflicted the same loss.
So while the "$12 million in training" line is a great ice-breaker for making the poor guy feel better, if someone broke a $1 million machine, then a $2 million machine (after all, he has $1 million of training now), then a $3 million machine, then a $6 million machine, would you say, "Great, he's got $12 million of training! Let's put him in charge of our $12 million machine!"?
Another Lockheed test pilot wrecked a F-16E at an airshow practice and ejected with injuries now flies F-35s.