Hacker News Reader

Comment by davidw | Hacker News Reader

> you should treat software quality like some 19 year old's life depends on it, because it might.

You should treat it like that if someone's life does depend on it and you have the resources to develop accordingly.

If you're developing something like, say, bingo software, you're probably better off devoting time to improving the product or marketing it, rather than working on it being 100% bug free.

It's all a tradeoff - time spent on eliminating every last little bug is time not spent on adding features or making it faster or marketing it or whatever.

Edit somewhat less clear-cut cases might be bits of software that you release publicly, and subsequently get used for life-critical systems. However, in that case, the onus is on those adapting the code for use in that environment to provide the testing/review/etc... rather than blaming the upstream developer.

View on HN

That approach works if you know exactly what applications your code will ever be used for. If you are writing a library or a compiler or anything that will potentially be reused by unknown 3rd parties, then you can't be sure just how critically it will be put to use. There is no certification to distinguish software that lives can depend on.

That doesn't mean it's your fault if someone uses your free XML parser in an amusement park ride and your bug makes it fly off the rails. But I still wouldn't feel very good about it and would like to do everything I can to avoid it.

Also, when it comes time for you to write life or death code, it would be good if you already knew how to meet the required quality standard.

You seem to have written this after my edit, which addresses that point. I'd feel bad too, but I'd feel bad if I had to hire a team of coders to review every line of my open source projects and document any change:

> "At the on-board shuttle group, about one-third of the process of writing software happens before anyone writes a line of code. NASA and the Lockheed Martin group agree in the most minute detail about everything the new code is supposed to do -- and they commit that understanding to paper, with the kind of specificity and precision usually found in blueprints. Nothing in the specs is changed without agreement and understanding from both sides. And no coder changes a single line of code without specs carefully outlining the change. Take the upgrade of the software to permit the shuttle to navigate with Global Positioning Satellites, a change that involves just 1.5% of the program, or 6,366 lines of code. The specs for that one change run 2,500 pages, a volume thicker than a phone book. The specs for the current program fill 30 volumes and run 40,000 pages."

From: http://www.fastcompany.com/node/28121/print

If there's no certification for ready-made life critical components, then that means the burden of reviewing, checking and verifying everything in the system is on whoever wants to use it in a life-critical environment.

There is no certification to distinguish software that lives can depend on.

Actually, there is. Google up "safety-critical software". And maybe "trusted software".

I work in military avionics; every dang line of code in the product, including any libraries we use, is vetted to death. If I were to try to just download a library off the internet and include it in the flight control software, I (A) wouldn't get away with it (B) would probably lose my job and (C) would confuse the hell out of my coworkers who all know I know better than that.

So don't worry someone will include your hastily-developed XML parser in safety-critical software without your knowledge. They won't unless you're willing to prove you've certified it to the level they require. And I promise you, that's not an exercise you'll forget having gone through. ;)

But then, how on earth did they manage to get this software certified if it did something as DailyWTF-worthy as using floating point for a clock?

Yeah. You should see some of the other stuff that makes it into production code.

I said there was a process. I didn't say the result was good software. ;)

Collection of bits are put to truly horrible uses in the name of efficiency.

Well I have to give them that this is kind of a curious bug. I mean, you have to wait over 4 days until it triggers. Probably, this just worked whenever it was booted and tested, and booted and certified, because probably the certification did not involve ignoring the thing for four full days.

According to a book I read about game testing it's common for commercial games to be run through a test of simply leaving the game on for hours or days to see if there are bugs that only show up in this way.

According to http://www.cs.unc.edu/~dm/UNC/COMP205/LECTURES/ERROR/lec23/n..., the system stored the time in integers, but it was converted to floating point when doing the conversions. This conversion contained a small error that accumulated over time.

Keep in mind that things that seem very WTF to you, might seem more plausible when given more details about the subject.

In part because the user manual included instructions to regularly reboot before precision became a problem. The users did not, because they didn't want to risk being in the middle of a reboot when a target went overhead.

My mistake, but this kind of methodology seems to be specific to a few industries like aviation and space, while being conspicuously absent from e.g. medicine. Ideally, the software industry would have its own life-critical standard that was applied across all domains, not that I'm suggesting such a standard is necessarily feasible right now.

Until we have such a universal standard, it is entirely possible for a bug in your free library to indirectly kill someone, in the course of everyday best practices.

I have an incredibly hard time believing the medical industry does not have stringent standards in place for safety-critical software. Do you have anything to demonstrate this, and what country are we talking about here? (To remind me not to ever see a doctor there... :) )

http://www.nytimes.com/2010/01/24/health/24radiation.html

In the US, medical devices are regulated by the FDA. When it comes to using 3rd party software, the company manufacturing the device has the responsibility to ensure any 3rd party components function correctly. Its unrealistic to impose regulations on every piece of software that is written "just in case". It is far more practical to put the responsibility on the company doing the integration and selling the device.

Here is the guidance from the FDA on off-the-shelf software: http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidanc...

This did meet the requirements for what it would be used for.

Intended to be based in Germany in the 60s facing the Russians it wouldn't be powered up for 600 hours, because in 60 hours you would have been overrun.

There's an old Tom Lehrer song, "So Long, Mom," about just that: http://www.youtube.com/watch?v=pklr0UD9eSo

The British vulcan bombers of the same era had the range to reach Moscow - but not the range to return to the UK.

I don't think that's true - Vulcans hold the record for the longest bombing mission even today (during the Falklands war). The secret is air-to-air refuelling...

might be hard to do that somewhere over poland if ww3 just started

USAF wargames discovered against F22s the most effective strategy to use would be tanker denial. Tho' with 4th generation fighters you would still need to outnumber them 6:1 (!)

The astronomer Freeman Dyson was part of the operational analysis team in WWII that worked out it made most sense to only search for and attack refuelling and resupply U-Boats (milkcows)

The Vulcan flight gear famously included hiking boots so the crews could walk to Turkey after completing their mission

ultimately, it is the responsibility of the integrator to test their systems, including the software. And they have the source code for everything. Its like Apple X 100.

Didn't see this last night. FWIW I agree with you, but that was not my recollection of the lecture as presented.

Reply on news.ycombinator.com