Hacker News Reader

Comment by bregma | Hacker News Reader

I work for a company that ships safety-certified software. We, or often our customers, have discovered bugs in that software. We do not fix the bugs because one single small bugfix means re-certifying the entire software, a process that takes months of producing proof of matching the safety case plus many more months of updating and approving accompanying documentation and going through an audit. Everything has to be re-touched.

We just issue an updated defect list to go with the software and our customers needs to work around the bug. Known unfixed bugs are a fact of life in certified software. Update releases are not. Customers pay a premium for this because they, too, would have to go through the same pain and expense on their side.

View on HN

This reminds me of how NASA would never have a Shuttle in flight during the transition from December 31 to January 1, because they were unsure as to whether the shuttle's computers could handle the rollover correctly [1]. Sure, they could have updated the software to make sure that the rollover was handled correctly, but that would have required them to recertify the entire OS running the Shuttle, and it was easier to just plan missions such that the Shuttle was never flying on New Year's Eve.

[1]: https://usatoday30.usatoday.com/tech/science/space/2006-11-0...

We have discovered a critical bug in QNX 6 kernel in a networking scenario. There is no workaround, since the bug was in the core of their message passing infrastructure - a non-blocking by design kernel call, SendPulse(), sometimes blocks.

It took me 9 months talking to them about this problem until I managed to reproduce it on just two nodes and half a page of code, and record kernel logs that clearly showed a race condition.

We have received a patched kernel in a few days, and it worked like that for a while. This fix was merged into the official release after almost two years.

After that - only Linux, where we can see and fix stuff. No proprietary code and bureaucracy, no "fast, robust and reliable" operating systems.

This is exactly my point. Also, even if there is a workaround, more often than not the complexity of the mountain of workarounds just creates the next set of certified bugs.

It's a valid point, but the solution is not obvious. It's a trade-off in a big design space. (Of course with software it seems "trivial" to make sure the certification can be done quickly and cheaply. Just automate it! Unfortunately we're not there yet. :/ )

See also this comment: https://news.ycombinator.com/item?id=35589690

There is no safety-certified Linux. As far as I know there was no safety-certified QNX 6 either (QOS 1.0 was based on QNX 6.5 SP1, which is not the same as QNX 6 despite the numbers looking eerily similar).

With a safety-certified system, you do not receive a patch because it violates the safety certification. Of course, you can get a patch and use it but then you're responsible for safety-certifying the entire stack including the closed-source vendor code, and best of luck.

Without this patch QNX was simply not usable, since it had a random race condition in the kernel, which just broke everything.

They have incorporated the patch into some QNX 6.5 version, but we had to deliver the product long before they did this.

> Known unfixed bugs are a fact of life in certified software.

And all the other software.

The difference is that those bugs are certified in one and not the other. In many cases, bugs are known to the SW provider but not disclosed to customers unless they happen upon them. With certified SW, bugs are by default disclosed up front.

Reply on news.ycombinator.com