Hacker News Reader

Comment by Aurornis | Hacker News Reader

I guess I don’t find that argument very compelling. If you’re convinced the code branch can’t ever be taken, you also should be confident that it doesn’t need to be tested.

This feels like chasing arbitrary 100% test coverage at the expense of safety. The code quality isn’t actually improved by omitting the checks even though it makes testing coverage go up.

View on HN

> If you’re convinced the code branch can’t ever be taken, you also should be confident that it doesn’t need to be tested.

I don't think I would (personally) ever be comfortable asserting that a code branch in the machine instructions emitted by a compiler can't ever be taken, no matter what, with 100% confidence, during a large fraction of situations in realistic application or library development, as to do so would require a type system powerful enough to express such an invariant, and in that case, surely the compiler would not emit the branch code in the first place.

One exception might be the presence of some external formal verification scheme which certifies that the branch code can't ever be executed, which is presumably what the article authors are gesturing towards in item D on their list of preconditions.

The argument here is that they're confident that the bounds check isn't needed, and would prefer the compiler not insert one.

The choices therefore are:

1. No bound check

2. Bounds check inserted, but that branch isn't covered by tests

3. Bounds check inserted, and that branch is covered by tests

I'm skeptical of the claim that if (3) is infeasible then the next best option is (1)

Because if it is indeed an impossible scenario, then the lack of coverage shouldn't matter. If it's not an impossible scenario then you have an untested case with option (1) - you've overrun the bounds of an array, which may not be a branch in the code but is definitely a different behaviour than the one you tested.

I think you’re misreading their statement. They aren’t saying they don’t want the compiler to insert the additional code. They’re saying they want to test all code the compiler generates.

> Because if it is indeed an impossible scenario, then the lack of coverage shouldn't matter.

At the point where a load-bearing piece of your quality assurance strategy is 100% branch coverage of the generated machine code, it very much does matter.

> I'm skeptical of the claim that if (3) is infeasible then the next best option is (1)

In the general case, obviously not. But, in the specific case we’re discussing, which is that (2) has the rider of “the development team will be forced to abandon a heretofore important facet of their testing strategy at the exact moment they are rewriting the entire codebase in a language they are guaranteed to have less expertise in,” I think (1) seems pretty defensible.

In safety critical spaces you need to be able to trace any piece of a binary back to code back to requirements. If a piece of running code is implicit in code, it makes that traceability back to requirements harder. But I'd be surprised if things like bounds checks are really a problem for that kind of analysis.

Yeah sounds too clever by half, memory safe languages are less safe because they have bounds checks...maybe I could see it on a space shuttle? Well, only in the most CYA scenarios, I'd imagine.

Bear in mind that SQLite is used in embedded systems, and I absolutely wouldn’t be surprised to learn it’s in space.

Critical applications like that used to use ADA to get much more sophisticated checking than just bounds. No certified engineer would (should) ever design a safety critical system without multiple “unreachable” fail safe mechanisms

Next they’ll have to tell me about how they had to turn off inlining because it creates copies of code which adds some dead branches. Bounds checks are just normal inlined code. Any bounds checked language worth its salt has that coverage for all that stuff already.

> maybe I could see it on a space shuttle?

"Airbus confirms that SQLite is being used in the flight software for the A350 XWB family of aircraft."

https://www.sqlite.org/famous.html

SQLite is used in a lot of hypercritical application areas. I’d almost be surprised if it’s not part of some if not all modern spaceflight stacks.

I don’t see the issue. The operations which produce a bounds check are traceable back to the code which indexes into something.

What tools do you use for this? PlantUML?

There is a whole 'nother level of safety validation that goes beyond your everyday OWASP, or heck even what we consider "highly regulated" industry requirements that 95-99% of us devs care about. SQLite is used in some highly specialized, highly sensitive environments, where they are concerned about bit flips, and corrupted memory. I had the luxury of sitting through Richard Hipp's talk about it one time, but I am certainly butchering it.

You didn't understand the argument. The testing is what instills the confidence.

I'm confused about the claim though. These branches are not at the source level, and test coverage usually is measured at the source level.

> If you’re convinced the code branch can’t ever be taken, you also should be confident that it doesn’t need to be tested.

“What gets us into trouble is not what we don't know. It's what we know for sure that just ain't so.” — Mark Twain, https://www.goodreads.com/quotes/738123

If a code branch can't be ever taken. Doesn't that mean you do not need it? Basically it must be code that will not get executed. So leaving it out does not matter.

If you then can come up a scenario where you need it. Well in fully tested code you do need to test it.

Reply on news.ycombinator.com