value = 1 / input;
You can get "100% coverage" if you test that with `input = 1`, but unless you check with `input = 0` you're missing a quite important logical check. data Nat = Z | S Nat
data NonZeroNat = OnePlus Nat
data NonZeroInt = Negative NonZeroNat | Positive NonZeroNat struct nonzero_t {
int is_negative;
unsigned int one_less_than_the_absolute_value;
};
which, under interpretation, ranges from -(2^32) to -1 and +1 to +(2^32).Consider a function which checks 5 simple if-statements in a row, always in the same order. Getting each branch means you tested 10 things.
But there are 32 ways for 5 if-statements to jointly evaluate. If there is a logical dependency between the state checked by one if-statement and the state checked by another one, your perfect coverage may not pick up on that.
If the if-statements might be checked in an arbitrary order... there are 120 ways to order 5 things. But you'll still get perfect branch coverage by checking 10 of them.
int returns_less_than_twelve(int input1, int input2) {
int sum = 0;
if (input >= 5) {
sum += 5;
}
if (input2 >= 5) {
sum += 8;
}
return sum;
}
The following test cases will pass and achieve 100% branch coverage. int x = returns_less_than_twelve(5,0);
test_assert(x < 12);
int y = returns_less_than_twelve(0,5);
test_assert(x < 12);
However, this does not cover the entire input space so returns_less_than_twelve(5,5);
test_assert(x < 12);
Will fail. However, branch coverage won't tell you that there's a hole in your test coverage.Generally however, writing full branch coverage will find a lot of issues, and also cause you to really think through how your code works; but still, it doesn't guarantee correctness. If you want that you need to start bringing tools that either exhaust your input space (a function which takes 5 booleans can be exhaustively tested for correctness in trivial amounts of time), or you start modeling your chosen language well enough that you can use a mathematical prover to demonstrate that your program or function is safe on all inputs.
This of course requires you to come up with a definition of 'correct' or 'safe'. For the above program, it's clear how to define correctness. For things like "Don't let an unauthorized individual access this data or data that is derived from it in a way contrary to the desires of the owner of said data" it gets 'tricky' ;).
I try to only use random data when possible, less and smaller tests to write with a proper setup. End result: more bugs found.
Random data is a great method of testing.