Tests are not meant for ensuring it works with all inputs. You cant simply just throw values at it hoping its all okay.
To prove it works with all possible inputs, there are other tools at your disposal.
To prove it works with all possible inputs, there are other tools at your disposal.
Brute force.
Oh other story. An OG (original geek) I know once proved the floating point unit on a mainframe was broken via brute force. Bad part meant the decimal floats used by and only by the accounting department sometimes produced erroneous results.
Me I have a pitiful amount of unit tests for the main codebase I work on. One module which does have unit tests, also had a bug that resulted in us having to replace a couple of thousand units in the field.
Otherwise most of the bugs that chap my keister aren't about procedural correctness, they're about state.
A + B == B + A
A + 0 == A
(A + B) + C == A + (B + C)
A + B != A + C (given B != C)When you have one available, testing against the results against a reference implementation is a little more direct. Though, given how easy those property tests are, you might as well do both.
FLOAT_EQ(sqrt(x) * sqrt(x), x) && sqrt(x)>=0.0
Some times I have, for example, matlab code that I 'know' is right and want to verify that my C/Python/Julia... code does the same thing. So then I just call the matlab code, call the new code and check if they're the same.
Or I have a slow exact method and want to see if my faster approximation always gives an answer 'close enough' to the correct answer.
This sounds pretty sketchy at first, but it turns out that almost all of the difficulty (and hence almost all of the bugs) are in getting the last few bits right, and the last few bits of a much-higher precision result don't matter when you're testing a lower-precision implementation, so combined with some basic sanity checks this is actually a very reasonable testing strategy (and widely used in industry).
I'd like to hear about real world test scenarios where all possible inputs are tested.
The input fuzzing process is rarely purely random. There are advanced techniques that allow the fuzzer to link input data to conditions of not covered branches.
It is quite useful mechanism for checking inputs, formats, behaviour patterns (if you have two solutions but one model, one simple that works 100% but is slowish and one more complex but very fast).
See: https://github.com/dvyukov/go-fuzz#trophies and http://lcamtuf.coredump.cx/afl/
What I meant was what @AstralStorm said about mathematical and logic proofs that it works for all defined values.
Proofs are powerful, but you can still make errors in the proof or the transcription into code. And of course you don't know what other emergent behaviours will surprise you when the proved code interacts with unproved code.
Depending on your means and needs, you want to try both.
A sane type system would be a start for example. And it's no coincidence that functional programming and practices with emphasis on immutability are on the rise; Rusts ownership system is a direct consequence as well.
TDD _is_ important, if simply for enabling well-factored code and somewhat guarding against regression bugs. But - decades after Dijkstras statement (which someone has already posted in this thread) - the code coverage honeymoon finally seems to be over.