I still haven't seen a test that can prove even an identity function is correct. Yet it's obvious to the programmer.
We lack the technology to properly test software. I like to think tests are a way to implement twice and hope you got it right at least once. But we aren't even to that point yet.
Techniques that are intended to prove correctness - like for example symbolic execution - easily handle the identity function (of course!).
Being engineers requires you to assess the risk, and if something is not worth checking, there is no reason to check it.
As with any hypothesis (empirical) testing, software testing is not about proving code is correct, but about striving to falsify that claim by poking at ways it would be likely to fail if it wasn't.
(There are formal analytical methods for proving code correct as opposed to empirical methods of falsifying it, as well, but those aren't tests.)
That is not 'implement twice and hope you get it right at least once'. It isn't a perfect proof of correctness either, but it gives much more guarantees than a simple double implementation.
Notably, when my code to solve a linear system using LU decomposition failed, I new I didn't need to worry about my decomposition, because those tests still passed.
"Sorting" should "preserve length" in {
forAll() { (l: List[Int]) => assert(sort(l).length == l.length) }
}
"Sorting" should "be idempotent" in {
forAll() { (l: List[Int]) => assert(sort(l) == sort(sort(l))) }
"Sorting" should "put elements in order" in {
forAll() { (l: List[Int]) =>
whenever(l.nonEmpty) {
sort(l).foldLeft((true, l.min))({
case ((result, max), elem) => (result && max <= elem, elem)
}) match {
case (result, max) => assert(result && max == l.max)
}
}
}
}
"Sorting" should "not change elements" in {
forAll() { (l: List[Int) => {
val sorted = sort(l)
assert(l.all(elem => sorted.contains(elem)))
assert(sorted.all(elem => l.contains(elem)))
}
}
In contrast, we can't implement the 'sort' function in such a piecemeal way. It needs to take every requirement into account, in case a step which satisfies one requirement invalidates the others. That's also why writing a whole new implementation to test against is best seen as a last resort.
Tests exist to help prevent you from changing behavior unknowingly.
Tests can be a tool for documenting expected behaviour, intended use, and edge cases. In contrast to written documentation, they have to be up-to-date by their very nature, which is an advantage over code comments or external docs.
This to me is the biggest thing that makes writing good tests a "grind". For a simple, well-defined function, you can toss up a bunch of inputs and outputs and feel satisfied that all the corner cases work. For any moderately complex behavior, however, so far the only effective way I've found to test the behavior is to effectively duplicate the logic a second time in the testing code, and check to see that I got the same result; which is pretty discouraging when you're doing it.
It's certainly nice once it's done, though, to be able to make a change, re-run your test suite, and have a pretty reasonable confidence that nothing broke.
The problem is, that there's no way to generally quantify "too much" or "not enough" when it comes to tests and documentation.
Even "harmful" isn't a well defined term on its own and needs to be properly defined on a case-by-case basis.
The statement is a platitude without any substance behind it and you can replace "testing and documentation" with just about anything and come to the same result:
Vitamines and calorie intake are both essential but too much of either becomes harmful.
> Individuals and interactions over processes and tools
> Working software over comprehensive documentation
> Customer collaboration over contract negotiation
> Responding to change over following a plan
The things on the right are only useful in service of the things on the left.
Because users always report bugs...
If you've ever been frustrated by the package manager Conda and the UX of its command-line interface ...I'm sorry. I tried.
Well, if you refuse to write tests and you are scared to make new changes without them... then you can't push new fixes at all. Which is apparently fine for some organizations.
Also, I agree that's exactly what testing is, in fact I often now test code like that, get someone else to do one simple implementation in Python, a second high-performance implementation in C++/Rust, and compare them.
Then, as long as the same bug doesn't occur in both versions (and that seems very unlikely, as they are implemented with different algorithms), we find all the bugs in both.