> they prove nothing
If they fail, they prove there's a bug (in either the test or the code.)
This is like literally any other kind of test.
If they fail, they prove there's a bug (in either the test or the code.)
This is like literally any other kind of test.
There are many places in programming where you don't care to prove properties of your program to this level of rigor; that's fine -- sufficiency is an important distinction: if a handful of examples are enough to convince you that your implementation is correct with regards to your specifications, then it's good enough. Does the file get copied to the right place? Cool.
However there are many more places where unit tests aren't sufficient. You can't express properties like, "this program can share memory and never allows information to escape to other threads." Or, with <= 10 writers all transactions will always complete. A unit test can demonstrate an example of one such case at a time... but you will never prove things one example at a time.
I will add that if you are verifying the correctness of some code, you have a formal specification of what the code is supposed to do. That is, you have a description of valid inputs, and a formula determining if the output is correct, given those inputs. But if you have those, you can also do property-based testing: generate random inputs that satisfy the input properties, and check that the output satisfies the output condition. This is all easier that proving correctness (it requires little or no manual intervention) and gives much of the same benefit.
However I think folks do get dogmatic about testing and will claim things like, “all you need is unit/integration/types.”
It’s easy to write a test that doesn’t provide useful information across time. Harder to write a test that does.
> It’s easy to write a test that doesn’t provide useful information across time.
I firmly believe this is one of those times.
(Currently my only issue with tests in the product I work on is that they take too long to run. Can't have it all.)
The bug is in the library that the test code invokes. Tests themselves should be simple, if there are bugs in tests they are trivial once you have a framework figured out.