Working Effectively with Unit Tests
blog.fogcreek.com
blog.fogcreek.com
Hopefully this will become a mainstream thought. Then maybe we will fix the legacy browser languages problem. Man can dream...
Read that and read oldie, but goodie Extreme Programming Explained, and IMO they do a good job of painting the bigger picture.
TL;DR Tests are live specs that break when you change the underlying spec of your code without updating the spec in the test. If you can express your code's spec in a test, then you have a much higher chance of deeply understanding your design... hence the article's recommendation that you first know what you are doing before diving into writing tests.
As for tests in the larger practical sense vs. the trivial... The vast majority of large systems are compositions of small, seemingly trivial pieces. A large system's code needs to be organized so that all its component pieces/concerns are comprehensible and separate; otherwise, it's an unmaintainable monolithic soup. The unit tests adhere to the same principle, in general, and follow the composition of the system.
I totally agree with "DRY might not be great in all cases" - In tests, it's often better to make each single test as expressive as possible. I once read (and forgot who said it):
"Code should be either DRY (don't repeat yourself) or WET (write expressive tests), but never damp)"
And here's a shameless plug: I recently wrote an article about how tests can help you achieve simple design, and the next article in the series will be coming soon: http://quickglance.at/en/simple_design/passes_its_tests
I'm getting so tired of people who try to capture the intricacies and complexities of our profession with witty one liners that have to be taken like dogma.
Just use your common sense, let your experience guide you and if you're not sure about something, ask your teammates.
The one liners can never replace a conversation or explanation. They cannot replace common sense or real knowlege. I would never treat them as laws or follow them by the letter...
Brilliant, added to our bugzilla quips list.
As for common mistakes with unit tests, my top 5 list is:
1. Testing algorithms together with coordinators.
2. Mocking too much.
3. Not using asserts.
4. Leaving print statements in the tests.
5. Checking the log statements, not the result.
More details here: 5 Unit Testing Mistakes http://henrikwarne.com/2014/02/19/5-unit-testing-mistakes/
Quis custodiet ipsos custodes?
Tests are additional code that is tightly coupled to the target code that is meant to be used in production. This additional code requires design, debugging, and maintenance just like the target code. It is also tightly coupled to the target codebase: a change in either necessitates at least an inspection to determine what, if any, changes must then be made in the other.
This all adds overhead to the whole process, and too often I see casual, glib statements implying or even asserting that the overhead is nonexistent or else pays for itself. It isn't free and it is especially not free the more dogmatic the test proponents and tests themselves are.
It's also what I mean by testing algorithmic code (where the pay off from the effort is large), versus testing coordinator code, where you are better off not unit testing it (because it is not complicated).
For more, see: http://blog.stevensanderson.com/2009/11/04/selective-unit-te...
Speaking of coupling, tests prevent tight coupling between different pieces of your codebase. Since there are then at least 2 consumers of any piece of code, the implementation does not get coupled to a particular one as easily.
Also, if you would like to work with a module or microservice based approach, tests are simply necessary to work. If you're storing different pieces of the codebase in different repos, you will need to have tests to work on them at all.
Basically, if you are "testing" your production code with other pieces of your production code (I hope you don't do this), it will become tightly coupled. Test your code with tests.
My point is that tests (at least, unit/integration tests of the variety usually mentioned in TDD) are more code that has to be maintained (and designed, and debugged). They incur overhead and as such impinge on development resources. As a result tests introduce the need for considering tradeoffs between developing tests and developing the software product itself. Simply writing tests for the sake of having tests to test code is naive; at least as naive as assuming that once working a given piece of code works forever, but it is a mistake that I've witnessed on more than one occasion.
I think the IDE could do more in some cases. For example, when unit testing a method, I'd like to be able to tag a reference to the method I'm testing. Then, when I'm in my code, my IDE could show me the docstrings of tests I have for that method.
It's the difference between "hello world" and software design principles used to build large scale systems. I see lots of "hello world" for testing, but not much of the other side. Does anyone have suggestions?
I do agree though that it would be nice to see an IDE have some smarts about tests...
More general advice is that your test structure should mirror your code structure. So a folder called Services should have a matching test folder called Services containing tests for the classes within it.
If we follow this logic then all code should follow the same methodology of readability because tracing through method calls is too cumbersome. Obviously that isn't the case for most code, so why would it be true for unit tests?
TestData.CreatePerson();
TestData.AddSsnToPerson();
TestData.PersonModule.checkWhetherPersonHasValidSsn();
...is pretty readable despite being broken up. It may or may not be true that if you just went through the setup here it may be more readable, but the benefit to this approach is that you only have to write a method to add an SSN to a person object one time and then everyone can use it without needing to understand the structure of different objects. Notwithstanding readability, I would argue that DRY makes writing tests easier than cramming a bunch of setters into every test case. Once somebody understands what a setup method does, in every place that it is used in other tests the tester can skip over that part and move on to testing their specific use case.
Refactor code, and it gives you that much more confidence you have not broken anything.
They're seriously useful mid-later stages of a project, which is probably why startups devs aren't doing them...
On the other hand, when the team has a KPI "we need a big test coverage!" and people are lazy or simply bad developers, you end up with a code like
try {
runTheWholeApp()
} catch (e) {}
assertTrue(itRan)Well actually if the organization is that dysfunctional it's probably game over, but Mutation Testing can catch you when you do this accidentally.