Stop Being Frantic About Code Testing
unixsheikh.com
unixsheikh.com
If you're building a low-level infrastructure component, like the storage engine for a database, or an encryption library, then basically every line of code, every piece of functionality should be tested. In these situations tests are pretty straightforward to run, the "right thing" for your software to do doesn't change very much, and without tests it's easy to introduce an accidental bug.
On the other end of the spectrum is things like video games. If you're scripting the behavior for a new object in a video game, it's often very hard to test it does exactly what you want. And you're likely to change your mind about this behavior, and a test isn't very useful when you simply change the test every time you change the feature. In situations like these, tests often aren't useful.
So basically the question of "how much unit testing should I do" is a perfect recipe for the two sides to argue past each other based on very different experiences.
Probably could not disagree more with this statement. I recently implemented an element-wise hash function over arrays of arbitrary length values. I wanted to be doubly sure that my logic for choosing memory offsets into these arrays was correct. So I tested the function extremely thoroughly on some important examples and also on randomly generated data. I did this because the hashing function was a core component of the system I was building. If it didn't work, especially on subtle edge cases, the consequences would be severe. So what did I get out of testing that function so thoroughly? Psychological effect, as the author puts it? Well, yes I suppose. I get to feel confident that the code works as expected. I get to believe this not just when I originally write it but also when changes are made to it on into the future. I get to believe the same about components built on top of and around it. But calling these benefits "psychological", while technically true, seems to miss the point. I'd also call them essential. If you want to increasingly waste your time chasing issues that pop up because your code isn't properly covered by tests (and do that in production because you deployed before realizing there was a problem), be my guest.
Some of the backlash around testing is about testing things that really shouldn't need to be. If there's a mandate from some clueless management that all code must be covered by tests and the new code is simple data containers, then one ends up having to write tests for methods like getValue() { return value; }, which is a bit absurd.
I'm assuming the author of the article is in the latter situation, not the situation you describe where tests are actually useful.
Well, if you're just measuring code coverage (which is what clueless management obsesses about), as long as something that uses that getter is tested, it will be covered and won't need its own separate unit test. And if it isn't covered, that means you're not using it in which case you can probably get rid of it...
A lot of junior (and not so junior) engineers write bad tests. Each test is huge, with lots of irrelevant setup, and many assertions. It’s hard to tell which requirement is being verified, and it can break for lots of reasons. So they change code anywhere and lots of tests break and it’s not clear what’s going wrong. Yeah I can understand how people with that kind of experience come to believe tests are a waste of time.
When an application is tested from day one, it ends up designed for testability, and individual tests are succinct, often one liners. Testing feels easy and delivers a high ROI.
I think of testing as a fundamental skill, like learning good form in a sport like tennis. You can be self taught with bad form and do alright for a while. But pretty soon you’ll hit a skill ceiling and get left behind by those who got coaching and practiced their fundamentals.
"Growing Object-Oriented Software, Guided by Tests" by Freeman and Pryce is a great book to get started learning how to test well.
Since most teams have an implicit testing time budget I find that teams which try to mollify useless unit tests that hold up CI tend to skimp on other less noisy forms of testing that work better.
Good unit tests are still better than manual equivalents, obviously.
It’s also driven up the cost of development or slows the time needed to release. That may be an obvious but I think because of the religious aspect people often make the leap to trying to get 100% code coverage instead of focusing on their original objectives.
Additionally, tests are some of the best documentation you can have, and you need them to refactor.
Finally, I actually really like TDD, it helps me to structure what I'm going to work on, and keeps me focus. So I guess I disagree with the OP, even though I do much of what they do in terms of actual application testing.
I just don't think it's actually and either/or.
If you are good at unit testing you can turn your brain off and the tests write themselves. It’s about preserving any method chain call flows and preserving the integrity of anything that is parsing something, like a Regex or whatever.
Focus on testing after you have built a nice little house of cards... it will make it harder for another dev to come along and blow it over in the future.
I've observed that way too much unit testing ends up having this effect, but it doesn't have to. There are two extremes with unit testing: "no unit testing at all" and "mock everything", and both are wrong for different reasons. I have yet to see a codebase with no unit testing which doesn't have problems (in production that impact real users) that could easily have been caught by a simple unit test. The mock-everything approach ensures that you can never make a change to the code, though. A simple middle-ground is to mock everything that's actually external to the system being tested: databases, file systems, remote services. Otherwise, use the actual code: if a function calls another function, internal to the system being tested, just call that function (but make sure it has a unit test of its own).
I've seen many code bases work much better with a bundle of integration tests and no unit tests.
IMHO tests which hit a real database are always more useful than ones that mock it out.
Unless you actively update fixture data for all of your unit tests, you can never be sure that all green check marks means everything will work once in production.
Most were pointless because of how much they were mocked out.
They lack the skills to make good tests and I don’t want to be 12 hours shifted from family to train/babysit them. I mostly try to get them to isolate what each section of code does. This keeps it more sane when debugging.
Reality of working with low wage offshore developers.