Reasons to Avoid Test Driven Development
softwareandi.com
softwareandi.com
There are perfectly good reasons to avoid test-driven development. Like being part of an existing large project, where there are already too many interconnected components. Or working in complex AI algorithms, where separate testing of components is extremely painful, and lots of whole system testing is necessary anyway.
Consider writing tests for the the Linux kernel's device drivers. It would be very difficult because most of the problems have to do with hardware being buggy/underspecified, not with the software being wrong in a testable way.
In order to test a UART driver you have to to simulate the way the UART worked, which means that you wouldn't catch bugs if your understanding of the hardware was wrong. Most of the time spent would be recreating the internal logic of the UART and it wouldn't actually benefit you unless you got it right.
The other option to test the UART is to have a rig set up that where you can run the code on the device and have the serial port looped back or go into another computer that checks the result. But that (by definition) means special hardware is required to run that test, which makes it way less useful as an automated test.
Neither of those situations is amenable to TDD.
First, write code that "tests" the actual hardware, that means it drives the hardware and checks its results. Verify using whatever means you have available that it drives the hardware correctly.
Second, create a variant of this first test (class) that runs against your simulator (checking for the same results). I used a trivial subclass that changed nothing except replacing the device interface with the simulator.
Third, write your simulator against these tests.
Fourth, use your simulator to test the remainder of your system without requiring the hardware/external system.
Extend when you make use of new features, and occasionally test against the real system/hardware...
Here is a talk for anyone who is wondering why TDD doesn't make sense as a default mode: http://www.youtube.com/watch?v=LeVvj4HENOQ
Bottom line is that writing TDD style unit tests won't make crap code better any more than removing them makes good code worse. TDD is a tool, nothing more, nothing less.
> TDD is a tool, nothing more, nothing less
Essentially TDD is just one more form of testing and/or specification. The reason it's so good is because of the properties of executable code: namely that it's fast, precise and repeatable. Human-based QA processes also have their advantages, and if you had unlimited human QA resources then the value of automated testing would be significantly (though not entirely) diminished.
But in the end, testability is ancillary, it's a second-order concern to the actual operation of the software. The ability to write a good test is secondary to the ability (and wisdom!) to choose the right architecture. In other words, a simple and correct program using some advanced techniques in, let's say Haskell, would be preferable to a kludgy Frankenstein beast of a program written in Ruby but with great test coverage over the areas of the problem domain that the programmer was aware of.
Obviously that's a straw man; automated testing is an extremely powerful technique, one which requires practice to master, and TDD shines in its ability to lift the scales from the eyes of the beginning programmer and see programming from a new perspective. However TDD, BDD et al are no silver bullet and no different from learning functional progamming, gdb, printlining, modular programming, meta programming, dynamic programming, regular expressions, or any other algorithm or technique. All just tools.
As for the educating yourself part, yes you'll become a better dev if you are just starting out writing tests. But at some point you'll either become less dependent on them or dogmatic about them. I prefer the former.
Why are too many interconnected components or a complex AI algorithm a valid reason not to test or use test driven development?
My view is that in a complex system, only by having confidence in the units can you have confidence in the integrated whole.
You might be able to build a house that stands and looks nice, but if the timbers, the bricks, the foundations etc are all flawed in their own small ways, sooner or later those problems are going to combine and you will end up squashed.
I can't think of a single example where automated testing couldn't potentially improve what you deliver and the process of working on it. Even the hackiest throwaway script could likely be produced faster and easier with a little automated driver.
However, often the units have properties which are very difficult to specify in isolation, and from experience almost all bugs come from many algorithms reading and writing the same data structure (I'm not thinking of threading issues here, although they are another issue).
Consider something like a sudoku solver (the things I work on are much more complicated than that). Usually writing the function which checks the condition 'a row/column/box are all different' is very simple. The errors come when these functions interact, each effecting each other as they are called.
Certainly, this kind of situation is very unusual. Most people write code which lends itself more naturally to test-driven development, and in those situations I approve of it.
Not engaging with the above argument, just trying to clarify an ambiguity. Just saying (and trying hard to keep tone neutral).
automated testing != test driven development.
The former has been in use for decades. The latter is a relatively new methodology/practice which emphasizes writing tests first and using tests to drive development.
Sarcasm is annoying and often hurtful. I've tried to eliminate it from my everyday speech. Maybe that's a reason this article irks me, and I like TDD.
And this is not to say anything at all about TDD. There are a lot of good uses for it. It just isn't the One True Paradigm of software development. But then again, neither were dynamic languages, object oriented programming, extreme programming, nor any of the other paradigms that were going to cure the problems of software development that went on to add value when creating well thought out plans for every unique project.
The worst part about TDD is the people trying to shove it down your throat.
One way I've convinced people to write unit tests is to remind them that they are already testing. People often write a little main to check the output of a program. This is a test. People often click on a link to validate that some text is present. This is a test. Keep them! Instead of throwing away your stand alone program, put it in the unit test folder. Instead of throwing away that point and click, automate it and put it into the integration tests folder. This alone will get you close to 100% coverage. Now, when you refactor your code and it no longer works the way you verified it would a while ago, you'll get a little red bar or a "FAIL" message. Usually this is because you wanted the behavior to change, so update the test. Occasionally, this is because you introduced unwanted side effects in your code, which is exactly why you want these tests.
TDD is substantially different from what I just described. If the author wants to defend it, that's fine, but let's make sure we understand it's not the only way to get high levels of code coverage.
The humor is so dry that it almost seems like a serious article. I got a few laughs though.
1) Unrealistic assumptions ('perfect' developers, 'perfect' architecture etc.)
2) You simply don't care about quality.
I've been in this position before: We have a large existing codebase with no unit tests to speak of, and the business is not willing to fund efforts to refactor or add unit tests. It is very hard to add unit tests after the fact. Not impossible, but its alot of work, and business folks really don't care.
So you can't really do TDD for your new code without refactoring the existing code-base. But refactoring the existing code-base without good test coverage is nasty.
And it belittles some of the complicated arguments such as: If you develop to the test, the test will always pass— but that doesn't mean that your software is any good. Loosely coupled development and testing can result in the tests being more powerful proxies for overall quality.
TDD doesn't work well if you don't design your code to be testable because you very quickly become frustrated by the problems TDD uncovers.
1. Slow Tests - If your tests are taking a long time, it's probably because your code isn't written to be tested easily.
For example, if your code is hitting the database a lot and it starts to take minutes or hours to run your test suite, your code has a problem. For example, if your entities/models are coupled to an ORM like ActiveRecord, testing those is going to suck because they probably sholdn't be coupled to an ORM. Many people try to fix this by writing frameworks that make testing their code easier instead of writing better code that isn't tied to an ORM.
Tests should be fast. Most of the time they don't need to hit the DB. Slow tests make TDD suck. Avoid slow tests.
2. Untestable Code - I've run into this a lot and it sucks, especially in languages that lend themselves easily to untestable code (In my experience, that's PHP).
Until you try to test your code, you don't realize how untestable your code is. A lot of things that seem like reasonable design decisions are so untestable, that you end up doing incredibly lazy integration tests because to fix things you have to rewrite whole portions of the system.
I've been there and it sucks. The choice seems to lie between rewriting the whole system on the same platform to be testable one chunk at a time, or to rewrite the system in something that lends itself better to testing.
In either case, the answer is to take the time to learn how to write testable code. To start learning techniques like Dependency Injection. To practice writing portions of your system using TDD to see what works and what doesn't.
3. Maintaining Tests Sucks - Actually, if you don't want to maintain a test suite, don't write one.
At its core, your tests are a specification of how your system should work at a given time. When they start breaking it means your system has likely changed, so either it should be fixed to match the specification, or the specification should be changed to match the appropriate system behavior.
At a fundamental level, tests aren't something you write once, they are something you constantly rewrite to keep them in line with what your system should be doing. Thus, when they are broken, you know that either your specs need to change or something in the code isn't working right.
Test maintenance is a price you pay for wanting higher quality software. It's the same kind of cost you pay in designing a building with blueprints. If people build a building without following the blueprints, it's not going to be the building that was supposed to be built. Tests are a living blueprint of your system. If you aren't willing to treat it that way, TDD isn't going to help you.
TDD isn't for everyone because not everyone is willing to make those kind of investments in code quality and maintainability. Also, TDD doesn't make your code better if you aren't willing to let it make your code better.
One thing I've noticed is that code written to be testable tends to suffer from over-abstraction. Sadly, writing nice, compact, easy to read code is somewhat at odds with TDD in my experience.
I personally don't like that I have to make sacrifices to code simplicity just to make it testable. But perhaps I just suck at TDD. :-)
I've gone to great lengths to determine what kind of architecture is needed to truly support TDD and what I've come to realize is that it's not so much about over-abstraction that is the problem, it's that testable code is pluggable, which is mostly at odds with how systems are designed.
For example, standard MVC apps tend to tie models to an ORM in some way shape or form. Think Rails and ActiveRecord and you'll understand what I mean. This makes writing a simple crud app fast, but testing it sucks because you have 2 things tied together - your actual data models and your database layer. Funny that nobody would ever do that with the filesystem, but it's super common with the database.
To make your models clean and easy to test, you need to pull out the ORM bit into something more pluggable.
The same principle applies to writing testable biz logic. A lot of times people don't know where to put it so it ends up in the View, Controller, or Model. All of which are the wrong place usually. Testing that code becomes not awesome because you then have to write a test against what the view outputs, the controller does, or possibly what the model is doing (usually by touching the database).
The answer I've found is to pull your business logic into a separate container that can be tested easily apart from any MVC structure.
None of this means you need to over-abstract anything. It just means you have to figure out the right abstractions and structure for the right layers. None of this is by any means obvious at first.
P.S. I'm working on an architecture framework that simplifies a lot of this to make doing TDD much easier.
If you are making code overly abstract, that does suggest you may be overthinking your TDD approach.
In any event, I find that using literate programming results in less buggy and more comprehensive code.