Is TDD Dead?
martinfowler.com
A series of conversations between Kent Beck, David Heinemeier Hansson, and Martin Fowler on the topic of Test-Driven Development (TDD) and its impact upon software design.
martinfowler.com
As far as actually improving code quality... not so much. For the money, I would much rather see an hour of functional test development rather than an hour of unit test development (and I put TDD in the unit test category). I think functional tests provide more value and our industry has put way too much emphasis on unit tests.
100% unit test coverage says nothing about code quality. 100% functional coverage does. Functional tests also tend to be less brittle – which is a nice bonus.
I hadn't heard of functional testing before. From a stackoverflow answer:
> Functional Tests are written from the user's perspective. They ensure that the system is functioning as users are expecting it to.
This makes much more sense to me than TDD. TDD is writing functions which test other functions. Are the users writing code, and passing arguments to function calls? If not, then time spent on TDD isn't adding much value for the users.
So what does it mean to write functional tests? I'd argue that writing modular React components is a form of functional testing. Devcards[1] is similar, and looks to me exactly like what would be considered functional testing. These two workflows feel much more natural than testing UIs methods like automated selenium browser interactions to generate screenshots.
1. http://rigsomelight.com/2014/06/03/devcards-taking-interacti...
yes?
> So what does it mean to write functional tests? I'd argue that writing modular React components is a form of functional testing. Devcards[1] is similar, and looks to me exactly like what would be considered functional testing. These two workflows feel much more natural than testing UIs methods like automated selenium browser interactions to generate screenshots.
no.
They can line up, but I'm not sure there's a causation. There are many different ways to write tests the wrong way. It's taken a lot of trail and error for me to get better at it (I can't even say I'm "great" yet). I'd have given up long ago if I wasn't a passionate developer. A less passionate developer would think the whole concept is BS and give up long ago if he took my path.
One of the first mistakes I made was using a mocking framework to test my implementation. The test read like, "first call this method, then call this, then call this". Needless to say it was worthless. I was just writing my code twice.
Then you can use mocks inside-out and end up writing code you don't even need to call, etc.
I'm sure there's many ways to do TDD right, but from in my personal experience, there's more ways to do it wrong.
Where I part ways, though is the notion that an hour of activity X substitutes for an hour of activity Y. For any given fear you have about your code failing, you should pick the testing approach that yields the most risk reduction per unit of effort. I don't think we can externally judge the right choice for a given team and code base. As long as it's reasonably easy for people to write tests at a variety of levels (unit, integration, functional, end-to-end), and as long as they're committed to continuous improvement, I'm inclined to trust that they'll figure out the right mix for their project.
I tried it as an experiment. With 100% coverage, TDD everything.
Code still fails in production. That is because unit tests detract from running functional/integration tests. Database, connections, hardware all mocked and each little cog is the right size. Put the cogs together into a gearbox and it shits the bed.
The one thing TDD helped was to find exact code breakage easier. Functional tests give higher level (less precise) error, maybe "something failed in the backend" then one has to look at logs to see what happens and such. But I'll take that over a false sense of security.
So am definitely not against unit tests, I am all for having a mix of both.
Another way to look at it, unit tests are a bit like drugs. It is easy to get addicted, create 10000 tests and a little micro-burst of pleasure hormone every time you see the dot on the screen for a passing test, it just feels good. But it is important to realize that is what it is (maybe I am writing all those tests just as a therapeutic exercise...), as it might not translate immediately to a ship-able working product.
If you want a talk that preaches to the/our choir, check out this one: http://vimeo.com/68375232
It's interesting how different they are. Mutually exclusive in fact. But is one wrong, both or neither?
Thank you, that was interesting! A lot of good points. I agree with some of them and don't agree with other parts of it.
First, let's start with this basic principle -- "customers don't care if I have 1M unit tests or 0 unit tests". They just don't. They will complain and not like my product if it keeps crashing or is showing other pathological behavior. First and foremost it is important to ship, and to ship working code as far as customers are concerned. Taken from that side, I want to ensure that when the system is setup in customer's configuration (and we do know it, in our line of work), then we want to stimulate the system as close as possible to what a customer would do. If that doesn't break and is behavior properly. We gain immediately a large degree of confidence because it is closer to "doesn't break when customer uses" use case.
Unit tests also promise that, and they would also promise that "if I have unit-tested my code well, there is a higher chance that the code will not break when it gets into customers' hands". However notice that this is further from being true than the first one.
The problem with integration tests as discussed in the talk it they take a long time to run and give imprecise results. When something is broken I might not be able to quickly zoom in on it. I can't experiment as often as I would like because it take too long to discover precise mistakes. And I agree with those, but, I still maintain it is better to have integration tests because once you get into a stable steady state and all of the sudden one commit breaks the tests it usually is pretty clear what broke, in the worst case just revert the commit or examine in detail until it works. That helps with the "I don't know what broke, something did" part. Usually that something is the one commit where tests started failing.
Yeah, it would be better to also have complete 100% unit test coverage as well and zoom in immediately on the problem. No doubt that 100% unit test coverage also costs a lot of time and money and I posit it is often un-necessary.
Another criticism, perhaps I misunderstood the presenter, it seems to me that the talk focuses more on machine-local, object oriented code. Today the systems are more and more distributed. In a pure, local, single threaded, object oriented code (say a 3D modeler or Photoshop).The behavior or objects is more predictable and it is easier to write what he calls "focused tests". Now introduce threads and well that starts to become hairy. Add in multiple machines (a distributed system) and it gets even more complicated. There is a whole slew of strange problems. Often involving consistency, timeouts, networks failure, partitions, hardware peculiarities. Un-expected input that is generated maybe because a front end service is broken and back-end write might never expect that case and crashes. Unit tests will catch basic errors but it will mask those harder to find those kind of errors.
One more criticism is psychological and it is the false sense of security. And I have seen this happen from those that advocate unit testing everything 100%, when their code breaks, they simply don't believe their code could break because it has 100% coverage and they "just ran it and it passed". Well in production it crashes. And customers don't pay for passing tests they pay for a working product. Anyway I am repeating myself at this point and this is longer rant, so, sorry about that.
I think you left out the main complaint: It's impossible to cover a majority of code paths.
there's your problem. TDD with functional tests, only unit test when it's a complicated & important piece of domain/business logic.
I don't understand - how is this useful? Can anything be learnt from this at all? Its very clear to me that no useful conclusions can be reached from debating like this - is it just me?
The only substantial content was a comment on the gist [1] that explores a problem that looks like it came from a production system. Methodology discussions would really benefit from at least one semi-realistic example like that.
[1]: https://gist.github.com/dhh/4849a20d2ba89b34b201#comment-122...
TDD is one of those things that only works when developers have a high degree of control over their own work. Anecdotally, that has been hard for me to find.
I think the only way to get widespread and useful adoption is to actually construct a high-level language around it, such that the code and the test are tied together by syntax, in the same file, and coverage is enforced by the compiler.
Otherwise, you're stuck trying to enforce TDD via voluntary human cooperation. While you may have confidence in writing and refactoring your code, that just shifts the uncertainty that remains in development of a complete product onto the screening of new hires and avoidance of certain management edicts.
Essentially, you need a fully automated test cop.
However, in my Clojure code, I feel 100% test coverage is unnecessary. REPL does a good job of proving my code works as intended. I wonder if I am doing it wrong!
¹Disclaimer: I have no experience with Clojure, working primarily in Scala these days. I do aim for good unit test coverage in Scala just the same as I did when I worked in Java.
That's especially important when multiple people work on a project: you have some chance of remembering what important REPL incantations prove a system works. They won't. If you haven't put your REPL magic in a test suite, then they're going to break your stuff.
Trust me, a large, old, code base like that is terrifying to work with.
Don't obsess over code coverage though, concentrate on areas of risk and scenarios, I've written code with 100% coverage that still had an exciting range of bugs, and I've seen extremely robust code that had relatively low coverage overall but had tests focusing on the really tricky areas.
https://hn.algolia.com/?q=is+tdd+dead#!/story/sort_by_date/0...
Making your whole development approach blindly revolve around one technique is self-evidently a poor idea.
Like any experienced developer I have a lot of respect for Martin Fowler and his work, but sometimes I wonder if his best skill is saying simple things in many, many, many words.
This is obviously an "article" where the author was legitimately interested in answering the question in the title. This is one of the cases where Betteridge's law does not apply.
Betteridge's law may be useful to remind people to watch out for poor journalism and fact checking. In that case, I propose that all articles shall henceforth use questions as titles because people should do that anyway.