Test-Driven Development is Stupid
geometrian.com
geometrian.com
Yeah, the fact that you can't comprehend why people do this is very clear; if you could, you wouldn't have written this terrible rant. It feels like the author is criticizing this before coming anywhere close to understanding why people do it.
I really wanted some kind of point to find to disprove, bu there aren't any in this post. The strongest proof the author offers to his point is using one word tautological sentences to repeat what he just said: "Tests don't work because they just. don't. work."
The more I read, the more I realized this post isn't about testing, it's about the author letting the world know how fucking awesome and smart they are. They build huge software suites (from the sound of it, completely alone) with no tests, and everything works out fine, even better than fine, spectacular. It seems like the author deleted this line from the post in response to ridicule in the comments:
> "As it happens, I do write code others depend on--as it happens, a lot of it--and, my code has never, even once, failed in production: a record I am extremely proud of."
I don't even particularly ascribe to TDD, but the arrogance and dismissiveness and contempt of this guy made want to see him proved completely and utterly wrong.
To piggyback on this... sure, the code doesn't exist yet, but the project specs do. And unit tests can help by making the required specification explicit.
Regarding the rest of the article it is pure trolling with nothing really useful to add to the debate.
The author's point, in the painting comparison and when mentioning other fields, is that you never have a good enough understanding of the problem you are trying to solve when you're just starting to work on it. Developing the code is an iterative process that refines your understanding of the problem and the correct solution, and only once you've done that do you know enough to write appropriate tests.
There are exceptions to this, but they tend to be less common or trivial. If you're implementing a known algorithm, such as an implementation of a mathematical function, you know enough to write the test cases first. You can do it for very generic libraries too, like a library of sorting methods. But in practice most code is not so easy to spec completely before you start to implement it.
No TDD advocates say you should spec completely before you start to implement. Most say you should write a single, very simple test. It's not much more than the function signature, and almost everybody in almost every language writes the function signature before writing the code.
Developing the tests is an iterative process that refines your understanding of the problem and the correct solution, and only once you've done that do you know enough to write appropriate code.
Quite the contrary, as we build anything, we necessarily pick some way of organizing our code and our data. When we inevitably discover that we were horribly wrong, it is tremendously valuable to be told more about what needs to change in concert as we reorganize.
If you don't know what you want your code to do then tests will just make it harder to experiment, and if you do know then there's no harm in writing them after the fact.
There is a difference between making the spec explicit (which is providing concrete examples of the spec's meaning) and replacing the spec. You seem to be talking about tests doing the latter, while the grandparent comment talked about the former.
> If you don't know what you want your code to do then tests will just make it harder to experiment, and if you do know then there's no harm in writing them after the fact.
If different people are responsible for the unit under design vs. the parts of the software that will consume that unit, then the concrete examples provided by test cases can help confirm that there is a common understanding of the spec; as such, tests serve to validate the spec.
You want to do that before you code the implementation. And once you've done that, the tests are also available to validate the implementation.
One option would be to start with a spec and gradually add citations, doing TDD along the way for a sort of Documentation Driven Development, although I've no particular confidence that's actually a sweet spot.
doctest is a move in this direction, but only really makes sense for code you wish to surface as examples. Cucumber is also obviously related, but - so far as I understand - isn't really suitable for producing arbitrary documentation. Types are (amongst other things) another form of machine checked documentation, but the audience (and, usually, expressiveness) is naturally limited.
Your point is an excellent one. If you see hundreds of really smart people doing X and seemingly very happy with it, there's probably something going on there that's more important than just a bunch of fools with a fad. It might not be appropriate in your situation, it might be oversold by those folks -- but it's highly unlikely that's it's stupid and dumb.
I work with training developers all the time. It continues to amaze me how people go from being a novice to being an expert. One day they can't code at all. Next day they know everything there is to know about it -- and can go on at length about why process X is a terrible thing, without even trying or understanding it.
Of course, we have the opposite problem in development also. People get sold on ideas that will never work simply because they sound cool or the cool kids are doing them. This reminds me of the old saying "Always be open-minded -- just not empty-headed"
The Dunning-Kruger effect in action.
But that also applies to ...
>> If you see hundreds of really smart people doing X and seemingly very happy with it, there's probably something going on there
Extreme Programming has through its "extremeness" brought automated unit testing front and center. That doesn't mean it's necessarily a good idea to do XP or TDD. Unit tests are now a mainstream idea, and being pedantic and militant about when exactly they should be written and what are the steps needed to be blessed as using the approved way is not being helpful.
TDD is not a unit testing religion. It's a way of designing software. Some have even advocated calling it Test-Driven Design, not Test-Driven Development, because the end result is a design, not a series of tests.
This is important to know, because the key factor in the red-green-blue cycle is the blue step: refactoring. The example given by the author is of a system that kept getting tests -- but was never refactored. Instead, more junk kept got thrown in. In this case, they were not doing the most important thing in TDD, refactoring. Not really sure you can even call this TDD. Looks more like a CF created by a bunch of folks under time pressure who thought it would be easier to add stuff than refactor.
If you don't understand what it is, probably not a good idea to criticize it. TDD is a way at looking at constructing OO programs. It's main competitor is probably more along the lines of Model-Driven Development.
TDD is a response to the large amount of commercial code being written in complex object-oriented environments. The goal is to ensure that changes to the code maintain the economic value of the code. In my opinion it is not appropriate where there is little or no economic value, like personal, academic, and start-up code. In cases where there are not complex object graphs full of dependencies, such as small pure FP projects, it's also not indicated -- there's no OO system for TDD to help you design. And if you're not designing stuff, you're not doing TDD.
People get religious about TDD, yep, and it drives me nuts. But like I said, that's no reason not to understand what the hell is going on before criticizing it. Unfortunately, people getting excited and over-selling things is just the way it is in tech, whether it's TDD or the latest cool programming language.
I am afraid I do not consider TDD to be a legitimate design method. There's an inherent tension between well designed OO code and testable code.
Well designed code is properly encapsulated, does one thing, has a minimal interface and few dependencies. Testable code is often less encapsulated, contains extra complexity for the test machinery, among which we can count extra methods (with potentially non-private visibility) and dependencies between classes (such as strategies, policies, factories, etc).
When I design I always start from the closest-to-ideal incarnation of the objects, based on OO principles and the way clients of my components will want to use them. Then I compromise on that design just enough to be able to test it to an acceptable level (which varies per component). It could be that I will write no unit test at all, and instead test several interacting components. Someone following TDD will immediately compromise in favor of testing and automatically add maintenance overhead.
Furthermore, while designing at the class level is important, it is much more important to architect at higher levels: component, subsystem, application, system/platform. All of these aren't considered when writing unit tests and what makes perfect sense in a "unit" might only be a local optimum.
Designing at the test level is to me as architecting a house at the brick level...
I think that's a more a consequence of poor testing libraries/frameworks than of TDD itself. A method shouldn't have to be turned non-private to be tested.
Furthermore, while designing at the class level is important, it is much more important to architect at higher levels: component, subsystem, application, system/platform. All of these aren't considered when writing unit tests and what makes perfect sense in a "unit" might only be a local optimum.
It's TDD, not UTDD.
No, there isn't. Code that is not testable is not well-designed.
> Well designed code is properly encapsulated, does one thing, has a minimal interface and few dependencies.
And testing tests what the consumer of code will use, using the interface.
> Testable code is often less encapsulated, contains extra complexity for the test machinery, among which we can count extra methods (with potentially non-private visibility) and dependencies between classes (such as strategies, policies, factories, etc).
Only if you are doing testing wrong; what you should be testing, even for unit testing, is the behavior of the unit under test as provided through its exposed interfaces to consumers of the unit under test.
> Furthermore, while designing at the class level is important, it is much more important to architect at higher levels: component, subsystem, application, system/platform.
Conventional, unit-test-focused TDD is a practice isolated to a particular level which does not conflict with higher-level practices of the same type that address that concern, like Acceptance Test Driven Development (ATDD).
We're pretty far down an HN thread, but if you're interested in pursuing this further, I shot about an hour of video to show how I would code a small app. You'll find the three parts here:http://tiny-giant-books.com/blog/technical-story-slicing-1-o...
I'd enjoy continuing the conversation via email if you'd like. Let me know!
To be fair, this is a common failure mode when people try to do TDD.
(to be clear, common amongst failures; not necessarily most common amongst failures, and not necessarily a terribly common outcome overall)
And he's lost me in his first sentence.
Unit testing, good unit testing at least, is not just about developing as it is about preventing regressions. A unit test that runs on every build ensure that something is true and that it stays true forever.
Obviously if you are against testing in general you are against TDD (it's just testing taken to the extreme).
I thought he at least was going to say that he preferred higher level testing, to low level unit tests (which can be a fair point, you can find 90% of the issues with 10% of the test code if you accept that you can't always isolate the failure based on the test report). But no. He is against testing.
If you're developing something which must work 100% every time or else someone will die, then unit tests are a must.
If you're developing with people who break the build all the time, then unit tests are useful.
Unit tests that aren't 100% up to date with the code are worse than no unit tests.
A bit of a tautology... broken code is broken. Unmaintained code, even unit tests, should be deleted.
If any piece doesn't succeed, the command stops. No push without unit tests passing.
Unit tests are a safety mechanism- you can easily bog yourself down with too many, tying things up in bad ways, or presuming a specific design. But do you really want to live with no safety at all?
The art, as I see it, lies in writing tests in such a way that you get the guarantees you need, the safety of not letting the next guy (or yourself) screw it all up during later changes while simultaneously not preventing you from doing needed refactoring.
It's a fun challenge.
If any test fails, it is because you broke it just now, so you go in and see if your test is broken, or, as is often the case, your refactored code is breaking the expected behaviour of the tested code.
Absolutely this! Unit testing helps you during the dark years of maintenance phase, when you must fix a bug or add a new feature and you don't have the knowledge, nor the time, to build a full representation of the application in your head: you just touch as little code as possible to get the work done, and unit tests help to ensure that all the rest stays equal.
I know that in my own work, the single most important thing for me is to be able to massively refactor, restructure and redesign my and others' code, as I'm working with it (this is probably why I like dynamic languages and macros so much, and probably why I benefit from static typing so much); anything which gets in the way of deleting, reorganising, clarifying, duplicating, altering, reducing and shifting code is going to slow me down and make the resulting code worse.
Perhaps, though, there is a middle ground: while the internals of a library must be free to mutate, the external interface ought not to change nearly as much. Perhaps all unit tests should be written to test the external interface, not the internal details. That might also help avoid the tests-which-test-that-the-code-does-what-it-does disease.
I do really appreciate tests, and they do help detect and prevent certain types of regressions.
Having worked with dynamic languages quite a bit, I found that iff your language has a decent REPL, so you can code iteratively, you don't need unit tests to drive your design. You perform those tests in REPL yourself, which helps you flesh out the design, but you don't have layers of code that'll make changing the design harder.
The way I personally work with Lisp is a mix of writing code in a file, compiling particular parts of it and playing around with it in REPL until I've figured out the right design - at which point I may as well start adding some tests around tricky places. The idea is to not burden yourself with permanent tests until you have your design figured out.
I reduced 20,000 lines of tests to 5,000, and caught a dozen bugs.
Yes, this is the most useful approach to unit testing. If you have a functional interface which doesn't cause side effects, (i.e. state changes of any kind, external actions that can't really be effectively verified by test code, etc) then unit testing is very valuable and useful given you just test the interface using the simplest possible approach: a table of some inputs which should map to some outputs.
I'd go further and say you should always write the tests only after this interface has solidified. Test first makes sense only under either the assumption that your first design is always the best design, or that you are happy to spend the time to morph the tests, as well as the code when you realise your first design wasn't the best and have to iterate and tweak.
We all know from experience that the first assumption is never true. As for the second assumption, it may be true, but then the question becomes, well, why waste the time? You know those 'test-first' tests will just be rewritten anyway. Why not just write the tests last and save yourself a lot of wasted effort?
If he thinks test driven development can produce horrible implementations then he's right, they can. But he's horribly wrong that the solution is to abandon unit testing.
Without unit tests developers begin to live in fear of parts of the code, and refuse to clean it up "in case it breaks something", and there's no way to verify those fears aren't founded.
Any refactoring job starts with annoying a lot of people by breaking things that previous worked and questions start to appear from people that "it used to work before" when some of those breakages slip through to the customer.
Refactoring becomes something to fear because "we'll have to retest everything" which is a large expense that cannot be afforded.
"Trying to improve software quality by increasing the amount of testing is like try[ing] to lose weight by weighing yourself more often."
From someone who has lost over 15kg in the past in few months, one of the things that helped most was starting weighing myself as I didn't do that previously. It kept reminding me that I wasn't still there and gave me more motivation
"Frequent Self-Weighing and Visual Feedback for Weight Loss in Overweight Adults"
"The major finding of this study is that the use of frequent weighing accompanied by visual feedback of weight, without a prescribed diet or exercise plan, was effective in producing a small but sustainable weight loss in overweight males."
I took the average of the 3-4 last days at the same time (about waking up) is a good enough approximation. Once I learned this, "more often" means once daily, as opposed to "once monthly" (if any). It wasn't only about the hard number, it was about the motivation.
http://david.heinemeierhansson.com/2014/tdd-is-dead-long-liv...
And yes, it may be stupid to test before a design emerges - but only if you start with a very fine-grained test. Usually when I'm coding from scratch I write a very, very, VERY coarse-grained test that "tests something", and when I reach the point of passing it (which may involved creating and designing multiple classes) I probably have a working design and I may begin creating other, smaller unit tests for individual components. The initial test may disappear or become an integration or acceptance test.
By the way there's little content in the article. It's just a rant. And the article about not writing test cases at all is simply ridiculous - since an error can exist in a test, we shouldn't write test cases?!?
More generally, it's about not 'coding too far into the future', even (especially) with tests.
But I like to code a little ways into the future.
'README driven development' is also interesting.
This reads pretty much how I work too.I don't think it is unusual. Start coarse, outside in. Write the finer grained 'unit' tests where appropriate. Focus on where you need the crutch (design and confidence) not on clambering to test all the things.
That is exactly how I start coding something from scratch, except I don't have that initial test. I don't think such a vague starting test adds anything real.
I mean, you can say that. But doing so makes you look... ignorant - at best. You know, there is a possibility that you just got it wrong.
So, many great programmers say that they get benefits from TDD. They get benefits. Not the suits. Not their co-workers. They.
TDD is hard. I had to learn it and practice it. And I'm still learning and practicing it, even though I'm now also teaching it to others and helping teams implement it. But I get benefits from it. Writing tests helps me to think about problems and to actually improve my designs [2]. And writing tests helps me to know when to stop - To not gold-plate my designs.
Sorry, but this "article" is just an angry rant, with no real arguments. Please, don't give up on TDD early because of rants like this. If you have any questions about TDD or need help getting started, feel free to ask (here or in private - you'll find my email address in my profile)...
[1] http://davidtanzer.net/no_true_scotsman_in_agile
[2] But you have to refactor ruthlessly. And learn and practice refactoring, which is hard.
What do you think is the right way to do it, then?
BTW, if any technique in software development leads you to bad design, and you don't stop and try to improve something, you're doing software development wrong. If a technique does not help you, you have two possibilities: Trying to do it better (maybe with outside help), or trying something different.
[1] http://www.makinggoodsoftware.com/2012/01/27/the-evil-unit-t...
Ultimately, we're too young a field to be able to replace common sense with a process.
Personally, what annoys me the most about TDD I've seen in the wild are two things: designing for tests instead of actual problems, and tests affecting the structure of "real" code.
Designing for tests - the standard TDD approach, first we write tests, then we write code to pass the tests. Quite often the consideration of the problem being solved disappears. It's a fine approach when your task is to write a small black box that takes some data in and transforms it into something else. But I've never seen a case where someone made it work for complex tasks. It always ends up the same - your tests become more complicated than the tested code. It happens with any non-trivial problem, because the thinking you have to do to write tests that make sense is the same as the thinking you need to do to solve the problem in the first place. So you're basically writing the program twice, only in a convoluted way, and without considerations for global design.
Tests affecting the structure - this is IMO a strong code smell. If you're modifying your design to accomodate for tests, by e.g. adding superfluous dependencies, hooks or injection points, you've screwed up. It only makes the code more complicated and less reliable.
The only tests I've found valuable so far (in terms of effect for effort spent) are regression tests - the ones you write to catch bugs in order to make sure they won't happen again. Everything else in TDD seems to be easily replaceable by proper iterative programming.
Maybe that's why TDD is popular in the languages without a sane REPL.
On the other hand, code being hard or impossible to test is often thought of as a code smell as well. Code that is easy to test is easier to understand — not in the least because there are tests demonstrating its use. Techniques such as dependency injection help a lot here.
I agree with you that a code that's hard or impossible to test is a code smell. I argue that having to modify a good design to accomodate for more testing is also a code smell. I think the two heuristics narrow down the design space nicely, pointing one towards designs that are easy to test because of their natural boundaries, and not because of additional testing cruft being added.
It's like saying that one is annoyed by programming in general because he's seen too many horrible things done with it! Anything can be abused, TDD is not an exception.
> your tests become more complicated than the tested code
Well, then don't tdd that code on unit level. Keep some high-level(smoke, integration etc) tests that executes it , relax and write/design it without TDD the best way you can.
> Tests affecting the structure - this is IMO a strong code smell.
Yes, and this smell (by definition) shows you a flaw in the code design. TDD helped to identify this. Apparently)
I'm not saying TDD makes sense everywhere, but being able to confirm that your 3000+ continuous integration tests all passed green before shipping a new version to all of your customer is a huge way to avoid embarrassingly predictable bugs. (Leaving, of course, the opportunity to ship the embarrassingly non-obvious bugs)
And, honestly, it's not clear that he was opposed to TDD, as he was opposed to being constrained to having to write huge test frameworks during the exploratory phase of code development, in which you may want to have a bit more freedom to write/discard while you feel out requirements/solutions.
Instead there were lots of parallels that I don't recognise.
It was disappointing the author didn't seem to try and find the advantages in TDD and discuss a counter argument or alternative, such as modifying a foreign code base.
Because it's easy to measure whether you're exercising every code path, software engineers think in terms of how close they are to "100% coverage" of tests. What you really should be thinking about is whether you're exercising every possible input, not every possible code path.
One of the facets of science is that you have a hypothesis. Writing a unit test before you start is analogous to having the hypothesis: "the solution I come up with for this problem will work." The rest of the exercise of TDD is a scientific process of creating a solution and proving that your solution works (and adjusting your hypothesis if you find it to be incorrect), albeit in a slightly strange way.
I could have a hypothesis about how earth is really traveling through the stars on the back of a turtle. We know that is not true because Einstein came up with a solution, followed by him and others proving that solution. Relatively is only valuable because proof of it exists.
If computer science is a science (which it is) we must put our solutions to the same degree of rigor that other scientists do.
While I no longer follow the strictest form of TDD (starting with tests that fail to compile) I'll never forget the singular most important lesson that it has taught me:
A solution is worthless if you cannot prove that it works.
Secondly, I wouldn't try to fit unit testing into scientific process - because if the test is a hypothesis, then what you're doing is the exact opposite of how science is done. You do not design your experiment to make your hypothesis come out true!
While the quote you cited is interesting, I'd treat it with a grain of salt, given that you can't prove that your solution works - and if you think you did, it usually turns out the proof itself is wrong. It happened even for formally proven algorithms.
Testing gives you increased confidence. It's a worthwhile goal. But IMO, driving your design by tests is going a bit too far, and something you don't need to do to have tests that ensure your code works.
True, but I think that objection can be trivially fixed by simply... doing that. I write tests that confirm the code does what I designed, but I also write code that tries to break my design, and tests that verify that it errors as expected. I do approach it with a scientific mindset. (And I tend to consider "scientific mindset" to be the more important part of science vs. some overprivileging some checkpoint list of specific techniques, which ought to have come from the scientific mindset in the first place.)
Of course it remains true that you can't do that perfectly, but that's a null objection in the end. Nothing ever can be, but at least I try.
I can't even count how many times I've tested a code's error case, only to discover that it unexpectedly "worked". Usually that's because there's a bug and I need to fix the error case... every once in a while it turns out my code corrects my own understanding when it reveals what I thought was an error case is actually perfectly valid and sensible, though. It's important to try to break the code.
True. If you just follow the checklist without following the spirit of the scientific method, you end up doing socio^H^H^H^H^Hcargo-cult science.
As for scientific mindset in programming, I think it's a very valuable thing to have on both larger scale - in various forms of testing - and smaller scale. I found that, when running your code, it's good to just ask yourself what exactly do you expect to happen beforehand, and if you see any deviation, immediately go figure it out, or at least note it down. If a program does something unexpected it means you don't understand it.
> art
In stark contrast to what I've said: I also believe this to be true. Software development is a cross section of multiple disciplines: art and mathematics possibly being two of the largest within that cross section. Science, while a smaller component, is still present.
> You do not design your experiment to make your hypothesis come out true!
Indeed, as stated the TDD approach is strange and is only analogous to the formal scientific process.
> driving your design by tests is going a bit too far [...]
I didn't stick to formal TDD for very long for exactly that reason: it takes ages and constrains you to think about the tiny details instead of the grander design of the project. The upshot of formal TDD is that it usually results in low coupling but there are other ways to achieve that.
I don't understand this. What is so absurd about specifying facts about the program you will write? When we have tools that can prove facts, we will be doing formal specifications instead of random sampling. But still, testing is a way of statistically specifying facts about your program, for which it seems sensible to be written before the program.
But, there is another common case where unit tests (written before or after) also come in tremendously handy: if you can't write good mistake-free code 100% of the time.
Which describes all programmers who have ever existed.
Not really absurd, just impractical. It's because all those facts are wrong, and you don't know yet why.
There are valid reasons to be against TDD, but it is not stupid. To call a methodology "stupid" is needlessly judgmental and really has NO place in the pragmatism and trade offs that so often go along with writing software. Anyone that calls a software methodology "stupid" so casually has no business writing software at all, nor blog posts about software.
I don't care how smart you are, or how right you are. If you want people to listen to you and consider your ideas, don't write posts like this, in this tone, at this length, with so little actual substance.
Due to poor planning / management, unit tests often weren't done. Bad decisions by others ended up biting me! I got pulled into a project to do a big refactor because somehow I was considered the DSP expert and a predecessor picked a lame DSP for the new version of the product. No unit tests meant I was pretty screwed.
1. Well-tested parts
2. Decoupled design
3. Rapid feedback
4. Local context to test in.
At the same time, there are many cases where they don't help much. More here: http://henrikwarne.com/2014/09/04/a-response-to-why-most-uni...
"Look, this code looks strange to you, you would have done it differently, and you are afraid you might break things you don't even know exist. But fear not, there are tests, they run, and they will guide you."
It's always relieve to find that some random old crap is actually well tested.
This is quite apparent especially when I conduct pair programming interviews. Developers who were exposed to TDD (or to projects with significant test code) approach the problem in a far more structured manner, and their code is much more pleasant to look at :)
This is just a rant. I was hoping for an actual study, but the title should have given it away.
The cycle is: red, green, refactor, repeat. That's per test. It shouldn't take long. It works nicely.
It also assumes a bug free app with 100% code coverage is the objective, regardless of the cost it takes to write all that test code. An app that has a few bugs but takes half the resources to develop can make better business sense.
I disagree on this. 100% coverage is actually a bad thing, in my opinion, because there's always a lot of code you do not need to test (simple constructors, accessors/mutators, etc).
A unit test should test things that you are relying on being correct. You write a unit test as a way to protect future developers (including yourself) from totally screwing things up by accidentally changing something important.
An application with a few bugs that took half the resources to develop is nice- until the customer needs a few extra changes 18 months later, and you turn 'a few' bugs into 'a whole lot of' bugs.
I get the point, but feel different. When a project becomes large enough, changes can result in breakage in the weirdest places. Proper testing can help discovering this breakage before shipping. For me, tests are a safety net for changes in the tested code.
Large projects without any tests are effectively unmaintainable, often times even for the original author, and most certainly for others.
PS. Regression is far worse than momentum against change, when your app is `broken` you can't deliver it.
>- You're trying to make a design before you learn anything about it.
>- People write something one way, but then are afraid to change it because they'll have to rewrite the testing code that goes along with it.
Both of these suggest tests of the wrong granularity; that people are, as he says, writing silly little fencepost error checks for every single function.
> Week 5-10: Write code.
??? what
The feedback cycle is one test, implement code. Not implement test suite for the entire program, then implement your code base.
- Function A is 147 lines long. It is the simple core of the program.
- Function A is committed to the repository on June 26th, 2002. Function A has four test cases. Nevertheless, a bug is found in Function A on the 28th and a patch is uploaded on July 6th. It contains two new test cases.
- This continues a bit. However, by August 2002, function A is mostly stable and has no fewer than thirteen test cases--mainly for fencepost errors and other idiotic things anyone can find with a stack trace. Except for a blip in early 2003, function A, now 152 lines long, is unchanged until mid-2006.
At the time I rationalized it and said - nah, I'm too lazy for doing that on a side project. But as soon as I tried to use it in the real world, I discovered that all those RFCs are just crude approximations of how people used the server (most of them are even incomplete). There's absolutely no compliant client or server out there, and TDD would catch none of the bugs I discovered at the time. And that's with email - a protocol that people are working on standardizing since before the Internet existed.
Sorry, but nowadays I'm extremely skeptical about TDD having any application at all. Not even for reinventing the wheel.
I'm a contract java developer and I know how to 'play the game'. The hypocrisy in the amount of fizbuz clones I've written in a TDD fashion on interviews, to then see the production code has no little to no unit tests.
I once got negative feedback on an interview because I wasn't TDD enough after expressing the opinion that TDD works great for most use cases but there are limits, e.g. positioning of a front end element isn't always best done TDD. The guy interviewing was non technical and of course any dissent to TDD meant I was a bad fit.
Anyways - end rant. Like I said I just learnt to play the game.
I don't mind writing a few test cases during or after I am done with my work. This is mostly to protect someone (including me) to change my perfect design after I have found it, though ;-)
Oh.
I've tried TDD, but I've found that it just stifles my productivity too much.
http://www.drdobbs.com/tdd-is-about-design-not-testing/22921...
If I remember correctly from the interview (I think it is here[2]), one conclusion was that TDD doesn't have a clear benefit when you add it to a project. On the other hand, in a survey, TDD projects are more likely to succeed because it is a habit common to good developers. I hope I am capturing the subtlety there. Essentially, TDD is not a silver bullet, but rather a good habit shared by many good developers. That was enough to convince me of the merits.
It's another problem altogether to try to institute TDD for a project, especially for a team. Like so many things in programming, TDD could be used and abused. The same could be said for JavaScript or [insert proper noun here]. If misunderstood or used incorrectly, TDD could be a drain on the project. A benefit--and this ties back into the idea of TDD as a habit--is that it forces the code you write to have at least one other client. This requirement would alter the way you write code and arguably for the better.
Sounds like most everyone's first years :)
In order to write a test for something, you need to know what it is supposed to do, and testing will not tell you that. In order to make something that passes your tests, you have to design it, and a test does not tell you what that design should be - it can tell you if you failed in a specific way, but not what to do about it. There is a whole lot of analytical reasoning and technical judgement to software development that is ignored by TDD, and while thinking about test cases can help with this, it is an insufficiently powerful method to complete the job.
Agile methods have not rescinded this fact. Insofar as they form a complete development methodology (and I am not sure about that), they offer an empirical approach to discovering your requirements (which may or may not be appropriate to your situation), and the use of short-cycle iteration to gauge how you are progressing, but they are largely silent on the issues I raised above.
Thats not to say all testing is bad. I do some unit test and some functional/integration tests but by no means do I strive for some arbitrary % of coverage as if that means anything. You can have 100% coverage with completely brainless useless tests that are testing lots of simple low risk code. Any sense of security from that is beside the point if your team is incompetent, the overall design calcifies and becomes a mess and nobody fully understands how it all works.
Targeted testing I guess is how I think of it, very targeted. I work on a small project in a small team, in a much larger project with lots of devs I would have to rethink maybe but then again with properly sized teams (two pizzas) that don’t even come into existence.
>"Trying to improve software quality by increasing the amount of testing is like try[ing] to lose weight by weighing yourself more often."
So this is really funny to me because I just read an article that in fact states that weighing yourself more often does help people lost weight. http://www.sciencedaily.com/releases/2015/06/150617134622.ht...
Just by looking at the page and the style of reading you can tell this is someone who has not progressed in their career skills since the early 90's
But it slowed development down massively.
In the start because I had to set up the testing as I had to setup the real software.
Then I had to mess around with the testing framework as I had with the projects frameworks.
Also I had to write features AND write their tests.
And later in the project, when new features broke old stuff not only the features had to be fixed but also the tests.
I don't know, but I had the feeling that the time I spent with the test-code was the same I would have spent with fixing possible bugs later.
What I like about unit testing that it's forcing you to separate irrelevant code and prevents you from writing spagetti code.
In my personal oppinion you should test core functions as someone already said, which validates 90% code with 10% of time. And leave parts which can fail gracefully in case of a bug. It's not practical/impossible to test 100% of use cases of a big codebase. But you can test critical parts.
> How often have you seen a program crash? If it was developed by a large software company, chances are it was written using TDD. Clearly, TDD is not a magic bullet. So, TDD does not "prove your code works".
> Developing software is like a painting commission
Several reasons I gave up a third of the way in. I rarely write tests first, but I can appreciate that it works for plenty of people - my brain just doesn't work that way.
It's hard to take anything in this article seriously because of the nerdrage and the dismissal of anything he disagrees with as "stupid".
People don't spend weeks writing tests for all the functionality and then write the tests. That's not what even the most die-hard TDD advocates do.
I think many people here (like me) clicked through to read a well-reasoned article about how TDD enthusiasm may have gone too far, but sadly, this isn't that.
HN readers would do well to just move on, and the author would probably be well served by the advice of some of the commenters on his blog to take this post offline.
I realize this is probably here to provoke a reaction, but it's still retarded and it's obviously wrong for the obvious reasons.
> You are writing code to test something that doesn't even exist yet. I am not rightly able to apprehend the kind of confusion of ideas that could provoke such a method.
I agree TDD can be taken too far and be enforced too strict (although I've yet to encounter that in practice).
That said: It's not stupid to write tests before the code which you are supposed to test. The order of these things are here for a reason: If you don't write your tests before you have working code, how do you know your tests will detect a failure mode, and thus can be used to prove that your code is now working?
Just the other day I thought I had fixed a bug, and then proceeded to write a unit-test for it. The unit-test went green and I was happy.
But then I decided to comment out my fix and re-run the test. I assumed it probably wasn't needed. I was confident and knew after all that I had "fixed the bug".
But that way I could at least say I had followed the TDD mantra, which claims to be there for a reason: 1. write test to reproduce bug, 2. write fix, 3. rerun test and if green 4. commit.
Upon commenting out my fix and "needlessly" rerunning the tests, lo and behold: The test was still green.
My test failed to detect the error-condition. Which meant that my patch had probably not fixed the reported bug. Quality had not improved.
I rewrote the tests and managed to make them go red, that is, detecting the failure mode. Uncommenting my fix and rebuilding, my test was still red. My fix was indeed invalid!
Another round of investigation showed that I had misinterpreted the error condition and produced a "fix" which didn't solve the real problem.
And the "stupid" principles of TDD helped me detect a invalid fix, find the real issue and verify a real fix for it. TDD helped me increase quality.
Stupid indeed, eh?
Giving the benefit of the doubt that this quote is in fact an accurate metaphor – to begin with this quote is to begin with the assumption that TDD's single motivation is improving software quality. TDD is a fantastic practice for ensuring conformity to a specification; which one might describe as software's correctness. A more objective metric than "quality".
If the author had bad results writing tests for an entire program before writing any of the program, I'm not terribly suprised. But since doing so isn't TDD, it doesn't add even a single data point to the TDD discussion.
Well yes, I use my unit tests to learn what works and what doesn't. You're allowed to rewrite or scrap tests, if they are no longer relevant. In the end I feel that having unit tests result in a better final design.
Yes, sometime unit tests are a little contrived. But they can also help you design cleaner interfaces and increase your code reuse.
After this sentence I abandoned the article.
What kind of serious software don't need automated tests? Maybe the kind of no one uses.
If you want to change what a function does, you have to change all the tests you wrote for it.
Do you drive your car (software) by banging against rail guards (unit tests) to reach a destination? They are there so that you do drive off the road accidentally not guide your way.
We'll probably find experience on both side of "good" and "bad". I'm curious about the "bad" and how that comes about!
> Week 5-10: Write code.
I'm not necessarily a TDD advocate but I've literally never heard anyone say you should spend two solid weeks writing tests and then six solid weeks writing code against those tests.
I mean, that's just idiotic.
http://www.drdobbs.com/tdd-is-about-design-not-testing/22921...
But other companies don't do it that way, and require TDD, so the article took a narrow, biased view without considering pros and cons and different scenarios.
it's a waste of time to read, as it don't bring anything new to the table. i wish the OP would at least attempt to reflect on what TDD is, what it's not and why oh why, so many devs seems to like it.
False dilemma. Are competence and good design exclusively from people that don't TDD?
> The basic idea--amazingly, one of the most popular methods of software engineering (and growing in popularity!)--is that, after you figure out what you want to do and how you want to do it, you write up a set of small test programs--each testing some tiny piece of the program that will exist. Then, you write the program.
No. You are wrong. It is not that.
> Let me emphasize: you write the test cases for your program, and then you write your program. You are writing code to test something that doesn't even exist yet. I am not rightly able to apprehend the kind of confusion of ideas that could provoke such a method.
You are writing specifications for a program that doesn't exist yet!? Nuts.
> The most important argument is a practical one: Test-First doesn't work.
It doesn't work because it doesn't work. Sad argument.
> I was once one of these newly educated kids. So, for a few years, I worked exclusively with the Test-First strategy. When it was over, the results were undeniable. The code--all of it--was horrible. Class projects, research code, contract programs, indie games, everything I'd written during that time--it was all slow, hard to read, and so buggy I probably should have cried. It passed the tests, but not much else.
So it didn't work for you so it is everybody else's problem but yours.
> So this is the first reason TDD fails: You're trying to make a design before you learn anything about it.
No. You are not. Indeed, depending on how you decide to do your tests you should be able to change your design and the underlying implementation and no test should break. Please, take a look to the outside in approach.
> Week 3-4: Write tests.
> Week 5-10: Write code.
Oh my god. This is terrible. And no, this is not TDD either. This is inverse waterfall :P
> Even if you somehow succeed, TDD prevents incremental drafts by functionally requiring all tests for a module to pass before you get any real results.
Probably this is a misconception too. You don't write all the tests before. One test, small implementation pass, another test, small implementation pass... You get the results incrementally.
> There is literally no substitute for competence. If your coders don't have it, TDD won't fix it.
I fully agree on this. And probably this is a really common misconception. But false dilemma again. I don't think TDD came in this world to turn bad programmers into good programmers. It is just another tool in the toolset. You can have terrible programmers that do TDD and excellent programmers that do it.
My recommendation: https://vimeo.com/68375232
The essay doesn't have the high quality of content I'd expect from PhD candidate in computer science. I'd expect less outbursts of passion and more discussion of pros/cons/tradeoffs.
First, without getting into "TDD" itself, let's just isolate whether "tests" are valuable. I think the examples of SQLite's rock solid reliability (extensive Tcl regression tests[1]) and NASA's disciplined software verification prove that tests help discover bugs and increase code quality.
Tests are valuable -- but they also have a "cost" to write. Let's cover the cost issue at the end.
When to write the tests. If the sequence is: write the code first and then the tests, you might call them regression tests. If you write the tests first and then the code (e.g. iterations to turn red FAILED into green PASSED), you can call it "TDD". And those TDD tests can still function later as regression tests.
To me, TDD acts as a "design" step or "outline" 10,000 foot view. Before any plumbing code is written, what do you want the REST API or group of functions to "look like". Is there unity and coherence to the collection of fucntions modeling the abstraction? The subsequent of fleshing out TDD with "expects()" and "asserts()" is just mechanical work to glue the edit+compile cycles to a verification target but it's not the most interesting aspect philosophically.
However, even though tests have a benefit, there is a cost. The cost-benefit works in some cases but not others:
In my experience, I'm completely sold on TDD (or regression tests) for foundational library type of code. If you're writing a core string library that 100 developers at the company (or open source community) will link into their projects, I prefer seeing extensive regression tests covering all edge cases that proves that it actually works the way the developer intended. It's not strange at all if the code for regression tests outnumber the actual code 10-to-1.
On the other hand, TDD that is mostly UI verification is extremely brittle. If you have TDD that simluates mouse clicks and has "expects()" on reading webpage UI elements to check if things like sales tax calc is correct, you could easily get overwhelmed by all the extra work that synchronizing the actual code and the TDD scenarios generates. (E.g. a UX designer moves an icon 2 pixels or adds a row to table and ends up breaking the entire TDD validation suite for developers.) I could see where TDD at that level would be counterproductive.
If, on the other hand, your problem is not very well defined, then your tests have a good chance of eventually becoming a liability. If you ever discover that your domain model, interfaces, or even selection of algorithms are insufficient (or just plain wrong), it's more likely that your pre-existing test code will be unusable, rather than the sort of guide for refactoring that they're touted to be. If you had guessed at the interface or algorithms correctly, but merely implemented them incorrectly, yes, the tests will guide you back to correctness, but I think that tends to be a big "if". Given that you don't understand the problem, the likelihood that you've made mistakes in the design are high.
Of course, this isn't the fault of TDD, that's exactly what it's meant to be. The problem is in people thinking that is equivalent to verification.
TDD is ultimately a design tool, one that is useful in cases where we have a very good idea of constraints and requirements of a problem, on where the problem is very, very well defined.
And that's why I say it's a code-smell. The problems that are well-defined are often that way because several people have created several implementations of it already. If I'm TDDing something, it usually means I'm rewriting code that already exists somewhere.
Now, I might have good reason to do that. Perhaps I have constraints that nobody else has ever considered. I generally hate the phrase "don't reinvent the wheel". I can think of at least 3 times off the top of my head that the wheel was successfully and usefully reinvented in the 20th century alone. But it's very important that you understand that is what you're doing. If you are aware you're reimplementing a known solution, you can now choose to study your forebears and get an even better understanding of the problem.
Personally, I think saved REPL sessions are better than TDD for problems that are not very well understood or well defined. If I were to try to build an MPEG encoder in JavaScript (for whatever reason), I'd certainly use TDD, because MPEG encoders have specifically well-known inputs and outputs. But if I am trying to invent a completely new UI paradigm for virtual reality, then TDD is just not an applicable tool.
Also, it's a lot easier to tell myself to just discard a saved REPL session that ceases to be valuable after a major rethink in how the problem works than it is to discard tests. Plus, I find them to be a little more informative in terms of demonstrating behavior to other developers than test code.
I agree so much so that the best systems I've built have always been the 2nd revision of a build one to throw it away prototype.
That said, I like to surround my code with lots of unit tests to eliminate absolutely stupid bugs.