The Duct Tape Programmer (Response from Uncle Bob Martin)
blog.objectmentor.com
blog.objectmentor.com
I've found that what makes the best programmer is not if they are a "duct tape" programmer or a "architecture astronaut" or whatever, but self awareness. They can call a hack a hack, and understand it's impact. They know when and how architecture can be improved, and when it should be improved. They can take criticism of their work because odds are, the criticism is something they already had an understanding of and can justify, or if not, they are willing to see where there was a gap in their thinking. They have a plan, and can forge a system ahead in a general direction of progress without getting bogged down in the details or taking one step forward and two steps back. You get the idea.
Nobody is talking about this, arguments generally deteriorate where someone sets up a false dichotomy and then the internet collectively rips them apart. But they get ripped apart because of the falseness of the dichotomy, not because of the fact that they are looking at the wrong things altogether.
My nutshell analogy in Joel-speak: "software perspective is to architecture astronaut as product perspective is to duct tape programmer".
The dangerous path you get on is "always shipping" without paying attention to what maintenance means. Of course, this only matters to a Real Business - you know, places that make money instead of burning it. But I find that there is a breaking point where all that code rushed out of the door starts to cost ya. And yeah, you might have to consider a revision, perhaps even adding fancy things like unit tests to figure out what the hell is breaking, or a template to reduce the code base size, whatever.
But even in maintenance mode, the perspective still doesn't change. It should be about making a good product (lowering maintenance costs to speed up new development) instead of "the right way to build software".
the dangerous path you get on is "always shipping" without paying attention to what maintenance means
The Joel article glosses over this too much. Why? Because this is what the unwashed masses of programmer wannabes attempt to do all day every day. You know these people, the ones that just have no business being programmers. In other industries there are standards, credentials and transparent performance evaluations that prevent people like this from even getting hired. Yet in the programming industry they make up a disturbing percentage of the work force. That's one extreme--the most common one.
Joel is more focused on the architecture astronauts who need code to be a little too elegant. I know first hand that these kinds of people can really damage your ability to ship a product if given too much control, but I think it's wrong to draw a distinction and favor a "duct-tape programmer". First, because most of these people probably have the capacity to ship amazing stuff with a little guidance and the occasional reality check from more "duct-tapey" colleagues. Second, because these guys push forward the state of the art. They often see the big architecture picture in a way that others don't. They may waste a lot of time by short-term metrics, but their work is the only kind that eventually leads to long-term dividends. Occasionally some of them may be so obsessive compulsive that they just can't be forced to ship. Fine, they should go get Ph.Ds and do great things in academia, but they're still infinitely more valuable than the people who don't even really grok programming.
His whole point is that short-term productivity is sometimes more important. No point building a product that'll be easy for you to extend in the future if you don't build it quick enough to have a future.
That's the difference between a duct tape programmer and a genius duct tape programmer. The genius has a huge brain that envelopes the entire program. To the genius, everything is local. The non-genius duct tape programmer can't comprehend the entire program, so he just blindly hacks on local problems. When you can't comprehend the whole system at once, you have to use abstraction and good design to ensure that your locally good parts will interact to create a good whole. Compare this quote from Charles Simonyi:
Right now, I have to really concentrate and I might even get a headache just trying to imagine something clearly and distinctly with twenty or thirty components. When I was young, I could imagine a castle with twenty rooms with each room having ten different objects in it. I would have no problem. I can't do that anymore. Now I think more in terms of earlier experiences. I see a network of inchoate clouds, instead of the picture-postcard clearness. But I do write better programs.
In those days, almost everyone writing desktop softare got their butt kicked my programmers writing multi threaded COM who worked at Microsoft, a company that indulged in blatantly illegal or semi legal tactics to do said "butt kicking". Eric Sink recalls how he had an epihpany when he came to know that Micrsoft had 1000 developers working on IE, a product that they gave away. I wouldn't judge programming ability by getting steamrollered by the MS juggernaut.
Same with COM, unit tests, design patterns, etc. Obviously, Joel doesn't believe these have no benefits and that they're only used because they're cool. He doesn't endorse them because he doesn't believe they are worth it.
It's fine if you disagree with the duct tape mindset, but don't construct strawmen to show "we all really agree".
Not everyone is Frank Gehry, but working architects can learn from him just as we working programmers can learn from Bob Martin.
Many of those classic rectangularish buildings are beautiful. Their beauty is not the kind that stuns you for a week and then becomes annoying. It's the kind that is comfortable to be around, and you appreciate more and more as you spend years living with it.
I'm sorry you don't like Frank Gehry. But surely you're not saying, categorically, that software architecture is pointless?
I'm making that point that there may be good design or bad design, but we should not characterize design itself as harmful-- which is what is implied by pitting "duct tapers" against "architecture astronauts."
Not at all, only that Frank Gehry might be a poor analogy to enlighten people about the value of software architecture and design. "Duct tapers good" vs. "architecture astronauts bad" is surely a false dichotomy and a horrible analysis of the situation. Here's an example that might shed better light on the real situation: Microsoft is crawling with duct tapers, and Apple takes design seriously. Microsoft is much wealthier, but I prefer to use Apple products, and I wouldn't want to work at Microsoft.
This seems logically impossible. Maybe I'm misunderstanding TDD, but at the heart of it, TDD seems to suggest you:
1) Give instructions that describe how to check the instructions you're about to give are correct
2) Give instructions
This is not the right solution to the problem "I'm unable to give the computer the instructions I intend". At the very fundamental level, it clashes with DRY, and is only effective if you're very uncertain in your ability to do (2). The right solution is improving your ability to do (2) - by creating better abstractions and metaphors, not by repeating yourself.
Two erroneous assumptions in your thesis are that 1) unit tests catch all stupid little errors so you can stop worrying about them, and 2) the time spent chasing down stupid little errors that would have been caught by your tests is made up for by the cost of creating those tests in the first place.
Unit tests are not magic fairy dust. Having them does not make your code bug-free and even if something is covered by a test it is still possible that both the test and the code are wrong in the same way. Putting too much belief in correctness due to tests passing can be just as bad as not having any tests at all. Unit tests are great at api boundaries and points of interface between data-munging code, but every function or process does not need its own unit test.
Unit tests also have a high up-front cost that is frequently not justified by the return on this investment. There is a subtle art to selecting the granularity of testing that takes a while to learn, and spending too much time around TDD zealots will often lead coders who have not learned this skill to waste time writing tests instead of getting things done.
Like IDEs that flag type errors, compiler warnings, and debugger breakpoints, a test is just a tool. If you can't write working, maintainable code without any of these tools then perhaps you should spend more time honing your craft instead of collecting another crutch.
Let me share a secret with you: developers are human beings. They make stupid mistakes.
driving defensively is a better approach -- you probably will never need insurance. that's why insurance companies give discounts to those who took defensive-driving course.
This is very true. The only way to learn TDD is to overdo it. You need at least one project (maybe on your own time) where you over-test and find that your unit tests are making the code harder to modify, not easier. It's sort of like inheritance. You have to abuse it to learn that it's best used pretty lightly and obviously.
I knew even before I started that TDD is an aid to design, not a substitute for QA. Still, I got charged up with the idea that my tests were going to cover the entire space of all possible inputs. Bad idea. The resulting pain taught me a lot.
For me, it helps. I'm going to go out on a limb and say it will help you too.
If you haven't tried TDD, I'd recommend attempting it at least once, maybe try running through a simple problem with a partner or at a coding dojo.
One assertion counters another. Joel is wrong because "Uncle" Bob says so. Why do TDD zealots sound like religious preachers?
No harm writing unit tests. They serve well for (a certain amount of) regression testing. The whole Test Driven approach to designing software is more dubious (but wildly endorsed by the Agile consultant crowd).
More sober developers seem to disagree.
From Coders at Work
"Seibel: What about the idea of using tests to drive design?
Norvig: I see tests more as a way of correcting errors rather than as a way of design. This extreme approach of saying, “Well, the first thing you do is write a test that says I get the right answer at the end,” and then you run it and see that it fails, and then you say, “What do I need next?”—that doesn’t seem like the right way to design something to me."
but do listen to their advice and do your own thinking.
The point is not to write tests because Norvig doesn't, or equally, to write tests because "Uncle" Bob does.
Just think about what they say before copying them blindly especially when you encounter methodology vendors, "naked men who sell you their shirt" as it were, and not succumb to the latest snake oil nostrum just because its vendors have loud voices.
1) You aren't perfect.
2) Your coworkers aren't perfect.
3) Your "abstractions" are still code that can have errors in it
Also, you are misunderstanding the argument that TDD speeds you up. Yes, at the micro level it may take a bit more time. But, as the system grows, your ability to add or change functionality without breaking things increases when you have tests to verify that it is still correct. (Also, automated testing is much, much faster than manual testing.)
I disagree unless all your changes are really shallow. You change the structure of your application in significant ways and you'll break the tests too. So now you have two problems.
You have some big tests that are general and test that all the components work together, but for each component you have lots of little tests, and if you are going to change the component you change the tests FIRST. That's TDD.
You deliberately break the tests before you change one line of the actual code.
Now certainly writing everything twice is actually a great way to catch errors. NASA does that with the shuttle. But I wouldn't argue that its faster.
I prefer to just liberally sprinkle my code with run-time assertions. We have a small defect rate even after pushing out massive structural changes to the application. The bugs that we do have tend to involve an unpredictable set of conditions that no automated test would ever be written to find.
You are correct sir! What the tests do is allow you to refactor (reorganize or clean up code) and verify that you haven't changed functionality. That's the theory.
In practice however you are hardly ever refactoring for its own sake. In most cases you can't really justify paying developers to change code without some benefit that someone (i.e. a customer) will pay YOU for. And as soon as you start changing behaviors, you now have to change code and tests (or write new tests).
Well, I'm not perfect, but I aspire to write perfect code, that is, code which has no bugs. One way of doing that is writing unit tests - telling the computer what to do, and then telling it how to check what I just told it to do. A better way is making sure I'm using abstractions that match my intentions.
Aside: I see TDD as a little different from regression testing, which seems to make a little more sense.
In one TDD shop I worked in, test failures were often as not due to bugs in the test code as in the app code.
Here are two things you have to do when programming: start with a problem that is somewhat vaguely specified and flesh it out, and break problems down into very tiny subproblems. We all have lots of ways of doing those things: talking with another programmer, writing things down in a notebook, writing a little bit of throwaway code to flesh out an idea or convince yourself that it works.
TDD gets the computer involved in that process. You can't solve the whole problem this minute, but you can bite off a piece. Saying, "I'll write a little unit test for this" means you're not just thinking, you're really seeing how your approach works. You're getting very clear. Now suppose you find that something is hard to test. This is often a sign that it's coupled to the rest of the system in a bad way. When you have a lot of tests, you can try out new ideas very quickly and see if you've overlooked something, or quickly get an idea of how much code needs to change to accommodate the new idea.
A big benefit of TDD is that it leads you to write much simpler code than you otherwise would. I don't expect my say-so to convince you of this; it's just something you have to play with and see for yourself. Many times, I've found that something like an if statement was all I actually needed to accomplish something that looked like it was going to be hard. I found the simple way because I wrote the test first. Sometimes, the resulting code doesn't feel like "I" wrote it. It has a strangely minimalist quality.
The biggest benefit of TDD, though, is as a tool for communication within a team. When pairing, the unit tests make very clear what the current problem is. When understanding someone else's code, the unit tests make very clear how you're supposed to call some code. When you modify someone else's complex code, well, that's when you really start to value unit tests. The unit tests store knowledge of the program in a way that humans can understand and the computer can verify.
Of course TDD does not solve all problems. It's just a neat tool, which often produces simpler code than you could write "by yourself". You should know what it's for, though. It's a design helper, not a QA department. If it's not helping you design better, then you shouldn't use it.
There are many nuggets there about IDE, unit testing, proof of program correctness, literate programming, design patterns. My reading of the book is that if your gig is agile or TDD or templates, you will have a cold audience in those coders. Even if you bring it on with attitude.
I do a lot of unit testing myself, but I know many programmers for whom unit testing does slow them down. Some people get so caught up in unit testing and refactoring for better testability that they either get nothing done. And some people do it wrong, test for extremely specific behavior, and their code becomes impossible to change. I have also known people that try TDD but end up failing to write code that they know how to write but they don't know how to test. It is not a panacea. You have to "do it right".
Contrast that with something like source control. It is hard to waste tons of time on source control that you could have saved if you just didn't use source control. It is fairly easy for even the most incompetent programmers to use source control.
Now is better than never. Although never is often better than right now.
I'd have rather had the product late.
Regarding credentials, we are judging the content on its merits, not deferring to Uncle Bob's authority.
In the same way, software advice that doesn't take the form of deductive argument from mutually agreed-upon principles is largely valuable insofar as the person peddling it is experienced and honest.
TDD != Software
Bullshit. I call TDD more like "insecurity driven development". Perfect fit for big corps, where mistakes are punished, and being fast is not really rewarded. *hence the myriad of processes to do even simple check-ins.
There is much better ways to deal with it: 1. Design interfaces, that implement the features you think you need. That's like the outline of your essay.
2. Have classes that implement them (put all messy code there). Write your essay.
3. Do some functionality testing on the features that are important, and can reasonable be measured (.ie. no functionality testing/unit testing in UI stuff). And make sure you use it. Unit test may catch only some stuff. Proof read it.
From my experience in my previous startups, If you have to ship in three weeks, unit testing is the very first thing that gets tossed out of the window.
At the end of the day, who cares if your product has few bugs. You got to ship something. And I am saying this from experience, where a product I worked is being used by over 2million of people. Buggy? Oh yes. Did the bugs kill the overrall experience, i don't think so. If we had missed that deadline, there was huge financial ramification for the company (apart 10s millions of dollars, it was a relationship breaker with the main client). We took our time to fix some of those bugs on subsequent releases.
If we had to do TDD, we probably would have never get done in time. Sure we might have found and fixed some of the bug earlier, but none of those bugs were detriment to the whole experience anyways.
The only time you really should care to test about even the most minuscule feature, is if you are doing some highly financial/sensitive stuff. Most starts are not in that field anyways.
If you start walking down a dark alley - you start feeling insecure. You've got a few options from that point.
You can back out of the alley and hey, no more insecurity. A little embarrassing but you're feeling OK, you just have to go the long way round the block.
You can push on through the alley in the dark - something bad could happen or you could get through fine. You don't really know until you've popped out the other side.
Or you could carry a god-damned torch. Obviously when carrying a torch, you don't have it turned on all the time, just when you wander into darker areas where you're less sure of the environment. It lets you walk at full pace without stubbing your toe or getting mugged or whatever.
In more concrete terms: I'm a bit of a poor-weather TDDer (same with pair programming) If i start working on something that could be algorithmically challenging, cuts across multiple layers of our app, touches something scary (like our accounting code, because i don't want to lose any of our $$$), or am just plain stumped - I write a test. It fails. I make it pass. I write the next. I make it pass. I realise my code looks like ass, I refactor. The tests still pass.
I can give an actual example from this week. Our app is a large RoR app that serves multiple sites from the one codebase based on an internal DSL we have. We're in the final week of a must launch redesign (date was set because of a bunch of marketing, etc) and I get an urgent and large last minute change dropped on my desk (i'm sure you all know that feeling). I'll admit my first move wasn't to go TDD, I just went for the quick and hopeful move of quickly adding some code that looks like it should work to our DSL.
It didn't. So my very next move, knowing that I have a deadline I must hit with a risky technical change, was to start writing tests. If i hadn't, I'd have to be verifying my code works by checking around 3 different pages after a server restart which is both slow and 4 layers away from the changes. So it was faster in real time.
I'm not a big fan of TDD zealots, TDD is just another tool in the box. But TDD haters really get my goat. You're willfully shutting yourself out of a tool that can make your life easier, and help deliver code faster.
At this scenario, the idea "Be smart. Be clean. Be simple. Ship! And keep a small roll of duct tape at the ready, and don’t be afraid to use it." is just impossible. You have to choose and you have to choose really fast.
AFAIK, he is the chief maintainer of Fitnesse. It is not particularly ground breaking code, and is by no means an epitome of design brilliance. That said, at least he walks the walk and puts some code out there (like Kent Beck [junit] and unlike the rest of the agile "gurus". )
Most of the agile "gurus" can't code for nuts(e.g: Ron Jeffries's "Sudoku Solver" attempt is a public example of how not to use TDD, as is his book on TDD ing which is full of pious agile homilies and terrible code).
It is instructive to make a list of the initiators of the Agile Manifesto and try to find some code written by them so we can judge their skills. Mostly crickets chirping.
Duct taping seems to be at odds with simplicity. Joel's advice to just duct tape it would lead to crappy, unmaintainable code just as often as over-design would.
Here, I'll summarize the conversation:
"Quit dicking around and ship!" "Sure, but you don't want me to ship shit!" "C++ sucks!" "C++ sucks, but sometimes it's a necessary evil!"
There, saved you an hour.
He just said that humans aren't smart enough. They, however, might think that they are. Which is sad.
I'm not sure I trust anyone that doesn't know the difference between a "blog" and a "post".