Why Joel Spolsky was wrong about 'duct tape programmers' being good programmers.
stochasticgeometry.ie
stochasticgeometry.ie
Sure, there are some traits that generally correlate with good programmers (clear thinking, love of scientific method, intellectual honesty) and some traits that generally correlate with bad programmers (magical thinking, forgetfulness) but after working with many, many different kinds of programmers, I've found that there's more than one way that works and having different kinds of people on your team is a good thing. Monoculture leads to groupthink and groupthink is an innovation killer.
I like to think that I'm a good programmer (not great like some people I know, but whatever). I'm good at the kind of work that I choose to do, but I'm aware that there are all sorts of scenarios that I'm not right for. I once left a job after just a few months because the technology/team/culture fit just wasn't right; to that group, I surely wasn't a good programmer.
Joel was writing about an interview of Jamie Zawinski.
Joel and jwz are of the same generation.
jwz worked at Netscape in the early 90s.
In the early 90s, C++, templates, multiple inheritance and multithreading did not work.
Dennehy also somehow seems to miss Joel's main point that programmers should be pragmatic and be wary of using technologies they don't understand, as this is bound to get in the way of shipping a product.
However, while Dyer was making a point based on insufficient reliability and robustness in the available tools, Spolsky is making the same assertion purely on the basis that it takes more time to produce a good design than to use a simpler tool and “duct tape” a solution.
This is not an accurate reflection of Spolsky's article. In particular there is this paragraph:
One principle duct tape programmers understand well is that any kind of coding technique that’s even slightly complicated is going to doom your project. Duct tape programmers tend to avoid C++, templates, multiple inheritance, multithreading, COM, CORBA, and a host of other technologies that are all totally reasonable, when you think long and hard about them, but are, honestly, just a little bit too hard for the human brain.
Added: Spolsky is not claiming that these techniques take too much time. He is claiming that it is very hard to use these techniques to make a product that is reliable and robust.
My reading of Spolsky is that he is making the claim that is it a good idea to have programmers on your team who are very focussed on shipping product, and don't really care whether the technology being used is particularly advanced or popular. They just want to do whatever it takes to ship.
I think he's right; it's good to have these people around. Programmers love learning about new technologies and programming techniques, and when we learn about them we want to apply them. Our enthusiasm can push us to use technology that isn't really required for the problem at hand, and when that technology is difficult to use correctly, this can cause big problems. Having someone around keep you focussed on what matters - shipping a working product - is very helpful.
These are two very different propositions. The OP's title suggests that duct tape is a bad idea and when you read the post you discover the essay was prompted by Spolsky's post. Whereas the title here suggests that the focus of the article is Joel Spolsky. In fact, the if the article is really about why Spolsky was wrong, this admits the possibility that Duct Tape programming is helpful, but that Spolsky got the arguments in its favour wrong.
I'm not arguing the editorialized title is a poor choice, just pointing out that it has a very different and much more personal focus than the OP's title. What do people think: Is it an improvement on the original?
"A 50%-good solution that people actually have solves more problems and survives longer than a 99% solution that nobody has because it’s in your lab where you’re endlessly polishing the damn thing"
I also agree with nickelplate's comment (summary: Joel's main point that programmers should be pragmatic and be wary of using technologies they don't understand, as this is bound to get in the way of shipping a product.)
Pointing to the Tacoma bridge seems a bit silly. I suspect the Tacoma bridge incident is more like the unpredictable black swan, so more meticulous engineering would not have prevented it either.
The first thing to point out is that writing software doesn't map very well onto civil engineering. In civil, your base requirements are supposed to be very well defined. 'I need a bridge from point a to point b, it has to support x amount of traffic weighing y kilos'.
The point of the Tacoma Narrows incident is that 'flutter' was something that was hereto unmentioned in the design requirements of the bridge. It wasn't accounted for, therefore the bridge collapsed. To imply that 'proper engineering' would have prevented the collapse is wrong. They used proper engineering... they just didn't account for everything that they needed to account for.
No one is going to argue that solid design, trusted components, testing and QA are not important parts of programming. (They are good, especially the components, testing, and QA bits).
What the original Spolsky 'duct tape programmer' post is railing against is the disguise of over-engineering (because of incompetence) as the above things (particularly the design part).
The point being that the extremely competent don't need to disguise their software in an elaborate drapery of buzzwords and patterns. (Because they don't need a justification to fall back on if it doesnt... 'well we used x and y and z hot new thing').
Which is not to say there isn't some overlap - writing software for, say, a MRI machine would likely be held to the same processes and standards as most traditional engineering, but let's be honest, 99% of programming isn't that.
> "The standard approach taken to minimise or prevent these unacceptable outcomes is conservative design, using trusted components, with testing and quality assurance being critical parts of the design and implementation processes."
I think the blog author is out of his expertise here - this is an extremely "software" abstraction of the engineering design process.
I'm not convinced the traditional engineering process works at all in software - traditional engineering is waterfall (where do you think it came from?), but unlike a bridge you rarely ever know all of your requirements and specifications up front. Also, you design 100% of a bridge up-front because you can't "iterate" a bridge, but you sure as hell can iterate a website.
We push code out quickly and iterate because we can, and the consequences for system failure are minimal at best. Want to try a new lane arrangement on a bridge? If you could build bridges like you build software, you'd just build two with the competing lane designs, and get half of the car traffic to drive over each. Oh wait, you can't do that in real life. This is why I balk every time someone thinks we should be held to the same standards as traditional engineers - there's no compelling gain and everything to lose.
Nope. You can, however, run rather detailed mathematical simulations of what the varying loads will do to the bridge and cross-check against established data.
Want to know the _real_ difference between civil engineering and software engineering?
About 126 years.
If for $200 civil engineers could test a new bridge design against real, live traffic, with no risk of injury or loss of life... you bet your ass they'd be doing it. But that's not going to happen, because we can't will multiple bridges into existence for almost no money, nor can they occupy the same space :)
This is also why transit planning is often such a clusterfuck - the mathematical models behind urban growth, commute patterns, etc, are often poor predictors of what will actually happen. We've got road layout down to a pretty precise science, but that's about the limit of it. The impact of said roads and thoroughfares on the communities surrounding it, for example, are still poorly understood.
Roundabout way of saying it: I just don't think civil engineering and software engineering will ever truly be comparable. The amount of consequence-free testing, combined with the ability to iterate for extremely cheaply, will never be matched by a field that must build expensive physical objects.
In civil engineering you are dealing the same constraints: physics, which can be refined to perfection because they will be used billions upon billions of times throughout history and will never change. In civil engineering the stakes are high, people will lose lives. Finally, the parameters for success are very easy to measure in advance... it's easy to calculate how much traffic a bridge will carry, and you can even render exactly what it will look like for aesthetic evaluation before construction begins at all.
Contrast with software where the problem space is infinitely larger (ie. the entire realm of abstract thought vs physical space, which also makes it less intuitive by people btw). The costs of failure and repair are most often trivial (especially for web software). And the suitability of software is often not known at all until it is delivered (in this regard architecture has more in common with software engineering). You can't just hand-wave these things away by stating that software engineering is an immature discipline. Even with the exact same number of man hours behind software engineering that you have behind civil engineering, there would still be a such a tiny dent in the problem space as to leave software engineering looking primitive.
It's not that no one takes software engineering seriously either, take military or NASA software. Those guys have the most advanced software engineering principles on the planet, but even there, it's specifically suited to their use case. I'm sure there's a lot we could all learn from them, but if you want Microsoft Word to have that quality are you prepared to pay $500,000 a head to run it?
I agree that Joel totally threw the baby out with the bathwater in the duct-tape programmer defense, but let's not disparage the state of software engineering. It's certainly much less disciplined than other forms of engineering, but that's simply a cost-benefit tradeoff that seems reasonably close to optimal.
If you're building a bridge, yes. But it's not always so critical.
This holds in software engineering as well - If you're mucking about writing the next tetris, odds are, noone's going to die if you stuff it up. If you're writing embedded code in a medical device, lives definitely are at risk (and if you step back and look, not only is embedded code the majority of deployed code out there, but it also has some of the most mature development practices - not everything in software is agile)
The problem arises when you try to draw the line between those two extremes - for example, who knew people were daft enough to follow GPS instructions that navigated them under bridges that were obviously too law, or off piers into harbours, or into oncoming traffic on one-way streets or onto train tracks (complete with oncoming trains). The criticality of the system you're developing isn't always a simple thing to determine.
And don't forget - history has recorded very very similar arguments to yours that claim exactly what you're claiming, while arguing that mechanical engineering will never go from art to science...
You give an excellent example of the scoping issue: Someone can write a computer program that will run detailed mathematical models of a bridge and cross-check against established data. I can also write a program to simulate flutter, run simulations of car engines, the power grid, the stock market (You can program almost anything imaginable).
I challenge you to write a program that will run a detailed mathematical model of a computer program and cross checks against established data gleaned from other programs. Even if you did, it wouldn't be terribly useful as it would just tell you that the program runs in such and such a way; you'd still not know if it actually does what it is supposed to do. Does it fit the problem domain, or is there some 'flutter' that you didn't know about?
A lot of the improvements in reliability of engineering disciplines of the last 50 years have been because of the ability to simulate things better... mostly by use of the computer.
There are therefore two types of error in a piece of software:
1. 'Stupid' error (I used the wrong units or I used the wrong function).
2. Thought error (I didn't understand the problem or I goofed in my reasoning about it).
I'd argue, therefore, that the 'stupid' errors are the errors that can be solved by software engineering. (Using higher level languages, better compilers, better type systems, etc.) The issue being that there is only so much that you can actually do to solve them with software.
The second set is a bit more difficult. Am I going to validate my bridge simulator by using a different bridge simulator? Sometimes we certainly do do this, and this is kind of the basis of iterative development (if each version of a program is a different program). At some point, I will need to understand Civil Engineering as well as Software Development to get my program to function as a Civil engineer would expect it to.
This would imply, that at least in some domains, software engineering is the equivalent of meta-engineering. (I am programming the set of rules used by civil engineers).
It gets more difficult the further away from engineering/scientific disciplines you get. What exactly would be the discipline be behind web programming or writing an HTTP server? Where can I get my thought guidelines to understand how I should think about the different problems?
I'm sure that there is progress to be made in software engineering, but I'd guess the progress won't come from looking at engineering disciplines, and I'm pretty sure it is wrong to say that the only difference is timeframe.
Model checking is well-established in hardware design and is making headway into software. Model checkers can verify high-level properties. For example, I am currently working on a p2p gossip algorithm. I use a probabilistic model checker to verify that the peer selection service provided by this algorithm fulfils its contract - in the steady state selected peers are uniformly distributed over the members of the network. I use a model checker because the maths involved is sufficiently complicated that I don't trust either my theoretical results or that the code matches the mathematical model I am working with.