> Really? Just because its structured like waterfall doesn't mean its "how to do waterfall".
That's funny.
925 karma · joined September 26, 2010
That's funny.
It's not the same thing as what we call "evidence", which consists of actually going out in the world and collecting information from more than a tiny and highly biased sample (which, like it or not, is a fair description of the set of programmers most of us will run into over the course of a career).
For instance, for one "Dudley Code-right" who is just as you describe, how many "Dudley Code-fast" exist, who are just as you describe except that no one else can understand their elegant constructions and it turns out that the "nearly bug-free" part, well... disappoints? Dudley Code-fasts look highly "productive" for a little while, then cause big problems down the road.
What we're dealing with here is the representativeness heuristic, which often makes us misjudge statistical truths. It's akin to the way people are much more afraid of flying than of driving, even though the former is orders of magnitude safer.
That's folklore, not fact. The short form of the argument is: the studies are old, not very credible, and based on a notion of "productivity" which is most generously described as vague.
See http://leanpub.com/leprechauns (disclosure: I'm the author) for the long form, with a detailed bibliographical investigation of the "10x studies".
The rest of the "truths" in the OP are of a similar caliber: they are more properly called "opinions". A better title would have been "Some opinions about programming that you don't necessarily share". I happen to agree with some of them; but that doesn't make them true.
I respectfully submit that people who are focusing overmuch attention on Light Table "because Bret Victor" are mostly missing the point. As Bret points out, there's a lot you can learn about these things from existing (even old) systems.
If you already know Smalltalk, Logo, HyperCard and Rocky's Boots (I'd missed out on this one but it reminds me of Robot Odyssey which I did play), you could do worse than go and play with LambdaMOO for a little while.
(ETA: it turns out that Roboy Odyssey was a sequel to Rocky's Boots.)
(And its invaluable companion service, online registered mail, for documents with legal significance.)
Hackers hate ad hominem, and with good reason. I too subscribe to the school of "harsh on the problem, soft on the person". On the other hand, it makes no sense not to call out things that keep us in the swamp, or to tiptoe around important epistemic issues just to spare hurt feelings.
One problem we have is that few people are willing to go to great lengths to check out the available evidence; instead the pattern is to repeat claims (and associated citations) made by people who sound authoritative, accepting them essentially on faith. This has the unfortunate side-effect of magnifying the mistakes of people who have become authorities.
In "Leprechauns" and elsewhere, my focus isn't on what any particular person says, but on specific claims. "The cost of fixing defects rises exponentially as a function of SDLC phase" isn't tied to a particular person - it originated with Boehm but many others have propagated it. My method has been to look up the evidence and to see if it held up. Also to think hard about why studies may have failed to show conclusively what they set out to prove, and how to overcome these challenges.
I'm doing the same kind of thing in my own area, i.e. I'm collecting all available evidence, pro or con, about whether various Agile practices "work".
> is there any finding in the software engineering literature that you think holds up
There are many good ideas and rules of thumb, but when it comes to very general "laws", solidly established - that's harder. I've been asking that question over and over again, hoping to get a convincing answer. Still haven't got one.
I've read a bunch of supposedly solid references, e.g. Pressman, or the "Handbook of Software and Systems Engineering" and have been underwhelmed. (The first "law" proposed in the Handbook: "Requirement deficiences are the prime source of project failures," based on evidence like the Chaos Reports. My rebuttal: http://lesswrong.com/lw/amt/causal_diagrams_and_software_eng... )
Author of "Leprechauns" here.
I'm not sure I follow you in the above sentence. What happened time and again in my investigations was that I would find a citation (in McConnell or in Boehm or elsewhere) that was metaphorically accompanied with the statement "this here is a solid piece of land in the swamp".
When I got there, however, what I found was, in fact, just more swamp.
It takes a higher standard of proof to demonstrate that something is solid ground than to demonstrate there's a flaw in it. That may be unfair, but it's why scientists need a lot of training.
To take just one example, Graham mentions the Hughes Aircraft study, cited (by McConnell and others) in support of the usual "exponential" curve for defect cost (as "Willis, Ron R., et al, 1998").
When you actually read that paper - which is both one of the easiest to obtain and one that has the more detailed data about what was studied and how - and look for the raw data, you find numbers that do not obey an exponential law OR a monotonic increase. For one column for instance the numbers in fact vary within a narrow range .36 to 2.00 with two maxima at the "Coding" and "Functional test" phases, dropping off before, between and after. In a different column the costs vary only by a factor of two between Unit Test and System Test, by a factor of less than four between Code and System Test. The exponential rise is generally not true for the pre-1991 period; some post-1991 measurements get closer, but many do not.
It's hard for me to see this evidence as doing anything else but undermining the original claim of the exponential rise as a generally reliable regularity in software development.
Given this, I don't think that citing this paper in support of the claim is appropriate.
There are 48 papers tagged with "tdd".
Redditor 1: "I bet not a single f$%k was given about that table among the readers of the book :)"
Redditor 2: "That table, or something like it, features strongly in every software engineering textbook that I've ever seen. The numbers differ, but the increasing order of magnitude differences are roughly the same. It's the essential justification for nearly everything in software engineering. "A single f$%k" doesn't begin to describe the importance of that table."
And yes, "the essential justification for nearly everything in software engineering" is broadly correct. Anytime you see an argument that "it's important to get the requirements right" for instance, it's based on studies that purport to show these numbers.
(Yes, that's my book.)
Things like the Cone, the rising-cost-of-defects or the 10x claim have been kicking around for decades.
The evidence for or against TDD is, admittedly, inconclusive, but it's more recent and of a better academic caliber. There have been a lot of studies. Most of these studies aren't any good - but at least someone is trying.
There's a deeper question, which is "granted that all the empirical evidence we have so far for claims in software engineering isn't all that good, how can we get good empirical evidence?"
I suspect that the answer is going to involve changing the very questions we ask. "Does TDD work" is too fuzzy and ill-defined, and there's no way you can test that in a blinded random experiment. People's biases about TDD (subjects' or experimenters') are going to contaminate the evidence.
Instead, we need to ask questions that aren't susceptible to this kind of bias and contamination. For instance, we might want to unobtrusively study actual programmers working on actual projects, and record what causes them to write defects.
1. What are the most important problems in your field?
2. Why aren't you working on them?
(See http://www.cs.virginia.edu/~robins/YouAndYourResearch.html)
(It's very tempting to reply with more multiple answers, in the form of a goal-subgoal tree. So for instance nodes in that tree might be "Help people think more effectively about their life situations", which in turn is connected to "Help people acquire better critical thinking skills", which, given my life history, has the greatest leverage when connected with the node "Help people become smarter about software development". But this kind of tree is dynamic; as you acquire new skills in the pursuit of intermediate nodes you often reprioritize nodes of the same level.)
I'm amused to find that this has been written up as a pattern, the Memory Overdraft Pattern: http://www.charlesweir.com/papers/Mempres8AsHtml.html
The "original" (FWIW) coding dojo was aimed at adults, but we've had something going where we code with kids as well, for a couple months now.
Do get in touch!
Given my specific issues, he citation indices are a bigger concern to me than the publishing portion. If you can address my concerns I'll probably end up using your product. It looks interesting enough that I'll probably give it a test drive anyway.
I'll try contacting you off-HN if you're easy to find.
Roughly, as I understand it, a researcher's annual performance review is based on the number of papers accepted by the "right" journals, and in turn what determines the "right" journals is their standing with the big science citation indexes such as ISI and Scopus.
The latter in particular gives lots of detail, not just in describing the situation (the imbalance isn't the same everywhere, in every company or in every job description) but also in in explaining how (historically) and why (causally) the situation became what it is today.
> Helms and Valberg lay out an innovative methodology that finally delivers on the promise of Agile Development.
I've been reading English since I was 13 or so - quite possibly the bulk of my reading has been in English (a quirk of mine).
Part of me wants to agree with you. Richard does "name names" and directly voice an opinion of another person's abilities and a judgement of their actions.
One of the comments in this HN thread reads: "Get off your high horse you fucking moron". That is abusive. Never mind that this comment is in response to someone I happen to disagree with.
On the other hand I did not find the "cutting off his fingers" comment abusive. It comes across an elaborate, possibly overwrought, way of saying "This guy is making me angry". It is OK in my book to give people feedback on how their actions are making you feel. The imagery is violent, the tone is not.
What I do find abusive is the phrase "people like you" which Richard uses in the same comment. So my considered opinion of these comments would be yes, they are somewhat abusive. But I would stop well short of "bullying".
Richard's attacks have a clear object-level target, which is not a person: he is clearly making legitimate (not necessarily true, but legitimate) claims about the TB development process - that it breaks features that worked, that the concerns of some users are being summarily dismissed, and that there is a pattern of these things happening over and over again.
Yes, the tone he uses for saying those things sometimes crosses a line. But on balance, the impression I'm taking away from this dispute is that Richard is being banned not so much for his tone, but more for saying things that are true and that the people wielding the power to censor him are unwilling to hear.
That aspect of the whole mess, the way the power asymmetry plays out, is what I suspect has people coming out in Richard's defense. People with ban buttons should usually err on the side of tolerance.
The success of open source rests on participation, and participation is a two-way street. When you open a bug reporting system, the intended message is "we care about your concerns". Marking a bug as "won't fix" sends the exact opposite message. That makes it, too, a power not to be used lightly: someone wielding that power indiscriminately can easily turn into what I'd not hesitate to call a "bully".
At some point these tensions stopped being about mere civility, and started being about the power politics of open source. At least, that's how it looks to this outside observer.
So I'll add two:
0. talk to other programmers 11. attend a Coding Dojo