The Making of Software Engineering Myths
modelviewculture.com
modelviewculture.com
the exponential cost thing is probably somewhat true. If the people who come up with the software actually knew what the fuck they wanted they would save the software from having to guess wrong and redo the code. Unless the software runs on a million dollar jet or something finding a bug during testing should not be 100x more then finding it during code. Infact I can make a decent argument that it is cheaper to pay a tester to find bugs then to waste an engineers time looking for them.
the software crises story about defence i think is real: http://www.reuters.com/investigates/pentagon/#article/part1
In this case, the 10x productivity boost comes from incurring technical debt that they personally will never have to deal with. Many organizations need these people in order to actually ship stuff, but their productivity comes at a cost that's paid for later, by other people. Maybe they're 10x up front, but it ends up being more like 1-2x when these costs are taken into account.
I'm more worried about the much more tangible 0.1x (or the feared but very real -1x) engineers, who exist in many organizations.
Oh, new rule! We can concede there are "10x" devs, but they have to write shit code! Give me a break.
But couldn't it be that those "real" 10x devs are so rare, that they aren't a "thing"?
With real I mean those who don't write shitty code.
By my count, I've worked with 6 of these mythical creatures at 3 jobs in 15 years. Is that rare? I don't think so. The fact is, in a lot of cases the team or even the entire company would not even exist as they did, without these people. And everyone knew it because it was blindingly obvious. What would id software have been without John Carmack? Did you know that the first version of Microsoft Math, which included a fully general algebra solver, was written by one guy as a side project? Nobody else on that whole team did anything like that. Just that guy. The 10x dev is the person all the other devs are intimidated by, strictly on the basis of the code they've written.
But I never met one like him.
To me, such people simply are far away in some big companys across the sea.
I always hope to meet such people to learn from them "how to do it".
There are more types of extraordinary developers, but 10x is the only one I hear consistently mentioned.
What I find most interesting is that Laurent's work is clearly about the pervasiveness of myth and the need to apply rigorous thinking to what we believe. It's not about the specific claims; those are (fascinating) examples.
And yet, every time, people get really worked up about the 10x issue. They ignore the broader point entirely and focus on that one claim. (I find it particularly interesting that they focus on 10x and not any of the other claims Laurent debunks. Perhaps because it's easily understood? Perhaps because it's easy to think of anecdotes? Perhaps—and I suspect this is the real truth—because they secretly believe in their hearts that they're among the Chosen, and Laurent's threatening their clubhouse?)
Laurent doesn't say, "there are no 10x programmers." He says, "the studies that claim 10x programmers are so weak as to be laughable, the notion of measuring productivity itself is controversial and poorly defined, and do you even know which ratio you're using when you say 10x, anyway?" (He's not quite so informal about it, but that's the gist.)
In other words, the 10x claim has yet to be defined, let alone subject to rigorous analysis.
But out of the woodwork come these people who say, "I've seen 10x, therefore it must be true!" (I can't help but believe that these are the same people who jump up and down saying "the plural of anecdote is not data.") To be blunt: irrelevant. Laurent didn't say "there are no 10x programmers." Sit down already. Read it again.
You know what? The 10x claim doesn't matter. Laurent's not even saying it matters. His argument is bigger, more interesting, and more valuable than that.
He's saying that people—us programmers, specifically, who so wish ourselves to be rational thinkers—believe things because we want them to be true. Not because of the "evidence" and "proof" we're so keen to demand when we hear something we don't like... but because they're the myths that surround us, and they make us feel good.
(And now I hope morendil will chime in and tell me how I've misinterpreted him. ;-) )
Could be my fault, could be that more people - on this site in particular - ought to be familiar with pg's Disagreement Hierarchy: http://paulgraham.com/disagree.html
...or with its even more powerful cousin http://lesswrong.com/lw/85h/better_disagreement/
Same goes for so much that passes as argument on Hacker News, especially on topics about which participants are largely ignorant.
x10. Anyone here do the TopCoder Algorithm SRMs? There's clear evidence of x10-ish performance difference between the "red" coders and the average coders in there. Does that always apply? No. In certain situations can certain developers blow the average away? Absolutely yes. This is true for things other than software development.
The author talks about "engineers of equal experience". No such thing. There are no two single developers with "equal" experience.
Bug fixes. It's a no-brainer to see the cost of fixing an issue by the same dev in reasonable proximity to when the code was written is a lot cheaper than someone else fixing that bug 6 months later. It's also a no-brainer to see that the cost of fixing a bug in nuclear reactor control software that can cause a core meltdown is cheaper before the core actually melts down. In between there is a lot of grey area but I think there's a lot that can be applied while still being safely in the no-brainer region, that is our intuition, that is our learning/pattern recognition over years of software development.
That logic seems to be all backwards to me. Of course developing software is a highly abstract, intellectually demanding activity. Claiming it’s not is as absurd as claiming women are somehow incapable of doing exactly that.
I don't mean to sound harsh, but "of course" and "it's absurd" aren't much of an argument.
My advice is to dig into this a bit deeper. What do you think makes software "a highly abstract, intellectually demanding activity"? What would the world look like if that were NOT the case, what would it look like if it WERE the case?
How does this claim coexist with that, which has been bandied about for decades, that software programming is on the brink of being automated away?
How does this claim square with sites like the Daily WTF, which provide evidence that many gainfully employed programmers do not, in fact, use their brains very much? With the more or less weekly announcements of "security breaches" caused by people ignoring basic best practice, for instance storing passwords in plain text?
How much mathematical(ish) ability is actually needed to put together a Web page, a bunch of CRUD fields, and some backend database template?
Why is it so easy to get a job writing software even if you have no formal credentials at all? How is it that some people can get these jobs even if all they can do is write a Web page, and that badly?
It's possible that our experiences differ: that for you software has been abstract and demanding. (I can see people who write kernels for a living getting the "of course" reaction.) Whereas I've often rubbed shoulders with the world of "IT programming", with self-taught games programmers and Web programmers and people who turned a half-baked idea and their gift for the gab into multimillion dollar businesses.
Whenever I did my homework and dug into the facts, I found that things that "of course" held true weren't all that obvious. I'm trying to encourage people in the profession to get into that habit.
> as absurd as claiming women are somehow incapable of doing exactly that
Yes, I too see that as a false claim. Do you deny that there are people out there who are in fact making that claim?
I take many people being bad at programming as evidence that programming is hard, not that it's easy. Maybe it's evidence that getting a programing job is easy, but that's not the same thing.
We are in violent agreement. Note that "hard" is not the same as "abstract and intellectually demanding". Running a marathon is hard. Becoming a US Marine is hard. Many things are hard that do not primarily require quasi-mathematical skills.
For that matter, finding gravitational waves is hard but may be just as much about investing enough money (think space-based laser interferometry) or just plain luck.
The ways in which people suck at programming are much more diverse than just a failure of abstraction, otherwise we would all be writing Haskell.
These ways include failure to ask what the user or sponsor wants, failure to make sure we've understood what the user or sponsor said, failure to communicate with other members of the team, failure to question the things we learned in school and always took for granted. All of these are common failure modes in the biz, none of them are a failure of abstraction.
In fact excessive or premature abstraction is a widely recognized failure mode of software engineers, and the myth is responsible in good part for that.
Also, common failure modes of people working as programmers != common failure modes of people trying to become programmers. Even if most people working as programmers understand the abstractions involved well enough to function (and I'm not certain that's actually true), the abstractions may still be too much for many other people.
>In fact excessive or premature abstraction is a widely recognized failure mode of software engineers, and the myth is responsible in good part for that.
Part of really understanding abstractions is knowing when not to use them. And since, as you say, many people have trouble with that, understanding abstractions must be tough.
Now there's a good myth. Unless you really think programming could be "on the brink of being automated away" for over half the time its been around?
In which case, where can I bet against this theory?
[Edit: deleted mistaken impression here.]
Article author here. Agree with your first para, so the above has me puzzled, a bit.
Would you please expand: what impressions do you think the article gives, that are misleading? (And if possible: what do you see or hear that makes you think they are misleading?)
You place so much emphasis on dismantling the 10x claim (writing an entire book about it) rather than software research in general, that an observer might reasonably be left with the impression that there's something more "mythy" about that claim than others. If so, I'd say that's misleading.
Folklore isn't ipso facto false. If folklore gets repeated as persistently as 10x does, I'd say that speaks weakly in its favor. (Please note that I said "weakly".) Because of this, I don't think it's irrational to continue to favor that view, even after debunking those studies. For one thing, the opposite claim is even less supported by data. For another, it fits loosely with my experience and that of others I talk to. Experience in the field may not be controlled measurement, but that doesn't make it rational to dismiss it.
I do think there's something to the 10x claim about individuals; I think interaction between individuals is even more important—100x, one might metaphorically say—and that question is even less amenable to formal study. I doubt that we'll ever see convincing quantitative results on either of these things. Building teams will remain an art, and folklore and myth are the stuff that art is made of.
"Leprechauns" is by no means entirely about 10x developers.
In it I also tackle things like the "exponentially increasing cost of bugs" claim, the Cone of Uncertainty, the "software crisis", the successive mythical reconstructions of the Waterfall bogeyman, the limitations of the "empirically based software engineering" movement and specifically the problem of discipline envy.
If it had only been the 10x thing that turned out to be ill-supported I wouldn't have gotten my knickers all atwist. My beef is precisely that when you look closely most of software engineering looks awfully like pseudoscience.
This article is, in a way, Chapter 1 of the next book, in which I'd look beyond "academic" myths, and try to confront a broader picture of how the software community thinks about itself, and how that perpetuates some of the problems we've been bitching about for decades. (And some of these problems turned out to be non-existent; other very real ones barely rate a nod from academia. That is also part of the problem.)
I may have asked you this before, but do you know of any finding in the research literature that you don't consider pseudoscience? The only one I know of that might come close is research on code size. I've heard that the literature on code inspections is good, but that may just be another myth.
I also get Citogenesis. This also gets abused in many fields. Think Cargo Cults, and stories of performance improving every time the lights go on and off, and how we're all evil because people in lab tests electrocute their test subjects.
That said, it is hard for anyone who has been in the world for a while to disbelieve the 10x programmers. We all have met them. They really are that much better. And they really can solve problems that others can't. They may not be better for every situation (Example: Following a structured testing methodology that requires a lot of human input) but they do exist. It would take a lot of evidence to prove that they DON'T.
I'm about halfway thru reading it, some of the findings presented are that TDD studies either aren't very good or give inconclusive results; that a number of code complexity metrics don't predict bug rates any better than file size does; that bug rates are correlated with how well organizational structure matches code structure; and that competent PHP programmers do in fact exist.
I did like the paper on code size very much. The principle that code size is the best measurement of complexity and a good predictor of error rates is probably the finding I'd name if I had to answer my own "name one that isn't folklore" question. At least that one has multiple studies behind it. Even so, most of them (that I've seen) aren't very good.
I can see why it would be so difficult to find correlations.
And when you get there: write about it (or sing, or make movies). This, really, is the key thing. Not only do we need to talk more, not less, about how we develop software, we also need to talk about the things we say: we need critique."
if only the discussion on hacker news would actually take this advice instead of arguing arcane bullshit. Hoping a few will take this tack. Please surprise me.
Yes, sure, we're talking about exactly 10x when we talk about 10x programmers. Etcetera, etcetera.
Even the most fanatical Agile evangelist gets both the tongue in cheek humor (with a serious undertone) of Zed's "programming, motherfucker", but not this guy.
Some of the claims or statements they make in a serious manner are so far off-base that they appear to be jokes to those with more experience or higher standards.
The "JavaScript is like Scheme" claim is a good example of this. A lot of JavaScripters will openly express this idea in complete seriousness. Yet anyone who has actually used JavaScript and Scheme will know that this really isn't true at all. The true intent of the person making the outrageous claim is hard to discern. They might be serious, or they might be joking. Often, it's safest to assume that they're being serious, until they explicitly admit otherwise.
http://journal.stuffwithstuff.com/2013/07/18/javascript-isnt...
Particularly poignant is the way they redacted "autistic idiots" to "socially awkward idiots": if you're going play the "politically incorrect" card, play it with panache instead of being hypocritical about it.
It's like 1.000 game, a hat-trick or triple-double - sometimes any given player has the potential (though not all may be given the opportunity) to make their mark.
I've had the pleasure of working with very high-performing developers and the positive impact they can have on a team is amazing. I can see how some people don't believe it if they have never worked on a team where one person amazed and inspired them.