Super-star programmers
economist.com
economist.com
What really genius-level things has Fog Creek done? I remember demoing FogBugz, and it was marginally better than its competitors, but it was far from a complete game-changer. Trello is ok, but it's only marginally better than a whiteboard with post-it notes. None of these reinvent an industry, or provide something so outstanding that people rush toward it.
Don't get me wrong; I think Joel has a lot of good ideas in general, and Stack Exchange is indeed a game-changer. But as someone who continually talks about hiring "rock star programmers", I simply don't see a result that I'd consider "rock star" equivalent. But, maybe I'm missing something?
Edit: More than that, I don't even consider myself a rock star developer, just a normal developer who loves learning new things and getting a lot of stuff done. I'd be thrilled to work for a Google or Microsoft, but what would attract "rock star developers" to Fog Creek? Basically, are fancy chairs enough? Isn't the most attractive thing a super interesting problem domain?
There's plenty of rock star coding out there, of course. But it tends to be either hidden inside companies due to secrecy concerns (Google's scaling architecture or the NVIDIA driver stack, say) or exposed in the open source community instead of the startup world (Fabrice Bellard is a great example here).
Startups all say they want rock stars, but then they put them to work doing "better versions" of the same boring stuff we all recognize. Which, if you think about it, is sort of what startups are supposed to do.
I sort of snicker when I see startups post jobs looking for "rock star" coders [1]. What self respecting person would call him or herself a "rock star", let alone apply for such a position? Maybe it's just a gimmick to get mediocre developers to get to work on boilerplate web work..
[1] I also saw a job posting recently where they were looking for a coder "to work on hard problems and blow a hole through the universe"... Why not have them blow bullshit of their face while they're at it? Is this just HR trying to sound too cool, or what?
I specifically do not applied for positions that are advertised in this way.
After all what does a rockstar get ?
If someone wanted me as a 'rock star' developer, I'd want some serious profit/revenue sharing plan in place - something more than "10% of profits are held back and divvied up amongst 90 other people based on seniority".
Words never heard backstage = "That's the banjo players Ferrari", "That supermodel is sleeping with the drummer"
A huge class of such systems that are waiting to be created are crowd labor systems. How do you create leaderless organizations that produce what google or boeing or walmart produces and that compensates individuals for performance.
The answer today is a messy unorganized market system. Proper information systems that better organize that would create a lot of value. The analogy would be this site. Thousands of people communicating, without the organizing software it would be much worse (thousands of people standing around talking to each other, the good ideas spoken would not easily reach the top as it does in this organized software system).
tldr; the biggest value opportunities lie in CRUD apps, not the low level code
Yes, CRUD pays. Being able to make more CRUD in less time (and/or CRUD that is better performing or more maintainable, etc...) is a valuable skill.
But it's boring. And more, it's just not impressive in the same way that, say, qemu/kvm is (or Mesa, or Linux, or llvm, etc...) That stuff is hard, and fun, and interesting in a way that CRUD will never be. So it has value too, and IMHO it's entirely reasonable to celebrate its practitioners even if they don't make as much money as CRUD jockeys like Zuckerberg do.
It's a pity though, there's a huge space of products that are needed but the talent is going into slugging it out in societally useless and relatively low paying niches (such as many of the most talented programmers entering the game industry or social networking or linux kernels). Meanwhile dumbasses are raking in millions with shit like SAP and the like.
I don't necessarily agree with SAP or SAP consultants but there are cases where people would prefer to use SAP to implement some of their core: http://www.infoq.com/presentations/Building-a-Hybrid-Cloud-a.... SAP is a complex, specialized tools that requires specific knowledge on how to tame it.
(I'm not arguing if complexity is good, bad, or anything like that. I'm focusing on what SAP is good at).
That might be a bit contradicting to what Joel is explaining. From what I've experienced first hand, there's programmers that can produce similar code/software in a week that takes 5 mediocre programmers to produce in a month. Those are extreme cases though.
There's not enough "rock star programmers" to go around, so not everyone can have a piece of that cake. Much of this is also very domain specific, it's not a given that a "rock star" game programmer is going to be a orders-of-magnitude better at developing business support systems. Finally, this is not that specific to programming. Not everyone's Einstein, Feynman or Michelangelo.
This idea that a single programmer is somehow so much better or superior may have its merits, but it's harmful to your overall business if you put all your weight on that single support.
You have to accept that above a certain number, your employees will fall on a bell-curve of ability, and you should support the efficiency and productiveness of the entire system, not depend on a few "rock stars" to drag it up with you.
Here are words I associate with Rock Star: driven (sure), but myopic, single-minded, selfish, unbalanced, and unfaithful. I say we strike it from the vocab.
In the rock star analogy, it'd be like getting the Beatles to write music for Justin Bieber fans, or getting Metallica to switch to the "easy listening" genre. That's not what you do with rock stars.
You can't stop people like that, the same is true for programming, if it's your thing you'll do it. If not.. you won't, you'll be a session musician, and every tune you hear (or function you see) you will be able to say "I could of written that" but you didn't.
Made himself independently wealthy. Repeatedly (IE: Not luck).
Not that I disagree with you. But that is worthy of respect in itself. Also, there is an art to the very difficult task of making the difficult look simple and easy.
All success is based on luck to some extent. It's silly to argue otherwise. Of course there is skill involved as well.
To quote The Spolsk himself: "This paper is the ultimate rebuttal to those grumpy people who email me, barely able to conceal their disgust, saying, 'why do you need to hire such smart people to work on bug tracking software?'"
"Don't get me wrong; I think Joel has a lot of good ideas in general"
The saying that comes to mind is perhaps the following. "Those who can, do, those who can't teach." I'm guessing that Joel, by his writings, is a combination of the two and as a result his output is not "rockstar" quality.
Another example might be Steve Blank. Who no doubt achieved more fame as a teacher then he ever did creating anything.
There is obviously quite a value in people who can do both in varying degrees.
To do something compelling you have to want to take that risk (which could result in spectacular success or failure). Joel might not want to bite off more than he can chew or he simply might not be creative enough to come up with anything "more compelling domain than bug tracking and PM".
That is why a good programmer needs to be around good business people (add: as well as people with ideas and a feel for the marketplace). They are two different skills. Same with medicine. Doctors specialize. You don't find that someone is the best thoracic surgeon and world famous also does urology. Maybe he also dables in photography but he's not going to be as good as Ansel Adams who does it full time. Business is a skill and takes full time attention. So does programming.
Look at PG. Running YC is a full time job. While he can certainly do programming on the side now and maybe write a book you're not going to tell me he can do all three better than a specialist who can devote full attention.
Partnering with a business person for my next venture is something I'd like to try, but it's not exactly clear where you find a good business person who wants to partner up. The business people I am in contact with are already busy with their own things.
Unlike a programer who might be presently doing the 9-5 corporate job, it seems like a good business person will always be busy doing their own businesses, as that is what business people do.
I have been mulling over something in particular as of late, but I presently don't have much interest in building it if I'm also going to be responsible for selling it. A partner seems like it would be a good fit.
My first question would be to ask you where you are located? Not sure if you want to reveal a specific city or not. Where you are matters.
"The business people I am in contact with are already busy with their own things."
True. Which is why you need to be somewhere where there is a chance of meeting people who are in between, recently graduated etc. There is also the possibility of someone who is employed full time starting to help and give advice on the side. When/if the venture takes off they would feel more secure in leaving their permanent job.
"it seems like a good business person will always be busy doing their own businesses, as that is what business people do."
You only need to find one person (it's like getting married). There are of course compatibility issues as well to consider.
"have much interest in building it if I'm also going to be responsible for selling it."
If you build it and it makes sense from a business perspective and have a prototype you can attract that one or two people you need. With just an idea of course you can't. Care to elaborate on the idea at all?
Rural Canada, which I assume does not bode well. There's certainly no startup scene, so to speak.
Where you are matters.
Which is unfortunate because one thing that has always excited me about software in particular is that it democratizes location. I've had the opportunity to work with some amazing organizations from around the world without having to be physically present for any of them.
If you build it and it makes sense from a business perspective and have a prototype you can attract that one or two people you need. With just an idea of course you can't. Care to elaborate on the idea at all?
That's fair. I've built "hundreds" of prototypes before. Some I've come to realize are simply bad ideas, others I've decided were still good and were taken to market with some mediocre success. In those cases I have felt were limited due to my lack of business acumen. What is not clear to me is where to go even after a prototype has been made and still shows merit.
I don't mind sharing the idea, but after some further research I discovered some companies who are already working on pretty exactly what I had envisioned, so I'll spare you the details. Though I do appreciate your advice in general as there are millions of other problems that need to be solved so it will no doubt come up again.
I think this gets to the heart of your bias. "Bug tracking and PM" is not compelling to you, but I suspect Joel and his hires find it a lot more fascinating than you do.
Not to answer someone else but I think what the parent means is that it's not original, unique, or greatly improving on whatever is already out there. (Is it?)
I mean it's kind of meat and potatoes addressing a problem that has been addressed and maybe doing it better.
Let me state that while I can program I am certainly not a programmer. But from my perspective it seems that Fog Creek, while a great company is not the place that a world class programmer would go to work on world class software. I mean this as no disrespect to Joel (who I admire greatly) or what he has done.
Using an analogy if you want to be in law enforcement are you going to work for the local big city police department or try to work for the secret service or the FBI?
> Using an analogy if you want to be in law enforcement are you going to work for the local big city police department or try to work for the secret service or the FBI?
Do tell me what makes the FBI or the Secret Service more "compelling", "original", or "unique" than a "local big city police department". Because I see law enforcement, full stop. Not Swanky Law Enforcement versus Ghetto Law Enforcement.
When I think of a rock star, I think of someone who not only produces stuff of quality, but actually changes the landscape of the industry. If FogBugs hires 40 rock star developers, I'd expect to see a lot more come out of them than products all in crowded spaces. While they are all quality products, they aren't changing the landscape of the industries involved.
And you know what? It's all about the same. It comes down to things like being respected, challenged, engaged, and working with people you like being around.
If Spolsky were to go slumming it and open an office in my town, I would be first in line to apply to work on FogBugz, and I don't care about the fancy chairs.
- god this is long. Isn't it ironic that a superstar programmer would have been bored by now and stopped reading.
- momentarily after that, I guess I just admitted that I'm not s superstar programmer.
- momentarily after that, I wonder if there are writers who can transmit ten times as much information using one tenth of the words.
- just now, I imagine magazines don't want superstar writers. That would diminish the amount of ad space they could sell.
I tend to do this somewhat subconsciously when I start to realize the content is sparse.
I'm reading it right now and, if nothing else, can totally relate to "I can't concentrate on things very well anymore" feeling.
I like how Carr's own sentiment mirror the sentiment of people who thought books will stop people from remembering things. And he wants us to return to books?
He's simply pointing out evidence that our tools change the way we think. He makes the same point about clocks, maps, books, etc.
The book has zilch to do with societal decline in the sense you mean it.
For example Paul Graham's essays are often very insightful, but they are frustratingly long. The core idea, which is the only reason why I am reading the essay, can be summarized into 1 - 3 sentences. The rest is just a boring extrapolation and repetition of that.
First time PG has been denounced for verbosity?
This suggests to me that the biggest influences on productivity for the organisation as a whole are probably to do with collaboration, culture and environment rather than extraordinary individual skill.
Maybe the very best programmers aren’t extraordinary just because they can solve the most difficult problems, but also because they are better at presenting their solutions in ways that are more accessible to other developers. That would have the twin benefits of improving productivity in its own right and of implicitly teaching by example so less experienced developers in the organisation would tend to develop the same skills themselves over time.
If you had a seed of programmers on that level and a culture that promoted mentoring, pair programming, code reviews or similar collaborative exercises, it’s not hard to imagine that such an organisation would develop significantly higher collective performance over time than another organisation without those benefits, even if most people joining each organisation were of a similar level of skill and experience to start with.
Not a complete understanding, just a basic idea.
This is why mediocre programmers are what they are---they lack this ability and end up "flailing around", looking at somewhat random issues, bouncing back and forth, unable to see the pattern, and so end up spending more time writing code that eventually has to get rewritten.
A great programmer will systematically take a problem apart so that even if they don't solve a problem in one minute, they've at least narrowed down the window of options for the future.
This is also why great programmers are always great debuggers.
For me, there are days of brilliance and days of complete distraction. I won't call myself a 10xer but I think much of my success has come because I truly love what I do.
EDIT - I couldn't find a reference to the study I was remembering, but Steve McConnell of "Code Complete" fame has written a blog post describing how flawed the studies he's reviewed have been - http://forums.construx.com/blogs/stevemcc/archive/2011/01/09.... There are links to a lot of interesting sources and he clearly isn't arguing against the idea of dramatic differences in programmer ability ... just sayin'.
Can I pay attention to the details without losing sight of the big picture?
Can I do a good job technically, but keep within any other project constraints like budgets and timescales?
Can I write code that is good enough to ship tomorrow, but maintainable enough to work on again next year?
My best work tends to get done when I figure out the right balances. If I let one aspect become too dominant and neglect something else as a result, that’s usually when the problems start.
This requires a high level of aptitude and experience (which comes with practice).
However, I feel the ability to write quality code quickly is just one facet of a great programmer's skillset.
Often overlooked is the ability to communicate and collaborate with others effectively--which is just as important.
With complex projects, I oftentimes find that the solution to my problems has already been solved. Sometimes the best code is no code at all.
I would argue that this presents several sub-skills:
Being able to do this quickly b/c you don't have to spend the time learning
Being able to do this effectively -- by not painting yourself into a corner b/c you well versed in best practices and anti-patterns with using said 3rd party technology
Being able to do this smartly -- because you are well-versed in a lot of the different options, you can effectively way their pros/cons, and make smart decisions on which 3rd party stuff to use, and when to build vs. buy
I think these skills are more effective (towards your end goal) than simply being able to quickly code some brilliant labyrinth of an algorithm in less SLOC than a 1 or 2-xer. I feel like the emphasis is always put on the latter, and these skills are overlooked.
Combine these with effective communication, and then you are talking about someone who can get things done.
That type of agility is hugely helpful in an organization.
(Still, working is more important than talking about it.)
Some people have objected to the "10x" label, making an argument like "We had a super programmer on our team once. He was so obnoxious he alienated the whole team, and overall productivity was actually better without him.
In general, any practical definition of a real 10x programmer has to include the effect the programmer has on the rest of the team. I have known super programmers who were obnoxious. More often, a supposed super programmer with an obnoxious personality was really an average programmer, or worse than average, using an obnoxious personality to hide a lack of performance. The true super programmers I've worked with have generally also been cooperative team players, although of course there are always exceptions."
Taking his blogs at face value, and assuming that my sample size of 1 is enough, here's what I've got:
"Superstars" get things done because they get things done. They may be ADD in their multitude of projects or in their personal lives, but while they're producing, they produce. They don't fuck around on reddit while they're coding. They might screw around interminably at other times - they might work only a few hours a day - but while they work, they're working, and that's it.
They also get things done because they understand what the hell's going on, at least one or two layers of abstraction below the one they're officially working on. Stevey claims that his understanding stops at transistors - transistors are black boxes, but he gets what has to happen at the layers of abstraction above that, from the machine code to the guts of the interpreter.
Particularly delicious? Steve Yegge worked at Geoworks (http://steve-yegge.blogspot.com/2008/05/dynamic-languages-st...), and he attributes their defeat by the likes of Microsoft not to superior marketing, but to the value of abstractions.
http://steve-yegge.blogspot.com/2007/12/codes-worst-enemy.ht...
10Xer imples 10X some base level. Depending on what base you choose, Yegge can easily fall short.
I "worked" on Wyvern in high school (in that I played a lot of Wyvern and made a few terribly shitty game areas and ended up being an administrator, or "Wizard" in classical MUD parlance).
You know how Dwarf Fortress, that ASCII game, simulates a whole lot of ridiculous detail? Over-the-top, "the goblin's spear strikes the baby dwarf in the back upper left tooth and it really hurts so he cries for mommy"? Wyvern didn't quite have that, but the API implied most of the functionality for it.
You didn't have inventory slots, you had body parts, which brought with them the ability to wear or wield certain other items.
You could've written Wyvern in a much simpler fashion. A bright high schooler could make a game that had all of Wyvern's functionality at the end of its life. It's the mountain of "do a lot of other shit later" that got in the way.
And that's what he's writing about in that blog post - not that he's perfect and Java sucks, but that he was trying to employ industry best practices and make everything perfect without compromises. It's not very hackerish, and sure, it was stupid. But the guy wasn't doing this for his day job and didn't have defined deadlines. He was a programmer, in other words, not a project manager.
And that's where he fell down. Project management, not programming. And if you read back through his massive wall-of-text blog archives, you can see him mature as a programmer and project management.
(Please note that I am an avowed fanboi, if you didn't catch that already, and as such everything I say must be taken with a grain of salt.)
"How strong is the support conferred to the 10x claim by the best-reputed list of references, for a reader persistent enough to follow the chain of citations back to primary sources?
"Based on our close reading of the “10x files”, we can now answer: quite weak.
"Not a single one of the references is to a replication, in the scientific sense of the term, of the original exploratory experiment.
"The empirical data is in general quite old, most if not all of it predating widespread use of the Internet - which we can safely expect to have wrought major changes in programming practice.
"None of the studies address the question of construct validity, that is, how meaningful it is to speak of an individual programmer’s productivity, and if it is meaningful, whether the experimental measurements line up adequately with that meaning."
Don't get me wrong--I've certainly seen 10x (and greater) variations in productivity. But they've been due to factors other than inherent individual capability. They're things like
* technical debt
* familiarity with the codebase and problem domain
* interruptions, distractions, multi-tasking and other office environment issues
* choice of language - but this is a distant fourth compared to other the other three
On top of all this, the Economist article is a thinly-veiled press release for tenXer, which is itself nothing more than an exercise in testerone-fueled egotism. tenXer measures activity, not productivity [2], and I expect the primary result of increasing your tenXer metrics will be less time thinking, more time hacking, and insufferable smugness.
[1] The Leprechauns of Software Engineering explores what science is and how we distinguish between fact and folklore in software engineering. It specifically explores the 10x claim, and determines that it's folklore. http://leanpub.com/leprechauns
[2] Productivity is defined as output over input, and the output of programming is software capability. No one's come up with a good measure of that yet. So any time someone claims they can prove increased productivity, ask them how they measure it. Chances are good you'll get a bogus value that measures activity instead, such as "lines of code." http://martinfowler.com/bliki/CannotMeasureProductivity.html
They found 10x differences in time spent on the average coding war (which was typically a pretty small sample) with those finishing faster typically producing programs that worked better. So by any reasonable measure, at least 10x productivity.
There is your 10x, right? Wrong. They found that the best predictor of programmer performance was the performance of another programmer at the same company. Furthermore they managed to correlate a lot of that performance factor to specific factors, such as having a phone that turned off, a private room, adequate desk space, and so on. There was still something like an unexplained factor of 3 left over, but they didn't know whether it was environment variables they had not looked at (eg training), individual ability (which might be correlated across institutions), etc.
It is true that this research happened in the 80s, before the Internet was widespread. I would love to see it replicated again.
(Note, I'm aware of the existence of many other poorly controlled studies, see http://www.flownet.com/gat/papers/lisp-java.pdf for an example, but what is critical in the Peopleware study is that they acquired information both about productivity and about factors that might cause productivity differences.)
I will jump the shark and speculate that these gel-members tend to be female more often than male, and the gelling-like effect comes from how their presence and patterns of behavior influence the group dynamic. I will further speculate that the dearth of women in development incurs a hidden cost to productivity, since statistically, fewer teams will have women, and consequently, the incidence of gel-members will be lower, and teams more fragmented.
I think there is a business opportunity here somewhere, or maybe it has already been realized in places, but the players have not been eager to trumpet their results.
1) Small.
2) By themselves.
3) Well understood and documented.
4) Doesn't have any threat of changing out from under them.
In the real world, the bottleneck usually isn't writing code, it's writing code that does the right thing and making the right people happy in the right way.
In some ways, trying to apply the lessons from Peopleware in the real world is like using your knowledge of unicorn anatomy to predict the Kentucky Derby.
Most developers suck at programming (and that's actually OK). 90% of the code they write is copy and pasted from elsewhere in the codebase or from the web, and then tweaked until it does something approximately correct. They usually need help when solving a new pattern of problem. What they do often have is domain knowledge not related to programming, and produce just about acceptable results out of the machine in a labour-intensive way that doesn't really scale over time or problem size, but they muddle through with determination.
I don't know about a 10x productivity differences though. I'm not really arguing in favour of that hypothesis. Rather, if anything, I've seen step differences in potential; that is, there are developers that can do things other developers can never do, not in 10x the time, not in 1000x the time.
I don't entirely disagree with you, but watch out for the Fundamental Attribution Error: http://en.wikipedia.org/wiki/Fundamental_attribution_error
But the work they did? It would have bored me to tears. I would have quit had I had to do it; or perhaps, I would have surreptitiously built a framework or macro system or something to remove the redundancy and repetition from the tasks and indirectly reducing the tedium of the workload, rather than doing it more explicitly in what I was actually doing (developing the v.next framework for the company). But there's only so much redundancy you can squeeze out of many of these things (turning business rules from specifications into code) before you create too much complexity elsewhere. We still need lots of these people doing this boring work (most software development does not occur in software companies), and it's good that they're expert in the business rules they're translating. At the end of the day, it's OK that they're not great at producing code.
Of course, my opinions are completely invalid because they are generalized from limited experience, have no scientific value, etc. But it's the world I've lived, so I'd have to see decent studies and censuses to convince me otherwise.
Let me explain. Productivity is defined as output over input. Programming input is activity--usually, time working--and output is software capability (or, more generally, business value). As architects, I'll bet that you didn't produce much that the company used. You probably produced designs, frameworks, or other tools for domain programmers to use. But the real software capability was produced by those domain experts you're criticizing, which means that they were the important actors and you were assistants at best. At worst, you could have been an impediment; I've seen "core" teams whose architecture-astronaut frameworks actually have to be subverted in order for the domain teams to get their work done.
I'm playing devil's advocate here, and I'm not entirely serious, but it's with a point: skill in coding isn't important unless it leads to tangible business results. In the short term, in my experience, business savvy outweighs coding skill.
(In the long term, technical debt becomes an overwhelming cost, so programmer skill does matter. But keeping technical debt low is more about maintainability, tests, and boringly understandable design than the kinds of things I see "rock star programmers" emphasize.)
I tried to be specific in what I said; I said that it was OK, and good, that these people existed. I was not making an argument that these people are necessarily less productive; but rather, that there was a qualitative difference in how they approached code that made a difference in what they could accomplish with it.
(Though I think in the medium to long term, they will end up less productive, because of a lack of abstraction and leverage of existing code beyond copy and paste. Probably, we don't disagree that much.)
Let alone the "business value" of an individual programmer's commits.
I think "business value" is a particularly bad phrase because it sounds like something that can be quantified and has a clear implication about which class of people will do the quantifying (namely, business people as opposed to technical people). In reality, it can't, and the term is really about power dynamics within companies.
I don't know why you say business value is actually about power dynamics, unless you mean that people use the term to promote their pet projects. If so, I think that has more to do with human nature than the term "business value."
The power dynamic aspect is this: who gets to decide what has "business value"? I would like the answer to be: anyone who has good insight and can convince their team. But when the answer instead is "a class of people called 'managers' or 'businessperson'", that is what I object to. Membership in that class is a matter of organizational status, not insight or merit.
But in a message board discussion it's hard to tell where we're talking about the same things and where we're not. Wish there were an easy way to get semantic diffs.
If it makes money, it has business value, in my book. If it makes MORE money, it has more business value.
Actually it might be the ONLY measurable thing in the whole discussion.
For business value to be measurable means you have a function M(x) that maps x to money. The trouble isn't with the range of that function, it's establishing what the domain is and how to compute M.
People often talk about business value on software projects as if they have such an M when they don't. What they have is an imaginary M whose domain is "what I care about" and whose output is "what I want to believe". Such discourse is dysfunctional.
The truth is that you have M for the business as a whole and maybe for specific products (though I can cite cases where even that was dubious) and it typically breaks as you try to get finer-grained.
I think the 10-50x observed increase in productivity is task specific. But, people also deal with a finite list of specific problems so what if bob is 2x as productive on random tasks if there is 10 years worth of work where he can be 10x as productive he is worth a lot more than the average programmer.
I haven't read the ebook you're advertising, but a book with an identical pitch, Robert Glass' Facts and Fallacies of Software Engineering, is an example. The difference between his facts and fallacies is that the "facts" tend to have one poorly controlled, never replicated study (usually from 1978) to their name. Better than nothing (maybe), but close to nothing.
My point is that at this stage in history, it's all still folklore. Even you, immediately after playing the science card, followed with mushy folklore of your own ("I've certainly seen...").
Edit: come to think of it, your quotes make it sound like there is empirical evidence for the 10x claim, just that it's old and wasn't replicated. The "old" criticism is weird; it's not at all obvious that the internet invalidated the data. So the critique seems to boil down to: there was a study, but it wasn't bullet-proof and it wasn't properly replicated. What studies of software development isn't that true of?
I'll admit to being guilty of using anecdotes to describe my experiences with software development. (That's not quite the same as folklore, but whatever.) You won't find me claiming that my anecdotes prove anything, though, and I try to be clear that my perspective is based on experiences, not objective truth.
[1] For example, this comment three years ago on reddit: http://www.reddit.com/r/programming/comments/7dkjx/rescuing_...
Another paper I like is Little's report on data from projects at Landmark Graphics [1]. It's not a controlled study, and shouldn't be treated like one, but it also looks at a large base of data from real projects.
My biggest complaint about academic papers is that they're often conducted with students, using practices out of context, working on projects of trivial size and depth. I like both of the above not because they're perfect, but because they get beyond those problems.
[1] http://www.toddlittleweb.com/Papers/Little%20Cone%20of%20Unc...
I'm immediately suspicious of the statement you quoted: "Not a single one of the references is to a replication, in the scientific sense of the term." Strict replication is not and never has been a requirement to good science. As a trivial example, how do you replicate a supernova observation? You can't. But you can make models and test the models against the observed data and against future supernovas, and you estimate the reasonableness of the model effectiveness.
If that quote is indicative of Bossavit's views, then he is a poor judge of what's good evidence.
Going on with more quotes you gave, "which we can safely expect to have wrought major changes in programming practice." That is a conjecture, not a statement of fact. It seems like he's saying that once upon a time there was 10x difference, but the Internet makes everything different now there isn't. However, without evidence to back it up, I don't know why I should believe it.
What tests has Bossavit carried out to show that this is the case?
To the contrary, L. Prechelt "An empirical comparison of C, C++, Java ..." was both carried out during the internet era, was done with good rigor, and shows a large spread in the overall time to complete the project. That's a small project, granted, but it's another data point in the overall trends.
(If you don't like the small project size, another example, given by McConnell, is the large productivity difference between the Excel and Lotus 1-2-3 teams, which produced roughly comparable spreadsheet programs but with about an order of magnitude productivity difference across several different metrics.)
Multiple independently determined data points which all trend in the same direction strengthen the likelihood that the ~10x hypothesis is a useful model. That's what scientific evidence looks like.
McConnell even has a section in his chapter on "how meaningful it is to speak of an individual programmer's productivity", along with all of the caveats and cautions that everyone here is bringing up (some programmers produce negative lines of code, the best programmers tend to get the hardest problems, the effect of teams, etc.), and concludes "So while I see the value in measuring individual performance in research settings, I think it's difficult to find cases in which the measurement effort is justified on real projects."
Edit: I see that Bossavit wrote in response to McConnell's chapter. McConnel's reply to Bossavit is at http://forums.construx.com/blogs/stevemcc/archive/2011/01/09... .
Rather than try to defend someone else's words, though, I'll just direct you to the book. You're going into a lot of detail critiquing quotes from an end-of-chapter summary, which isn't fair to Bossavit's work. I hope you'll read the book, because I'd like to hear your critique of the full work.
I can be convinced otherwise for this one case. Could you elaborate for me on why my critique on these points is unjustified? Why is sufficient scientific evidence for to Bossavit? Why does the Internet necessarily discard older studies, and how does Bossavit establish that that's the case?
"Within individual firms, the difference in performance was only 20% or so."
Is this because good developers congregate together, as suggested, or because having at least one good developer raises the productivity of those around him or her? They can certainly influence an organization through their choice of platforms and tools, aiding the productivity of everyone on the team.
Also, taking advantage of existing software makes a huge difference in 'productivity' at least as far as business people can measure. And someone who does that without acknowledging that they are doing it may appear to be a 'superstar' to management.
For example, someone may just consider all of the source code they have ever written, regardless of the circumstances they wrote it in, to be a code library they can copy and paste in whatever company they currently work at. And not even acknowledge that they are reusing code they wrote years ago.
So that sort of dishonesty irritates me, but generally I think taking advantage of existing software is the right thing to do and is the number one factor in what non-technical people would perceive as productivity. There is a huge difference in the time required to build a system using an easily configurable base like WordPress with plugins or components or by integrating open source software or libraries versus building various parts or all of the system from scratch.
That's where you will see orders of magnitude difference in productivity where one person or group is reinventing a bunch of wheels that another group is not. And again there are many different ways of avoiding reinventing wheels, from using Google to basing software on existing open source programs to selecting more practical application programming languages/tools (like ones with things like garbage collection or more straightforward syntax or support for interactively configurable graphical components/plugins), or for example doing test driven development or in general having any type of automated QA rather than relying on a manual QA process entirely.
Whoa. Either I'm a worst programmer or this is clueless.
http://plus.google.com/110440139189906861022/posts/84784VhCm...
"An outfit in San Francisco called 'tenXer' has begun testing a service that aims to help people boost their mental accomplishments by up to tenfold—hence its name. . . . "
. . . .
"For those wishing to have a go, tenXer
in San Francisco is currently offering a beta version of its tracking software to help people analyse their own performance, set goals for themselves and improve their lot. At present, the tracking software works only for programmers. But down the road, the same techniques could just as readily be applied to other occupations. . . . "
The great thing about this is that many of us on Hacker News can put the claim to the test. Commenting on the claims of tenXer, which is what the writer for The Economist and several of the commenters here on Hacker News are doing, is interesting, and trying this out and seeing whether or not it works might be even more interesting. It's still unclear how much most individual human beings can improve themselves if they apply effort to self-improvement. It might be a good opportunity for career development for many HN participants to try this out.
For the record I've worked with EPOC-32, Windows, NextStep, Linux, VxWorks, and Nucleus - none of them was as painful as PC/GEOS (although EPOC-32 sometimes came close...)
In any case, superprogrammers != super marketplace conquerors, and IMO should not be judged as (un)impressive on that metric.
So much for moving up in the world.
i'd argue that individual contribution to any given project of substance is greatly overestimated, while contribution of the team (organization/culture/whatever) is unfairly underestimated.
even in professional sports, superstar a winning team does not make.
Another study in the book "Imagine" by Lehrer Jonah implies that creativity depends on the Q factor, inside a group. Here is a quote: "Uzzi’s data clearly demonstrates that the best Broadway shows were produced with intermediate levels of social intimacy. A musical produced at the ideal level of Q (2.6) was two and a half times more likely to be a commercial success than a musical produced with a low Q (<1.4) or a high Q (>3.2). It was also three times more likely to be lauded by the critics."
I can see how hiring only rockstars which probably knew each other would increase the Q beyond the ideal level.