Why Good Developers Are Not Getting 10 times the Pay
itscommonsensestupid.blogspot.com
itscommonsensestupid.blogspot.com
Imagine a writer is contracted by Rolling Stone for 2,000 words on a rock band (ie., the plot for Almost Famous, vaguely autobiographical for Cameron Crowe). By the "10x more productive" argument, we'd be most impressed with a young journalist who comes back with a high quality story in less time and at a lower cost.
The real issue here, though, isn't the productivity - it's that if you give this assignment to 100 different writers, you'll end up with 100 different stories. Some will be great, some will be fine, some will be boring, some will be incomplete. But while deadlines are important, that's more of a baseline: you need to finish in time, and you need to write competently. Beyond that, as a publisher I'd worry a lot less about the writer's "productivity" and a lot more about the product itself.
So a top developer can be someone who gets a well defined project done much more quickly to a high standard, or a top developer can be someone who takes vague notions and emerges with incredible software within a reasonable timeframe and budget. While there's probably some overlap, one isn't necessarily good at the other (in fact, I'd guess there's a negative correlation, mainly because people who would be happy working under one set of circumstances would generally be very unhappy in the other).
Somewhat related, it does seem like a major factor in the importance of people's software accomplishments (measured economically or however) is the importance of the problem they were trying to solve. Build an easy-to-use web site builder for Microsoft Windows in 1994? Sell to AOL, retire to Switzerland at 35, spend your spare time mentoring the young, designing prosthetic limbs, writing new operating systems in Python, or whatever floats your boat. (Incidentally, to me, that seems like getting paid 10× as much as the average COBOL cubicle drone who retires at 65.) Build it for NeXTStep instead? Nobody will remember.
Hamming's "You and Your Research" makes this point about scientific accomplishments: working on great problems is a sine qua non of scientific greatness.
To tie back to your point, when someone has some vague notions about what they need done, the first thing you have to do to make incredible software for them is to figure out what they would think was "incredible" — that is, figure out which problems to solve, and which to postpone. Because if you get that wrong, all the rest of your work is more or less wasted.
As Hamming points out, it's not enough for the solution of a problem to be desirable; it also has to be feasible. That clearly involves some risk.
I've always performed better when I had wide latitude, but this is difficult to pull off in "the real world." In college and especially grad school, I could get out of all kinds of exams and assignments by suggesting a different project to a professor. They'd get interested and say "sure, do that instead."
Thing is, it really didn't cost them anything. Giving a student an "A" even though he didn't do the assigned homework is, of course, very different from cutting a programmer a check even though he didn't complete the assigned project.
In general, I'd figure employers are looking for the 10X productivity, whereas investors are looking for the creativity.
You have to live an edgier life to get to be creative. The closest thing I've heard in the corporate world is Google's 20% thing. There are other avenues as well: I worked a while at Sun, and I managed to get a couple of projects accepted well enough that they became my full time job.
I think the author glances over too lightly the idea of good developers working for equity. A core part of the YC philosophy is that it is much easier to take a great hacker and teach them the business skills they need to succeed, than to take someone with business skills and make them a great hacker. (I'm basing this on something from one of pg's essays, sorry, can't remember which one at the moment.)
Based on the kind of output that the average YC team puts out during the program, a 10x multiple over the average corporate programmer's output during the same time is a reasonable estimate. Part of this is because, even if you have a corporate programmer capable of 10x more productivity than his peers, the processes surrounding him likely make it impossible for him to get that much work deployed in that amount of time.
And I think that is another important thing to consider. Corporations do not pay developers who are 10x as productive 10x as much because the way they develop and deploy software makes it impossible for them to realize a 10x productivity improvement, in any case.
EDIT: Just thought of a possible rebuttal I would like to address. You might compare this to the sports star, and point out that the star is not taking on the risk associated with getting paid in equity. However, an athlete's career is also an extremely high risk endeavor. Just as most startups fail, obviously the vast majority of athletes fail to make a lucrative career out of their athletic prowess. I would venture that the selectivity for an athlete looking for a lucrative contract is much higher than the selectivity for a hacker managing to start a lucrative business.
Even if they know you're 10x as productive as the other developers, they may not reward you so as not to make the other developers jealous or resentful.
Also 3 lines of APL are a very different beast to 3 (or even 300) lines of COBOL...
I think it's routine for me, and most anyone, to write a first version of a system over a week or two and keep noticing ways the design could be improved. Over time cruft accumulates--and you continue to notice new problems you didn't anticipate when you first designed--and almost always it is much quicker to rewrite a second version than the first.
The 3 lines of code per day metric seems fishy to me, but it seems to be based on systems level projects (lets build a new operating system) rather than making a new PHP photoalbum for Grandma. It would be very interesting to see numbers which broke down LOC/day for different subsets of programming jobs.
A "Complete rewrite" sounds hard, and like a big task, but usually it's a simple operation of copying existing functionality.
In my experience it's completely possible to spend 6 months writing version 1, and then completely rewrite it in a week.
I'll bet Netscape wished you worked for them in 1997.
He doesn't get to his real question until the last few paragraphs, which is why don't we see the same pay disparity with developers that we do with other professions? His answer is that measuring who's better is difficult and unreliable.
I admit I was confused, too; since he spends so much time on the issue of are some programmers orders of magnitude better than others, I thought that's what he was questioning. He doesn't get to his real question until the end, and then his answer is weak.
"It's not that good developers can churn out 10 times as many features as the bad ones. It's rather that good developers write code that have less bugs, more extensible and more maintainable. These are the things that you can't measure by statistics or figures."
I've given a few examples of good developers making systems that are orders of magnitude harder to complete, in the same time. In many cases it's not within the current talents of most programmers to write those systems at all. Another good example is the Quake engines. Additionally, in programming contests or student assignments there are many examples of people completing the same task in vastly different amounts of time. Finally, he states that reliability, buglessness, maintainability and extensibility are things you can measure with statistics, but many people have quantified uptime, maintenance, density-of-bugs, and extension time, to moderate success.
I find this sort of rhetoric honestly reeks of inexperience. Lines is a horrible metric for measuring productivity. Speed for finishing something isn't a particularly good metric either, since I've met a lot of hackers that can go on a coding binge (myself included) and accomplish as much in a weekend as they could in a couple of normal reasonably productive weeks.
"Comparisons between Ericsson-internal development projects indicate similar line/hour productivity, including all phases of software development, rather independently of which language (Erlang, PLEX, C, C++ or Java) was used. What differentiates the different languages then becomes source code volume."
Do you have a better metric? How about skill in solving tough algorithmic problems which haven't been before solved? Often on, for example, TopCoder, the top competitor can solve a set of three problems in 15 - 30 minutes. The median competitor, even in the upper division, solves zero. Some of the problems at only the 'medium-level' in those sets are in fact so difficult that they are given as two-week assignments in university courses, and only a fraction of students can complete it. This doesn't seem like a difference in reliability and bug count, it seems like a difference in kind in both algorithmic ability and productivity.
I think the only reasonable metric is someone's portfolio. You wouldn't pick an architect by how many nails they'd used or a musician by how many notes they'd written. You have to look at what they've actually done.
And in fact, that's pretty close to what good hiring is about. You then see if real performance matches up to what's in the portfolio.
For being on call at that time, I got a week's time in lieu and #500. For getting called, I got the same again. The fix was 10 lines of code, and the day after that I took it out and replaced it with a change to the original line.
Knowing what line of code: priceless :-)
I'm not trying to defend mediocre programmers here, but I think you've misinterpreted a study that intentionally combines individual productivity with organizational productivity.
* Unfortunately, I wasn't able to find the actual study report. If someone can locate it I'd appreciate it.
I don't know about that situation, but in general, rewriting a system is much easier than writing it in the first place.
1) Large companies don't yield the flexibility for top developers to produce 10X the value of the average.
2) Startups don't have the cash to pay 10X. However, I'll bet there are 10X equity differences.
If you were one of these magical 10x producitity guys, would YOU hold out for the job offer with 10x the normal salary? Are you sitting at home, right now, by the phone?
It is not that the managers don't know that good developers are 10 times more productive than a bad one. Managers do know; some of them are developers or former developers.
and
It's not that good developers can churn out 10 times as many features as the bad ones. It's rather that good developers write code that have less bugs, more extensible and more maintainable. These are the things that you can't measure by statistics or figures.
What sets really productive people apart in my experience is architecture and domain vision. This is certainly the lesson in the Viaweb example. Without good architecture innovation consists of "feature creep". You've got a good thing when you find yourself with changes that simplify a code base or system while increasing its power.
I suspect this is why people who develop their own tools are often seen by others are great programmers, and why Eric Raymond's definition of a hacker is someone who builds their own tools. I'm a worse programmer than a lot of people, but am orders upon orders of magnitude faster than anyone else in my niche domain by virtue of knowing how to do things and leverage existing assets. I assume really good programmers are leveraging their expertise in other areas too.
If this is true, I'd argue that what keeps the good people from earning higher multiples is that architecture and vision don't matter in a lot of cases. Companies that innovate less have less need for development characterized by rapid innovation. Systems get developed by spec and implemented by indifferent teams. You budget for the features you want, not the flexibility you need. Innovation matters more to smaller companies, but even there non-technical individuals rarely see the architecture through the interface. Flexibility becomes conspicuous mostly by its absence, as development gets "over budget and behind schedule everywhere".
One of the characteristics of a bad programmers is that they will say they are "done" when in fact they aren't or the quality is so poor that the schedule will ultimately slip. Employers are bad at judging internal code quality or correlating bugs with bad programmers.
In fact, at my old company, they actually gave bonuses to people who fixed the most bugs -- usally their own. They ended up rewarding programmers who made lots of bugs versus people who had solid code. Fortunately, such a reverse incentive program ended.
J. Random programmer who works at Do-No-Evil Geek-owned Company works on software that millions of normal people will use, but gets paid 40k, is much better off than J. Random programmer who works for Enterprise Software Factory who gets paid 40k and has to wear a tie and work in a cubicle on software 20 people may really give a crap about.
You will see the same skewed numbers at work in the video games industry, where employers know they can pay you barely anything for the right to wear whatever you want to work and work on something cool that people care about.
Musicians are not paid proportionately to some independent measure of ability, but to sales.
So neither of those example support the contention.
The article says the difference is not that good devs produce 10 times as many features. But that contradicts the original research reference, ie. productivity.
[1] "more extensible and more maintainable" (long term) [2] "meets a date with sufficient functionality" (short term)
In either case, intolererable bugs make the contribution invalid. The stuff has to "work" for the cheque-writer's definition of "work".
A manager setting pay rates has less latitude than people might think. The perception of bean-counters and other upper-management is that programming talent is a commodity that does not vary as much as it truly does. Even if they were to pay a superstar (defined however you like) what they were worth (in term of the bottom line), then they get nervous about having too many of their eggs in one basket. e.g. "Uhhhmm ... how about you get me 4 mediocre programmers instead of that superstar so that if one leaves or gets sick I am still covered".
I can tell you that most managers are driven by item [2] above with mostly just lip-service being paid to [1].
The peers of developers are much more slanted to [1]. This is why open source can kick so much butt. Developers actually get to do the right long-term thing.
So, in practice, a superstar "date meeter" trumps a "extensibility maven" in a managers (i.e. his bosses) eyes nearly every time.
It's hard for a company to realize the value from a 10x good developer, because their work is averaged with that of their 1x, 2x, 0.5x good teammates. Therefore the 10x good developer isn't worth much more than their more conventionally stellar 2x good colleagues; all he's doing is bumping up the average somewhat. The salary difference he commands is the the amount he bumps up the average.
Startups are more easily able to increase the concentration of 10x good developers, and that's where developers can make 10x+ money.
Programming skills is not something that is rare like sports skills. For e.g. An extremely good basketball player is a combination of athletic ability, height, work ethic, ball handling skills and so on.
Sports executives have less alternatives than Tech executives. The market determines the salaries.
Also, sports players and anchors more directly contribute to the revenue.
Its really capitalism and is fair.
Programmers have no power over the people who set pay rates while CEOs have a great deal of power.
Being in management gives you leverage. Being in a union gives you leverage. Being a replaceable widget -- even a gold-plated widget that is x10 better than the other widgets -- gives you no leverage.
I hope I've cleared things up for you. Now go out, throw away any Ayn Rand books that you own, and download something by Machiavelli. You'll learn a lot more about the way that capitalism really works.
Being in management gives you leverage. Being in a union gives you leverage.
I hadn't thought of this. Personally, I see programmers as workers that should be in small, elite groups, like surgeon wards. OTOH, surgeons have a cartel that programmers do not.
When a producer is negotiating a contract with Tom Cruise, I don't think it enters his mind how much unknown actors in community theater make. I think he's thinking about how many people bought tickets to see the last Tom Cruise movie and his estimate of how many of them went because of Tom Cruise.
I don't know enough about hairdressers to know whether the same thing applies to them. It does apply to fashion designers, and in some cases it seems to apply to programmers.
In most cases I suspect "brand" acts like a multiplyer so Apple's HDD might be worth 1.25 times as much as another just because of the reputation. IBM might be 30% better and have 2x the brand so they charge 2.6x what other consultant companies charge etc. Clearly major movies have great hairdressers but they don't get to charge 50,000x as much because a lower increase in value and a lower level of brand recognition.
PS: This get's confusing when considering something like breitling watches where a lot of the value comes from the price tag and not the physical watch. When someone knows you spent 250,000 on your watch they know you have money even if they can't see your car / house etc.
Perhaps what Rockstar Programmers need are Agents.
I met someone at a fencing club that was a programmer for a health care software company. He was my age, graduated a year earlier, and his salary was half what mine was (we were both on our first jobs out of college). I didn't know programmer salaries could go so low, for college grads. And his skillset was completely ghettoized - the company used a proprietary programming language and involved lots of domain specific knowledge. I suspect the difference was that I got my job through a friend of a friend that I'd met on the C2 Wiki, while he took the first available job he could find upon graduation, probably through Craigslist or Monster.
FogBugz developer?
Any line of reasoning that begins with "since I'm so good" and ends with "being here is suffering" sounds like there's probably some jackassery going on.
Honestly, a manager who was ok with him going home after one day would be stupid. Then he's giving them no additional benefit for his fabled skills, and their best developer wouldn't be there most of the time and the rest of the team would probably resent it.
Instead of the manager forcing him to sit at his desk for 40 hours a week, he should be giving this guy more autonomy. You don't keep good developers by squelching their abilities or freedom, they'll get fed up and leave.
But the really insightful one would let this guy try to find ways to make the whole group more productive or reduce its bug rate.
And why should his colleagues be any more jealous of him then they are of the CEO who makes tons more, or anyone else who gets paid more or works less for that matter?
There are 2 problems: 1) Having a clear way to measure productivity. 2) Stodgy ideas most manager's have about time spent, instead of production.
It's a problem because the company culture (probably) won't tolerate it, but instead will act to preserve itself at all costs, including the cost of reduced productivity. So when I said "obvious solution" I should perhaps have added "within the parameters of the existing culture".
I sympathize with your values, but the idea that you can change a corporate culture to make it recognize productivity in a more rational way is a fantasy. It's actually a harmful fantasy because it deceives many creative people into spending their time and energy on environments that will never fully recognize them.
To paraphrase what Max Born said about science, new models are accepted in the market not because companies change, but because old companies die.
Edit: Having a clear way to measure productivity wouldn't change a thing. If maximizing productivity were the value driving behavior, things would already be different even without a measure.
Actually they do: they can turn down an offer to work at any existing company and start their own instead. And many do. (That's how capitalism really works.)
thras was using it to refer to a role - a non-management role - and in that sense a programmer does have a lot less ability to influence the people who set pay rates.
You're using it to refer to an actual person who can try to change the role he occupies, in your example by starting a company.
When a company hires a CEO, they usually decide who they want first, and then they negotiate over price. There are few substitutes for their choice - usually, a company will find only a handful of candidates that are suitable for the position. This gives the CEO enormous leverage. There's only one of them, yet there are probably a couple companies that want to have them as CEO, so the price goes up.
Similarly, brand-name actors can drive moviegoers to theaters just by their name alone. There is only one Brad Pitt, but there are several movies that would love to have Brad Pitt star in them. Dakota Fanning gets like $4M/movie, because her name alone ensures the movie would get more attention than it otherwise would. David Beckham got a $250M contract, even though he's kinda washed up, because his reputation means people will watch.
An average programmer is filling a role, however. If they ask for too much money, the company just goes out and gets a different programmer. At hiring time, they don't know who'll be 10x more productive, unless that programmer already has a strong brand behind them.
I'm very curious to know how much "brand name" programmers like Peter Norvig or Guido van Rossum get paid. There are many Python programmers, but there is only one Guido, and many places that want him. I'm guessing I'll never find out, though.
BTW, this is behind a lot of the career advice that Internet celebrities give. Seth Godin's "you should throw away your resume" really means "your personal brand should be strong enough that employers want you and aren't just filling a position." Paul Graham's "work hard at things that excite you" is because if you do so, people will notice (eventually), and then they'll want you for what you've done and not just any old programmer. Marc Andreesen says "set the controls for the heart of sun" because that's the work that people notice, and that'll make you stand out from the crowd.
http://www.businessweek.com/magazine/content/05_05/b3918001_...
"For his part, Torvalds has been amply rewarded for his role, but he's no Bill Gates billionaire. OSDL pays him a salary of nearly $200,000. In addition, he sold initial public offering shares that he got as gifts from a couple of Linux companies, including VA Linux Systems. That helped him afford his house and put money away for his daughters' educations."
Anders Hejlsberg got a $3 million signing bonus from Microsoft when he left Borland.
http://delphi.about.com/od/delphifornet/a/conspiracydnet_2.h...
But as soon as you become an employee, earning a salary, you aren't selling directly to the end user, you are selling your skills to your company, which then produces a product that is used by the end user. Or worse still, you are selling your skills to a company that produces a tool that is sold to another company that is finally used to do something deemed useful by the end user. The longer that chain is, the less likely you are to get paid what you are worth if you are above average. Why? Because the entity that is buying your product doesn't actually care about the product, so if you have made it three times better than it would have been without you, don't care - just as long as it meets the minimum specification necessary to be able to sell it to the next person in the chain. On the other hand, if you are selling directly to the end user, they certainly appreciate the product that goes above and beyond minimum spec - insert Apple/Ferrari story here...
I can think of only one special case exception to this rule, and that is sports stars - and they are a special case because they are the only market where you actually have to beat the competition to be considered as performing. That means that there is a massive advantage in having someone on your team's payroll that is just slightly better than the other guy - you only have to be slightly better to win, and winner takes all.
Ditto for CEOs.
They treat the CEO well because he's their buddy.
1) The supply and demand of labor determines wages.
2) The factors that determine where you might lie on the demand curve for labor are: skill level, education, experience, reputation, etc.
3) The higher up the Demand curve, these factors become in shorter supply because it is hard to level up in real life. (MMO reference, sorry.)
4) The higher the demand of skills, experience, reputation, education, etc. >>> the lower the supply of labor will be >>> the higher wages will be.
Mediocre developers are short-term money losers, but some of them (the ones with curiosity and intelligence, whose mediocrity is a result of inexperience) will become great hackers, and thus, money-makers who, if treated well, will stick around for a salary much less than their value to their operation. Mediocre programmers are "overpaid" because this "call option" exists.