Challenging the myth of individual programmer productivity | Hacker News Reader

Challenging the myth of individual programmer productivity

insights.sei.cmu.edu

78 pointsvo2maxer121 comments

It's very convenient to think it's a myth or to think that the only way you get to extreme productivity is to be a cowboy, but I have personally seen that it's not the case. Obviously this is just an anecdote, but I have worked with at least one freak productivity monster so I know they can be real.

At one of my employers we had a very large and highly skilled dev community and being quant-biased, we tracked code checkins over time, number of tests written, number of pushes to production etc. I was very proud to be about number 3 or 4 out of about 1000 devs on these productivity metrics consistently over a period of 8 years there.

There was one guy however who literally dominated. Not only was he number 1 on any metric, but he did more checkins than the number 2, 3 and 4 devs combined. He had so many checkins that eventually our glimpse code search index whitelisted his userid because it thought it was a keyword in the language.

And it wasn't that he was a cowboy, he was obsessive about code quality, fixing bugs etc and also was top if you looked at test coverage etc. He was just a flat out machine - every day he would just come in and code all day whereas I have spurts of productivity with downtime etc. He didn't seem to need to be intellectually fascinated by a project and was just as productive at the beginning as at the end of a project - one of my colleagues joked that this guy's interest level in any project was always 83% ("Almost but not quite very interested") no matter what it was. He was very skilled and just churned out massive amounts of code all day every day. He never fell into the classic engineer's trap of overcomplicating things - in fact a lot of what he wrote was very simple when you looked at it (and I think that's part of what made him so good - he had a brilliance for finding really obvious, simple ways of solving hard problems) and because he wrote tests for everything he didn't really need to go back and or refactor things much.

> At one of my employers we had a very large and highly skilled dev community and being quant-biased, we tracked code checkins over time, number of tests written, number of pushes to production etc.

Why are those objectively useful metrics for productivity though? I can think of a dozen reasons why they aren't, as opposed to why they might be. Which, I guess, makes their objective usefulness subjective.

Some people will be working on longer lived feature branches because they're more complex, so they end up with one merge to master in the time it took the dev updating REST APIs to accommodate a new optional parameter to merge four branches.

Someone writes four tests that another dev might cover in one because the four assertions were all closely related to the same invariants and code paths. Who was actually more productive?

One developer pushes to production three times in one day because they typoed a property name twice... ...etc. etc. ad nauseum.

If you seriously believe that these metrics are objective methods of measuring developer productivity, I strongly suggest you spin off a business, because you're about to make millions.

No developer metrics are great, especially not if devs are incentivized to game them. However, I wouldn't be surprised at all if there's a correlation between good developers and those metrics.

If you incentivize the metrics, I would be surprised if the correlation held.

Well I suppose the correlation would still hold for the good developers but it would then also hold for the not as good developers. Over time maybe the bad developers would show a higher correlation than the good developers, but maybe not.

Number of check-ins, if the check-ins are high quality as he implied, are a pretty good rule of thumb to measure the productivity. The guy working on the most complex feature with only one merge to master should check-in even more often than the guys working on the simple features.

I don't understand that logic - the complex piece might require tons of research, experimenting with different approaches, benchmarking etc, before getting to something committable. And the successful result might only be a small number of lines.

I worked on project where I had dozens of commits every day. Now I work on project where I have one commit per few days. Sometimes I need to research for a week just to create one small commit.

And I am still the same person - only my position changed.

What is the point of checking in a horrendously incomplete feature that nobody else is going to work on, that probably cannot be tested by anyone until it's actually mostly done?

I'm working on such a thing, and I will check in ~100k lines of code when it is the right time to do so. That is not now.

In a CI/CD setup, you could make lots of atomic commits growing the feature behind a flag.

A side effect of atomic commits is that number of commits does better correlate to "work done", I suppose.

You could, but does that really solve any real problem? Now you're wasting more time on CI, commit messages, and trying to keep them atomic instead of delivering the feature. It's not even clear what makes up an atomic commit. If the feature does not work, it's arguably not very atomic. Stubs, mocks of parts that need to be mocked to push forward now and redone later properly, unimplemented branches that end up in abort(), these are arguably not very useful history either.

Also, I disagree that number of commits is a useful measure of work done. Not any more useful than butt in seat time. Nobody can say whether that feature is 200 or 2000 commits, it really depends on how you split it up. It could be 2 commits, plus the fixes that follow once it actually runs and gets tested and fleshed out.

If tracking progress is a priority, you break the job down to some reasonably small number (no more than a couple dozen) of items in a TODO list and update it with what's done and rough estimates of what's left to do. That's a hundred times more useful than "N commits, 123 hours of butt in seat."

Or maybe you can just demonstrate what works. Or tell, when it comes up in a meeting.

(I'd argue that butt in seat time is actually more useful than number of commits, because the latter can change by an order of magnitude depending on how you split it up but I've never estimated to take a month to deliver something that ends up taking 10 months.)

This is probably not the place to debate the pros and cons of CI/CD in general, but if a team is already doing it, isn't the list of atomic commits effectively the TODO list you're talking about?

Of course, over-engineered solutions will score you more commits (or more lines of code, or more TODO items, etc) but still be a bad outcome.

If you don't know how many commits it will be before you start, no, it is not a TODO list. I've never worked with such perfect knowledge. Usually, there's a high level understanding of what it takes to deliver a feature, which allows for the creation of a coarse TODO list that works for tracking progress & coordination. And then there's the devil, in the details. One commit is not one feature, and there are too many details for an a priori list of all the commits you're going to make. And very often you end up doing a bit of opportunistic refactoring & cleanups & little fixes to related code when fitting the new stuff in.

Even very small and simple features can end up being anywhere from 1 to 15 commits.

Yes, you're right; it's not a TODO list because it's not known ahead of time. Therefore, it can't be used to track progress on the particular project itself.

This all came up in the context of measuring productivity, though, not measuring progress on a specific project. Looking backward, it is a list of items completed. If the teams strives to keep them atomic in general, they are more likely (though by no means guaranteed) to be closer in size. So my guess is that this would correlate better to productivity than lines-of-code or time-in-seat, while still falling well short of the ideal.

By check-in, do you and the OP mean commit? I had interpreted it as a feature branch being merged to master - in the days of Git über alles, you can commit like crazy.

It seems to me that while any one of those metrics isn’t particularly useful in isolation, if the same person is consistently at the top of all of them, that’s telling you something.

_What_ it tells you, however, depends on whether or not the combined metrics are used to evaluate people (or more accurately, whether people _think_ they're being evaluated by those metrics). Goodhart's Law[1], ruining everything since 1975.

[1] https://www.lesswrong.com/posts/YtvZxRpZjcFNwJecS/the-import...

Maybe. but what if their peer has far fewer commits to their name, far fewer written unit tests attributable to them, because they spend most of their time using their indepth domain knowledge to assist other developers, who then make commits based on that contribution?

Do they have to sit around and ensure that their coworker includes a "Co-Authored-By" in the commit message? One commit message? All commit messsages? etc. etc.

This guy was also top in terms of the number of FAQ entries written to help people. In fact he wrote the FAQ system itself and also wrote a right-click plugin so if you rightclicked on a question in chat it would auto-search the FAQ.

See now that might be a better metric.

Presumably someone, or some group, was writing all the requirements and acceptance criteria necessary for him to code all day, and they were managing the changes to those criteria as users gave feedback on what he was doing.

And then there were the other devs peer reviewing his code, happily accepting his PRs without needing to add comments or request changes.

And then there was the QA team doing the integration and end-to-end tests, user testing, training, etc.

He couldn't have been quite so productive without those all those people.

That's not to put his amazing work down. No doubt he was brilliant. It's just that we shouldn't ignore the efforts of the people who enable developers like him to get on with what they're good at. Development is a team sport.

You’d be surprised. Sometimes the level of support from others is basically zero but you still make it happen. That’s the real sign of 10x

How did it work out practice? After each Sprint the numbers got revealed?

Did that not lead to a super competitive workplace? I cannot imagine that this was very motivating to the remaining 990 devs... Or was it?

> in fact a lot of what he wrote was very simple when you looked at it (and I think that's part of what made him so good

Genius is when you write a piece of code that's so simple everyone else thinks they could have done it. And yet they didn't.

The only relevant question is whether that person has a life outside programming and work. If yes, then it's truly a case of remarkable talent. If not, it's a case of sacrificing something in exchange for something else.

Moderation is not for everyone.

The world would be a much less vibrant place if not for the obsessives among us who devote themselves religiously to their craft. Whether that be programming, writing, art, or otherwise.

If you have someone who is so obsessed with programming that they have time for little else, the lack of social contact has the potential to turn that person into a problem later on. Case in point: Richard Stallman.

Inappropriate behavior may have been well tolerated in the patriarchal past, but in 2020 we are expected to do better and our institutions can ill tolerate the risks presented by asocial white men.

I think blub was just trying to distinguish whether the person delivered more value per hour, or just more hours. The outcome may be the same, but the former sounds more like "10x" than the latter.

What you are describing here is a code monkey. Did he had any opinions or took any design decisions or he was just paid to simplify things?

I know the type

Used to be like that when I was young

I also know that there's the guy that never writes tests, is incapable of writing non reduntant-code, pushes everything to master as a blob of unrelated changes, using git as the backup server, comments are sparse and mostly useless, code is a mess and 'deploy to production' means starting a Jupiter notebook on the Dev machine.

The same guy saves to the company millions in expenses and gets the promotion.

10-20-100x Devs are just very good Devs, nothing that automatically changes the fate of a company

You can have the best guitar player in the World and your band goes nowhere or you could be the Ramones...

I call BS, that's not even possible. Even the legendary "tourist" cannot do it, in his own words. He programs like 2-4 hours on good days.

As a manager of devs, the times I've seen someone be much more productive than usual have all been where they:

- Understood the requirements. Not just in a specific way, but also by having enough context to know what's likely to come ahead.

- Already have the tools familiarised. So the right languages are in their brain-cache, the CI/CD is familiar, and there's little need to read up on any new frameworks.

- Have already done something similar in the past. I work in the financial space, and the sheer number of wheel reinventions is mind boggling. Feed handlers, orderbooks, decision logic, aggregators, wire protocols. Someone who's been through the grinder a couple of times will know what the fixed points are and what you can change.

All of these point towards experience not simply in time but also in terms of projects done.

I've also come across a handful of people who were inexperienced but thought they could solve all the same problems without the benefit of having seen how others have done it, and it's been a disaster each time. Sometimes these people are hailed as heroes, because management thinks it's somehow to their credit that their Rube Goldberg machine seems to work.

The one or two really good junior staff who "get it" have tended to be the humble personality type. They don't bite off more than they can chew, but they eventually eat the whole cake.

How much of being a 10x dev comes down to understanding the existing system really well and being able to iterate quickly?

I wouldn't call myself a 10x dev but at my last startup I was very productive, definitely more than expected at my skill level, because early on I had a lot of access to the people who designed our (undocumented, untested, and buggy) backend system. Then when I got a feature ticket (consisting of 3 words) I was able to quickly bang it out and deliver it, whereas other more senior engineers struggled because they didn't fully understand how the backend microservices actually worked and talked to each other.

Now I work at a big tech co and I'm in the latter position: I have a hard time understanding our complex CI/CD, and don't have nearly as much understanding of all the existing components. As a result I'm not nearly as productive and back to feeling like a 1x or 0.1x dev.

Of course a 10x dev understands the existing system, but it's more than that. A 10x dev IMO, understands how to logically take apart and put together a system. I find it amusing that we recognize grand masters in chess and elite performs in other fields, but don't wish to acknowledge that such people exist in our fields.

What I have seen with such developers is that they can jump into a system they have no idea about and ramp up faster than most people. I don't believe that everyone can be a 10x, I also don't believe that everyone that can be a 10x is even going to reach that field. From what I have seen, these folks are the sorts that enjoy working on non user centric app. They hate those ambiguities, so you might find them in security, or working on compilers, networking and lower level sort of codes where it's possible to conquer whatever demon that will arise from the code. It's almost impossible to WIN in user centric app where what's correct is subjective.

> I find it amusing that we recognize grand masters in chess and elite performs in other fields, but don't wish to acknowledge that such people exist in our fields.

It’s easy to rank people playing a zero-sum game where the only thing that varies is opponent, use ELO and call it a day. Software development is more complicated because the problems and quality of solutions aren’t even clear. So to me it makes sense that we don’t have software development grandmasters outside of things like IOI.

I personal believe problems and quality are pretty clear. The challenge is in understanding what the problem is, which is not a "programming" activity, but rather a communication activity. You could have 2 customers who want a solution that clash and conflict, if you make one happy, you make the other sad. No matter how you tackle it, you lose.

e.g. You build something for sales that makes customer service unhappy or vice versa.

You build something and keep it backwards compatible and that makes your prior users happy but puts off new users who were willing to adopt or vice versa.

At this point, it's no longer about programming. Doesn't diminish that those activities are just important. We could also argue that there are 10x user interface and interaction designers, 10x project managers, 10x product managers, etc. The confluence of such folks is a beautiful thing to experience, however it's not needed, just 1 or 2 of those with a bunch of "average" folks can ship and release a really good product.

I disagree that it’s almost impossible to win with what is essentially UI and UX challenges.

Part of it is subjective yes but a lot of the field can be measured and ranked in terms of quality and effectiveness.

> I find it amusing that we recognize grand masters in chess

What would you consider the chess of software development?

How would you rank the participants?

I took a new job this past fall, and I found our "git push to production" pipeline to be quite opaque. I solved this by taking on small projects to implement incremental improvements: improving log level configuration for production, simplifying database migrations, etc. I learned a bunch about different parts of the system, and it gave me a chance to gain some visibility in a good way. And each improvement helps me feel more productive, and that increase in perceived/actual productivity comes from better tooling or having knowledge of how the system works.

Developer productivity is not linear in function of task difficulty. A 10x developer may be just as productive as a junior if given easy enough task. Or even actually worse, because he could get bored much more easily. It may not be fun to move data from a database to a website when doing it for the 100th time.

In a company I worked for, there were two teams working on the same product: the mobile frontend team and the backend team. The productivity of all developers was very similar, and both teams seemed to be working in lockstep and features were delivered mostly timely without big issues. Until the moment the frontend team hit a serious bug affecting the core functionality. After they struggled with it for a week, a senior Java developer moved from the backend to help them. He nailed down the bug in a day... in third party C code. This is why you need 10x devs, but they may not be immediately visible. A 10x developer is not 10x more productive. A 10x developer can solve 10x harder tasks when 1x developers fail to find answers in stack overflow.

>Developer productivity is not linear in function of task difficulty.

Tried to come up with the draft of an equation (upgrades welcome ;):

cDev = capability(dev); // > 0

dTask = difficulty(task); // > 0

ssttTask = shortestSolutionTypingTime(task); // > 0

pk = priorKnowledge(dev,task); // in [0,1]

lcd = log(cDev / dTask);

timeTaken(dev,task) = (cDev <= dTask) ? +Infinity : ssttTask * ((1 - pk + lcd) / lcd);

productivity(dev,task) = ssttTask / timeTaken(dev,task) = (cDev <= dTask) ? 0.0 : lcd / (1 - pk + lcd); // in [0,1]

10x performance has very little to do with algorithmic/academic agility, and everything to do with broad knowledge of software engineering, not creating unnecessary tech debt, pro-actively owning and solving problems, and documenting and automating the shit out of things. 10x devs are seldom ten times faster at wiggling their fingers on the keyboard to shit LOC out. It has very little to do with smarts (although there's a bar) and more to do with stewardship, vision, and common sense.

Something this article just can't capture is that many devs add net negative value by being allowed to work in an organization. They create complexity, chaos, and confusion. They might be smart, they might be dumb - the problem is that they have no clue what they're doing within a software organization, and often they don't care. They ship broken code, awful bugs, badly designed systems. They make bad technology choices. They don't measure their work (metrics, logs). They put no thought into anything other than manual W.O.M.B deployments. etc, etc.

It's very hard to not be 10x more effective than a net-negative dev. And then beyond that there are very good engineers who we typically put on the senior -> principal track who are able to get a lot done across a large organization, and generally create a culture and an economy within a company that is conducive to sustainable engineering. I expect the average principal engineer to be slightly slower than a college student on an algorithms problem (depending on the problem). I don't expect a college student to be able to drive cross-organizational software initiatives in a large company. I also don't expect that of most Senior engineers, and I don't think most Senior engineers I've met are even willing or capable of doing that.

I'm not sure I'd trust one dude's college experiment to be relevant in this domain.

Also a net negative engineer in one setting might not be a net negative in another. Where I used to work we had a senior programmer who never wrote any tests, claimed c++ was the best language ever while pushing c++ code with correctness errors, switched to go and wrote an algorithm with stateful variables that couldn't be multithreaded. Yet I'm sure that probably at least a few tines he must have written a few lines code at Facebook that saved them on the order of millions of dollars

> , not creating unnecessary tech debt, pro-actively owning and solving problems

Yea lots of junior programmers 'design for future' and 'follow best practices'.

Summary: "We took a classroom of students and measured their performance with coding challenges in a situation that bears no resemblance to actual business operations with no middle-management, insane CEO requests, or office politics. The top performers didn't excel at the mundane tasks, and we couldn't find many 10x performers in this classroom of students."

Slight correction: "We took a classroom of students and measured their performance with coding challenges in a situation that bears no resemblance to actual business operations with no middle-management, insane CEO requests, or office politics. We observed that several high-impact comparisons could be made, similar to literature suggesting a 10x programmer. However, we found that comparing a student to himself often results in >10x results, and that other comparisons often used in literature could produce >10x from this dataset, which obviously had very few true outlier talents or performances. Therefore, we conclude that the story is varied and that individual variance overtime outweighs cross-individual variance at any specific time."

Their sample size was 494 students from classes over seven years (they could have included 3800 but wanted to limit it to a shared choice of language). And not for nothing, their students were not your typical newbie grad students - they had an average of 3.7 years of industry experience.

A friend of mine asked me to help him with piece of code. I looked at it and it was all over the place, nest of ifs and loops. To understand it would take me hours so I rewrote it from scratch. Nothing really fancy or overly clever just simple, minimal, straightforward code that followed the assignment. When I showed it to him I discovered that mine had exactly 10x fewer LOC (250 his vs. 25 mine). That wasn't my intention it just happened so. And I know programmers who are 10x more productive than me.

How do they write half a line of code?

shell script to do the same thing?

A library call? )

maybe

if (x < 10) {

becomes

if (

   x < 10 ) 

  {

I mean yes, in many languages you can write all your code in one line or split it up in all sorts of ways if you prefer, but there is sort of a natural line and an unnatural line for any particular language that is an attribute of its commonly preferred formatting.

on edit: reformatted

That's still not half a line of code. Your first example is 1 line and your second example is 3 lines.

If a 10x developer can solve the problem in 1/10 of 25 lines that means they need to write 2.5 lines. How do you write .5 of a line?

It was meant to be a joke but now it's making me annoyed that 3 people's replies have failed to grasp that you can't write half a line of code given that even a single character counts as a line. Surely that's obvious. Do people on HN not understand what the LoC metric is?

Maybe on Windows if you omit half of the CRLF ;-)

Oh wait, then it’s a Unix komme or a Mac line. Bummer...

well I've certainly experienced my fair share of people on HN not understanding things, case in point you me now.

I thought it would be obvious what I said was a joke since it is absurd, but now I'm getting annoyed at the person who didn't understood my sense of humor.

However there are obviously also different ways you can write half a line of code, for example I can write half of any specific line of code you give me (allowing of course for rounding up or down when that line is an odd number of characters in length)

Finally there are actually different ways you can count lines of code, generally it is considered best to use logical lines of code http://codebetter.com/patricksmacchia/2007/10/03/how-do-you-... which it seems is not your definition, at any rate it is not necessarily the case that a single character counts as a line depending on what way you are counting lines.

Otherwise thank you for the education, I consider the hours I have spent on HN golden hours, and the lines

of com

ment

time well

spent.

The biggest challenge in debunking that myth is non-tech cooperate positions above developers. Once a cowboy coder delivers results and gets noticed for fixing stuff quickly, it's all over. That person is hailed as a hero and he'll be the first pick for leading the "A-team" of devs to make 2.0 once they realize how buggy and resiliant the current system is to change. This repeats adnauseum in every company I've worked in. The only solution is to embrace a cultural change that's centered around professionalism in software and puts the developer at the top of the org chart.

I think people think of 10x programmers as someone that will rip through your average backlog of stories 10x faster than anyone else on the team. That's pretty unlikely if you're not involved in creating those stories.

I think being a 10x developer is about getting from point A to point B ten times more efficiently. Its not just speed, its reduced waste and smarter planning. To do that, it's not just about coding, it's about deep understanding of the problem at hand. A great developer will be able to recognise things that are superfluous, find shortcuts, know when not to write something, etc. Its not just about coding faster, its about finding the shortest path. So they'll get you to having a completed work item 10x faster, but often by doing it in a way that might not be obvious from the outset to others.

But if you're working in the standard model where a PM is micromanaging... sorry "grooming" tasks.. then you frequently wont get that sort of performance because the developer doesn't have the autonomy. Without autonomy you lose what makes good developers good.

Of course just giving developers more autonomy isn't a panacea, most cant use it, but I think putting people into Scrum situations precludes most "10x" developers from being able to shine.

Agree, basically that is the point. If you have just stupid tasks, everyone will perform the same. Then sometimes you have a challenging problem. A good skilled dev (let us call him 10x) will develop a clean, easy solution on the other hand one not that good dev might just make things extremely complicated and ruin the whole solution. Happens all the time.

A lot of people can write code, but on the path to making a shippable product a ton of programmers don’t have clarity on “what to write next” to arrive at the finish line. It often seems like a product is 80 or 90 percent done, but by the time it is actually shippable as a product, 2x or 4x or more work will be done, and major revisions of the code will happen to reach that point. The real 10x programmers are the ones that have done this enough times that they don’t get distracted doing naive things that will be discarded and redone as often.

I wonder if this study looked at a lot of programmers who just fix bugs or add well-defined features to apps, or really distinguish between those people and people who can build shippable products from ideas.

The study looked at small assignments in a course. So it’s definitely ignoring these types of long term productivity advantages.

Most engineers are 0.1x engineers, just trying to remain employed while browsing a lot of HN. Then there are 1x engineers using the rare skill of actually trying reasonably hard and having discipline. (These 1x engineers may eventually experience a significant life disruption, mental or physical illness, or simply lose the feeling of meaning from the work, and subsequently become 0.1x engineers temporarily or permanently.) There may be actual 10x engineers for short stretches. I may have been something like this for a short period in grad school when I was obsessed, skilled, healthy and knowledgeable all at the same time. But I’m certainly not anymore.

Software is one of those things where the right small changes can easily produce a 10x difference in value.

For example, not having an overflow bug can save your company $millions. Or avoiding a deadlock. Or speeding up your datacenter software by 1%. Or picking the right architecture at the beginning.

I've been coding for 15 years and for some kinds of projects, I'm 10x to 100x more productive than I would have been 10 years ago. But it's only the case if the project's problem domain is complex enough and I get to design the solution from scratch.

If you asked me to work on an existing spaghetti code project or on a very simple front end project, you wouldn't be able to see any difference at all between my old self and new self. In fact, my new self might underperform because I would be bored and going at a very slow pace.

There are 10x and 100x developer but you only notice them when the problem is very complex and you give them the opportunity to lead. The idea that 10x devs don't exist is self-fulfilling in a way because managers who believe that don't look out for these devs within their own organizations, so they don't promote the right technical people and they never get to see that 10x return.

I find it deeply disturbing that many people have no trouble believing that some CEOs are 100x more productive than others but they don't believe the same is possible for developers despite the fact that many software projects are in fact highly complex systems; in the same way that companies are highly complex systems.

I suspect that 10x programmers are productive because they control the abstractions and tools that makes them productive (for example, programming language they use), and write their own that help to solve the problem faster.

If you control for this autonomy, by assigning them small "routine" tasks, then of course the effect is lost.

The paper somewhat advocates an approach to SW development that can easily result in this. If you give people lot of small tasks to be done, they will never be able to articulate that they are actually all very similar and can be factored out into an abstraction.

I agree: once the development process is organized to the level that for every few lines of code three meetings on higher level managers and "designers" have to happen and be signed and the developer just has to write three lines and make ten times more paperwork (in triplicate), and constantly wait for other parts of the organization, no developer can be better than any other, to the joy of the managers, and all developers became provably replaceable. Not because the developer can't do better, but because the whole organization mostly doesn't prioritize effective development.

I however suspect that such an organization, happy to meter the development, would not meter the efficiency of the whole, and especially would not allow comparing that whole to some other organizing model -- it could prove that the managers are the weak point after all.

Yep, I think you've hit the nail on the head here. I've worked with a lot of colleagues that could implement features faster than me, polish off more story points, or wreck me on whatever the metric of the day is, but if we were both CTOs of startups in the same market I wouldn't even break a sweat worrying about them (from a tech perspective).

Most businesses don't have access to top talent and they know it. Most businesses are specifically trying to create an environment where devs are interchangeable, and optimising their processes for the 50th percentile dev (if that).

If you do your measurements within that context then of course 10xers don't exist. But when it's John Carmack vs Scrum Legend #34598295 1v1 on Final Destintation with No Items, I know who I'm betting the house on.

> Most businesses don't have access to top talent and they know it.

I don't believe it's a matter of finding talent, but a matter of learning it, and you learn it by doing.

The problem is people will only learn skills they need - so if they don't need the skill to be 10x developer, because the management doesn't expect it or doesn't give them the autonomy, then they will not learn it.

And of course, learning something as well goes against short-term expectations of productivity.

It seems that there would be a huge selection bias in this study. They only looked at programmers taking their course. But wouldn’t the best programmers be much less likely to take a course for improving their skills? So it’s no wonder they found that only a tiny % were very good. We can’t really conclude anything about the population at large from this.

They found no true outliers in this group, but were able to make comparisons which did seem to show fantastic 10x (or 24x) outperformances. From this you may conclude that comparing programmers is difficult and likely to produce misleading metrics.

They found that 95% of programmers had such high variance in their day-to-day that Bob on Monday might be >10x with respect to Bob on Tuesday. And, this variance dwarfed the variance between Bob and Alice, one of whom may be a supposed 10x programmer.

Through this particular thing may be due to them being students. They tend to code overnight and then be clearly sleep deprived - but still insist on coding and being unaffected.

So you get high performance one day and horrible one after.

Imo, once your lifecycle becomes more regular, a lot of these ups and downs disappear.

I find the study poorly done as they only looked at people who took a "Personal Software Process (PSP)" course with an average of 3.7 years of experience.

I would assume that people who are extraordinary good at their job will resist taking a course geared towards beginners, so they wouldn't be counted for this study.

> The best are not that much more productive than merely good (on routine tasks).

> There just aren't that many "best performers" available.

> The real world, of course, is far more complex than the limited, controlled study that we conducted.

I think this pretty much sums up why I don't like this article.

They had some real world-data, but didn't really have any interesting conclusions. Obviously someone isn't going to be good at everything, and if they were, there aren't going to be that many people. The issue of individual programmer productivity is very complex, and studies like these are way too abstract to draw any meaningful conclusions.

This is one of those cases where it's easier to talk about things using common sense because it involves things that are easy to break down by the human mind but hard to break down into a scientific theory. We know certain people are better at their jobs, maybe even 10x the average programmer for specific situations. But at the same time, it might just be that the comparison being made is unfair, or the person is good at one specific thing but bad at something else, these are things that have to be evaluated on a case-by-case basis.

People have gotten way too obsessed with the term as as a concept and it doesn't help to talk about 10xers because the discussion is at a high enough level that it's too vague and meaningless. It always just devolves into what even is a 10xer and what if someone is 0.1xer and so on...

The weird thing about the article is that figure 3 clearly shows that some programmers did have consistently excellent performance, but they don't show the relative performance of the top performers, only their rank. We are supposed to infer it from table 1, I suppose, but that only shows the relative performance for the top 5%.

If you're smart enough to see a simpler way, there's easily 10x gains.

If it's only because you know the "trick", you still get 10x gains.

Exactly. I talk about this at https://riskfirst.org/Estimates

Knowing the way is 10x over finding the way.

But I think this is still an interesting piece of research, backing up a lot of what we expect empirically.

"For the study, we used data from the PSP class programming assignments for which students record effort, size, and defects for a series of programs."

What?

While the data points are nice - I think the author has completely misunderstood the issue here.

I don't think in any given group of young devs there's going to be someone '10x better' than the rest, and this was never want '10x' was meant to refer to.

Though I don't really believe true 10x-ers exist, I have no doubt that senior, competent, experienced, creative and focused engineers who know how to communicate and collaborate are often worth 10x relative to many of their peers.

Particularly in tricky domains, in funky languages that have a lot of room for self immolation, doing non-standard things, a fully competent senior dev who has 'done most of it before' can basically ignore most of the pitfalls, knows most of the little idioms, and can get to solving the problem in a reasonable way, whereas it may take quite a long time for a lesser, junior dev, and may be entirely above the head of others.

I have to disagree. There are people who, outside of any job, discovered programming to be fun at a young age, such as modding games, and if they keep at it, they become at least intermediate-level at a young age. Then there are people who heard coders make good money, studied it in college, and got a job doing it (at a place that doesn't need to pay for, or can't determine, the former type), but wouldn't do it for fun. They know the concepts, but they have a weak link to enjoyment of using them, and only learn what they need as a minimum to make some contribution. They want to like what they do, but it's not easy. The former can easily be 10x more productive than the latter, but they're on such different levels, it's unlikely they'd work side by side so you could compare them. This is ignoring the rare savant who's closer to a 100xer, but I've seen them too.

I don't think there is such dichotomy and early youth passion is strongly overrated. Many people who liked modding and similar activities as young are not that great - especially when it comes to difficult tasks. And many people who started to learn it at college or even after are great.

The modders and early starters have massive advantages in beginner courses and maybe in intermediate. But many of them don't progress all that much further or hit some kind of ceiling (difficulty to learn algorithms, it becomes boring once not creative) or even actively dislike programming for business (difficulty to deal with changes in requirements and other business annoyances).

And I know people who studied something else and then changed to programming - most of them come from physics or math background, two teachers. The original decision was for money, but they did learned fast and made up the difference. Mostly because they were used to learn hard things. Learning something makes you like it in the end and that happened too, but was not the actual reason for anything.

Do you believe that 0.1x developers exist? In a crowd of "0.1x" developers a "1x" developer becomes a 10x developer.

0.1x developers definitely exist. One of the ways they can get a job is nepotism. "Free market" efficiencies go only so far.

It could be efficient to hire via nepotism. For example to maintain valuable connections, improve your standing in a group, etc.

Market != meritocracy

-10x developers exist, those that cause more damage than the value they create. For sure.

> I don't think in any given group of young devs there's going to be someone '10x better' than the rest

John Carmack was in his early 20s when Wolf3D and Doom happened. Mid 20s for Quake. I have reason to believe he was vastly more productive than most developers his age. I don't think it's unreasonable to think there are other young developers who are similarly much more productive than the average.

10x or not, I do believe in significant differences in productivity.

In school I was drastically more productive than my peers. In my final course where we made a 3D game, I relied on some asian kid with aspergers who was drastically more productive than me. Seems like if there's two instances of drastically than the distance between my peers and the asian kid is probably easily 10x.

I was under the impression that 10x isn't in terms of speed of coding, but rather value added to the business, including proposing/implementing solutions that weren't in the spec to begin with but open up new possibilities.

Seems like something difficult to test, probably why it wasn't attempted in this study.

In the '90s, Siemens and Ericsson made a joint venture called Ellemtel. They had a 6-month death-march project: deliver in 6 months and the principals get paid; not and don't.

Each company sent 250 engineers. At the end of the (successful) project, a friend of mine checked who wrote the code. Fully half was by a single engineer.

So, that engineer delivered as much code as 499 others, combined. I know him. He is not arrogant, because he knows someone 10x more productive than he is, and wears out two keyboards each year (or did, then).

The numbers are a little biased: this engineer was responsible for parceling out tasks, in two-week chunks. If a task wasn't done on the Friday, he would do it himself on Saturday, and commit that. So, a fair amount of work was done and discarded. So, maybe count him as 250x+ rather than 500x.

The key to improving the productivity of individual programmers is:

keep the code(system) simple, standard, and unified.

https://github.com/linpengcheng/PurefunctionPipelineDataflow

I often walk in to codebases that are quite unified, but that unification happened too soon, resulting in a system that locks programmers into working at far too low level of abstraction.

Trying to change that is nearly impossible because doing anything different would break the cohesiveness of the code. Any new abstraction must be judged against the huge downside of splitting the codebase style.

IMO abstractions need to be actively and continuously fostered in the early days. Writing “simple” code to encourage cohesiveness early in a project is a missed opportunity to develop systems that illuminate the problem area.

In The Pure Function Pipeline Data Flow v3.0 with Warehouse/Workshop Model

1. The replaceability, insertability, observability, and readability of the pipeline are very strong.

2. The warehouse/workshop model is widely adaptable, simple, and reliable, and the workshops are isolated from each other.

These two aspects can solve your problem.

Should the conclusion not rather be that the average CMU student is really smart, some are really fucking smart, and a few struggle to keep up? This was conducted through a class assignment at CMU no? I would expect to find a good proportion of 10x engineers no???

"Keep tasks small"

Thats the key.. Break down the task into smaller chunks and work on them with tests, then bring them into the larger codebase. Once you start ping-ponging all over the place you don't stand a chance at being 10x.

"Provide a quiet work place so that programmers can focus."

Thats another great one. I get a lot done some nights because its really quiet.

Also, some programmers might be faster but they introduce a load of bugs and you end up playing ping-pong bug hunt.

I had this outsourcing company that would fix one bug, but introduce another 1 or 2 bugs. It was really frustrating.

Its like there was a lack of thought into the program flow, just hacking until it sort of works.

It's omitting a lot of details of the methodology.

Is this only including students who did all the problems, ran the test cases, and submitted all the statistics they asked for? That would tend to exclude all the low performers.

And what happened with the defects? Did they just go unfixed, and data for broken programs is included, or did they have to rework them until all tests pass?

From the article:

> Of the 3,800 students in our classes (from 2000 to 2006), this study included only the 494 who used the C programming language and who also completed all 10 programming exercises.

That should answer your first question. As for the defects: I’m not sure (but perhaps defect-free = passing all tests?).

The methodology seems fluid, too, because there are disparities between the numbers in the text and the figures.

The difference between 1x and 10x devs is not (just) their skill - it's mostly that they're just more motivated and do their stuff with high urgency, passion and focus on detail opposed to just doing barely anything at all. In the end it's about dopamine, like so many things in life.

I think this is quite an oversimplification. If it was purely neurochemical, you wouldn't find any developers (or any other profession where performance is measured) who are productive during times of depression or grief. Yet sometimes those are the most prolific periods for folks (see many NBA stars this past week after Kobe's death, or the output of artists dealing with pain).

Depression is weird and influences so much stuff. But give any mediocre dev (or any professional) a dopamine-active drug of your choice and they'll see them skyrocket. It's not a popular opinion but that's the biggest factor in anything. Sure there are IQ differences etc, but they don't matter as much.

> Since quality can be taught and benefits apply to the total lifecycle cost, emphasize quality rather than speed.

I have seen, that this is actually challenging to teach... obviously my sample size is small.

How about you guys, in your experience, were you able to teach quality / lifecycle costs ?

There are 10x programmers. A lot of people hate to hear this, but get over it.

Let's take something that's very controversy in the tech world, white boarding. Why do many programmers hate it and are not good at it? Sure, you can claim it's lack of practice. No problem. But guess what? Does that can do it and do it well are more than 1x programmers. For the sake of argument, let's call them a mere 2x. What does it take to whiteboard code? It means you need to be able to understand the problem, clarify requirement, understand the constraint, solve the problem a bit mentally before writing code, write the code, explain your thought process, improve the code, consider multiple solutions, perform algorithm analysis in terms of performance, and you do all this in 15-40 minutes.

It's not that most programmer's won't, most programmers can't with their current skill sets. Perhaps they can learn how to, but they refuse to. Doing the necessary drills to be able to do those shouldn't be underestimated, it makes you an all around better programmer.

With that said, there's a big difference between "programming" and "application development" A great programmer could be a terrible application developer, a mediocre programmer could be a great app developer. Application development and shipping software is more than code artifact it's about collaboration with teams, writing great documentations, user documentation, installation documents, operation docs. It's about great UI, great UX, designing the code to be easily extensible and readable. There's a lot to it. So you might be a 1x programmer, but make up for it in other areas. This also explains why some 10x programmers don't produce 10x the output because they are not solving algorithm challenge everyday, but having to deal with the vague human oriented requirements.

There are 10x programmers, they can make a team much better if they understand their strengths and weaknesses and don't think they are gods. There are 10x programmers who are also very strong in design and other areas and it's amazing to watch them work.

It's no myth, they are out there. The very thought of them makes some of us feel less capable, get over it. You can increase your output if you consciously work to be better, and this is not about being a specialist in a specific domain area.

A good test of such thing is take a 10x programmer, a 1x programmer, give them a language they have never worked in, put them in a new domain. The difference is clear. Have you ever seen such folks? Yup, we have, they are folks they moved to new domains, they were writing C code yesterday and now writing go lang and contributing to k8s, or they were writing java code yesterday and today writing C++ code and contributing heavily in the field of ML.

We have been been fortunate to know some of them due to this very site. I'm not going to name them, it's been done multiple times. But I'm sure HN can name at least 50 such individuals.

This whole argument does not take into account what article and research was about.

It is not about documentation, UI/UX, collaboration, and other skills. Research was only about programming as in "here is specification for task, code it, please". That is what word "programmer" also conveys. Such person should not be concerned about making user experience better, that is role of a designer. Such person should not be concerned about designing code to be extensible, that is role of a software architect.

What you describe is a one man orchestra, which in reality everyone wants because no one wants to pay for architects, designers, managers, separately.

In the end there are folks who can work on multiple things at the same time like ML and some k8s, I do not believe you can describe what they are doing simply as "programming" or them as just "programmers". Yes they are programmers or coders but there is much more to it. Exactly like patio11 was writing in "Don't Call Yourself A Programmer, And Other Career Advice", because you will be seen just as a code monkey and not as that "one man orchestra".

My main argument is we should stick to the definition.

The article isn't what I thought it might be.

The article doesn't really challenge the existence of the 10x programmers but merely puts it in perspective from project management's view.

It is mostly about not relying on the superprogrammers when it comes managing software projects because they are so rare and need highly complex free-form work field to execute their magic. Just merely hiring the best programmers and subjecting them to a working schedule and distribution of tasks designed to accommodate average programmers doesn't improve overall productivity much and certainly doesn't give you 10x or even 2x productivity improvements. This all makes sense and is probably known to people who manage software engineers and software projects.

The article does mention the high differences between the worst and the best programmers and says that it doesn't make much sense to compare the best to the worst. But the 10x has always been used to underline the whole scale itself, or that order-of-magnitude differences exist in the first place. The best-to-worst can even be higher, from 10x to 50x, but 10x is the illustrative reference point. But that's not the shocking property.

What's really unsettling in this phenomenon is that if you take a seriously good programmer and compare him/her to the near-magical best programmers, there's still easily a 2-3x difference and that's just unbelievable. For example, take the best programmer at your workplace and compare how much better people like Fabrice Bellard are? In other words, what's the Bellard ratio of your co-worker? 2x? 3x? Are five fingers enough or do you need another hand?

Then take an established, skilled professional of any other more traditional field and make a comparison to the very best individuals on the same field: what's the max skill factor you can find? The best race car driver to a good race car driver? The best golf player to a good, successful golf player? One point...what?

Higher multiples can be seen in fields that tie together more dimensions of skills into a more complex domain, such as stem fields. The best mathematicians or the best physicists are likely to be a lot, lot better than just good mathematicians or physicists. But in my experience the differences in programming are even greater. I tend to think that it's because of the field of domains computing appears in the sweetspot where it is both bound by practical, physical limitations of hardware and unbounded, abstract theory and imagination together form a huge complexity rabbit hole. If computer science could proceed without the limits of processing capability and the speed of electrical circuits it would converge with mathematics to a closer degree, and things would be less complex. And if computer science would only focus on few simple things at a time like early relay computers it would resonate more with electronics and physics. But throw in a few million lines of code that has to do something useful instead of just being highly theoretical or something complex enough that a simple, practical approach is out of the question, and you've got the essence of programming.

In other words, it's very easy to have things get very complex in computing and because we're still bound by the limitations of cpu and i/o we can't allow ourselves to go too deep in that complexity just to be able to even run any of our programs within a meaningful time, it becomes a jungle of tradeoffs and to navigate that jungle you basically have to be able to considered everything that's involved in it. That gets complex. And because handling complexity is hard it only makes sense that the very skill of handling behaves very exponentially: if the best decile is close to ~10x better than the worst decile, the best in that the best decile are likely to be close to ~10 better than the worst in the best decile.

  > For example, take the best programmer at your workplace and compare 
  > how much better people like Fabrice Bellard are? In other words, what's
  > the Bellard ratio of your co-worker? 2x? 3x? Are five fingers enough or
  > do you need another hand?

Let me share a personal anecdote here. I once visited a large service company that worked on a micro base station for LTE. The project manager told me with pride that up to 1500 engineers worked on the whole project, which comprised the eNB and the EPC code network with all the management support.

Fabrice Bellard, with his company Amarisoft [1] developed a full software based eNB on his own, with a collegue to develop the "EPC in a box" software.

Now it may not be apples to apples, and since then Amarisoft team grew a bit (to ~10 nowadays?). But even if their eNB scheduler is not exactly comparable to an E///, Nokia or Huawei it does work and is commonly use in labs for testing. While I'm not sure the service company micro eNB was deployed for real.

[1] https://www.amarisoft.com/

Of course it's a myth. That would literally mean that a 10x programmer can do in one day what it takes the average programmer two weeks. It's absurd. And we're not talking about the worst of the worst programmers in the myth. The myth is about average programmers. What's amazing is that people are willing to and want to believe this nonsense about themselves rather than realize it for what it is: arrogance.

If you don't know how to do a thing, or you don't know what to do, or you don't know what not to do, then someone who does know those things will have infinitely higher productivity than you.

I personally know people who can do in a day what a "regular" programmers won't be able to do at all, no matter the timeframe. I also know people who can crank out a couple of KLOCs of pristine, nontrivial C++ a day, every day, with ease (whereas a "regular programmer" does like 20, if you average it out).

Agree with infinite ratio argument. Debunking 10x "myth" may work for relatively routine and mundane tasks. For high complexity problems, where there is no off-the-shelf or book solution, you basically have two categories: majority of "competent" programmers, which can't do it and a small minority, which can. Productivity ratio is infinite in such cases.

The ratio in your case is not only infinite, it's also meaningless. So what if a hypothetical programmer can code while a baby can't? That's an infinite productivity ratio that's just as meaningless.

Say a technically ambitious software startup crashes and burns because, despite their best efforts, they never managed to get a working product out the door before running out of money.

If you were asked to do a postmortem on that, would you be insisting that the skill level and productivity of the programmers involved was "meaningless" to the outcome? Not even address it in the report?

This has nothing to do with the parent post which is comparing programmers of different skill levels. You want th compare programmers to non programmers. That's meaningless. Of course programmers can program and non programmers cannot. If a company hires non programmers and expects then to program without any training, then yes the skill of the non programmers is not the issue. The sanity and competence of the hiring manager is and is the only thing that would show up in the postmortem. You're going to blame the non programmers for not programming? What do you expect them to do?

Are you going to hire someone who is 4'11" and never played basketball before in the NBA? Are you going to hire a janitor to lead your space program? All so you can say that your qualified employees are infinitely better? That's ludicrous but exactly what you're suggesting.

> You want to compare programmers to non programmers.

No I don't. I said technically ambitious startup because I didn't mean a scenario where the group was too clueless to even program something easy. Read my comment assuming the programmers in that startup scenario are average industry programmers. Does your answer change?

My point exactly. And this also has secondary effects in terms of motivation: if a person knows exactly what needs to be done and how to do it, the startup cost is much less, so they'll just go ahead and do it, whereas someone who does not have this clarity and skill could sit on it for hours/days/weeks and just generally drag their feet because it's "hard" for them.

I would agree with you that it's absurd that the 10x programmer would be 10 times more productive than the average programmer at ALL tasks. But in certain specialized scenarios, it's easy to imagine one programmer being 10x or even infinitely more productive than the average. In other scenarios, the situation could be reversed.

Here is an example anecdote of what it means when people talk about 10x programmer. Here the "hero" of the story was infinitely more productive than everybody on the team.

Henry Ford had ordered a dynamo for one of his plants. The dynamo didn't work, and not even the manufacturers could figure out why. A Ford employee told his boss that von Neumann was "the smartest man in America," so Ford called von Neumann and asked him to come out and take a look at the dynamo. Von Neumann came, looked at the schematics, walked around the dynamo, then took out a pencil. He marked a line on the outside casing and said, "If you'll go in and cut the coil here, the dynamo will work fine." They cut the coil, and the dynamo did work fine. Ford then told von Neumann to send him a bill for the work. Von Neumann sent Ford a bill for $5,000. Ford was astounded - $5,000 was a lot in the 1950s - and asked von Neumann for an itemised account. Here's what he submitted: Drawing a line with the pencil: $ 1 Knowing where to draw the line with the pencil: $4,999 Ford paid the bill.

The von Neumann story is apocryphal. I've read it before but it was Nikola Tesla instead of von Neumann in that version.

https://www.snopes.com/fact-check/know-where-man/

Thanks for the correction . I didn’t mean to convey the anecdote as factually true , but only as an illustrative example.

For me the “10x” meme was never about 10x vs the average (which is a unicorn) but rather 10x variation where there are 10x variation within a group of developers. This is very much a thing in my experience, but it’s because you can have a 1/5x average developer and a 2x average developer. That’s a 10x difference. There is simply no limit to how unproductive you can be, but there is a limit to how much you can produce.

If you have a group with a very low productivity as average, then a single very good developer can be a 10x developer compared to that group. From the perspective of one of the 1/5 developers this senior “2x average” dev will look like the unicorn 10x dev. Perhaps this is where the myth comes from?

Your 10x developer is still only 2x the average. That's exactly my point.

I believe that it's your comment which is arrogant, not the 10x-er concept.

Have some humility and accept that there are people out there who are immensely more gifted than you are.

Aggressive mediocrity is distasteful.

I thought we were talking about the myth not just slinging personal attacks, but seeing as to how you resort to the latter, I assume you have nothing meaningful to say about the former, let alone an argument against it.

This is very short term thinking. You can't tell who is a 10x developer based on a two week project. That project is not complex enough to challenge anyone.

You can see 10x or 100x developers only when you have large complex problems that could take years to solve correctly by a regular developer (e.g. scalable load balancers, scalable messaging systems, decentralized trading engines, advanced ERP systems, scalable machine learning solutions). Some of these projects can and do take years to implement, but a 10x developer can deliver in 1/10th of the time.

For example, from my own personal experience, I worked and maintained a popular open source project for 5 years (part time), but I rewrote the whole thing (to the same standard as before in terms of meeting functional requirements/stability) in just 3 months (part time); that's 20x productivity.

The irony of 20x productivity is that during the rewrite, there are times when I would sit a whole day without implementing a single line of code; just thinking and searching for the best solution to a tiny sub-problem that had significant implications for future sub-problems. So the productivity gains were not a result of faster coding; it was the result of being able to foresee future problems and considering all possible alternatives and possible permutations of design decisions.

So by the 10x logic, that would mean that 10x programmer will:

- Finishes 100 sprints, in the same time that your average programmer finishes 10; Given that they are in the same size and duration. This one seems pretty debunkable if you ask me; Never in my life have I seen that kind of proficiency

- Bring 10x more value to the table, compared to the average developer. Sure, maybe - a 10x programmer could probably detect and fix bugs, or improve systems, which will be of 10x more value than the rest. You see this in sales, where rainmakers will literally bring in orders of magnitude more revenue.

But tbh, I don't think it's really possible for some single developer to be 10x, other than through short periods of time. To be consistently 10x over the course of many years, seems extremely unlikely.

However, I DO think it's possible to bring in 10x results if you're in charge of the more strategic aspects - but that seems to go more towards leadership, than anything else. But then again, is it possible to be 10x on a consistent basis, throughout your whole career?

Most of the successful software startup products in Silicon Valley were written by one guy. Same with most CAD and drawing products.

Those are 10x to 100x guys who write 100k to 1M LOC per product. Then they do it again.

One of the points of the article is that the environment is a huge factor to productivity. In the environment you talk about (startup program), you have zero code review, full comprehension across the entire system (because you've written everything) and zero debates about the proper way to proceed.

It's the environment that every programmer yearns for, and the environment that every programmer is 10x their normal proficiency (certainly I am).

This is how most programmers can make massive contributions to a 2h/day side project in the evenings compared to their normal 9-5 output at work.

The insidious problem is that people not in the trenches look at the output of programmers in these isolated environments and deduce that this output is constant for all environments.

If you want to reduce output, add more programmers to help.

Reply on news.ycombinator.com