High costs and negative value of pair programming | Hacker News Reader

High costs and negative value of pair programming

namcookanalytics.com

41 pointsllambda47 comments

I don't mind a good critique of pair programming -- it deserves a hard look to think about its pros and cons -- but I do take issue with a paper written in this way.

I want to mention the opening line in the introduction: "The pair programming method could only exist in an industry that does not measure well and does not understand economics." This argument is speculative and unpersuasive.

Speaking of measurement, I take issue with this paper because it suggests that lines of code (LOC) is a good metric. The author demonstrates a lack of economic savvy by writing, "As can be seen from table 1 the economics of pair programming are severely negative." That table uses lines of code (LOC) as the key metric. There are so many problems with the choice of such a wrongheaded metric. First, writing better software often involves writing less code! Second, as a commenter points out, knowledge transfer is very important. Third, pair programming may build team cohesion.

Many industries use variations of pair programming, as the author recognizes. That said, I think the author fumbles in suggesting that quarterbacks trading time is similar to pairing. I wonder why the author didn't look at apprenticeships in manufacturing, where skills are demonstrated in a hands-on way.

Finally, as a model-builder myself, it pains me to see weak, flawed models baked into papers such as this. Take this section of text: "Table 1 is extracted from a pair-programming calculator developed by the author. It allows researchers to input a number of variables including staff compensation, application size in lines of code, and coding speeds for both pairs and individual programmers."

Such language, to me, makes it appear that the author is trying to make his claims seem "mechanical" -- just the matter of turning a crank and running some calculations. Such a claim is silly and a sham, of course. The choice of what to measure and how to weigh it is central. This paper makes no serious attempt to promote such a discussion.

In summary, this paper did not make a serious attempt to understand the true (economic, yes!) value of pair programming, in my opinion. If I were to guess, the author may well have had his conclusion in mind before writing the paper or doing his research.

This reads like it was written by someone with a management background, not a development one. No space given to future maintainability or bugs, only on raw productivity. When I saw that his entire analysis rested on LoC metrics, I could only shake my head. And his comparisons to other industries/activities were just laughable. Apparently pair programming is comparable to having two generals.

Did you not actually read the article? He also compares quality in terms of defects.

A "defect", I'm gathering from the context of the article, has little to do with forward maintainability and seems only to relate to the results of some QA testing heuristic.

I think the coding environment being analyzed is already well-defined and the projects being coded are one-offs being sold to clients where maintenance is done through a lucrative support contract. In this setting, a "defect" would be anything keeping the product from shipping. Anything else is overlooked and untracked.

It's the software equivalent of shoveling whatever crap you can find into a sausage casing, pricing it like fine steak, then selling your own medicine when it makes them sick.

Code reviews are a much more efficient way of achieving this.

It's very, very hard to do objective research on this topic. There are tons of reasons that LoC isn't a great way to measure productivity, including the times when you refactor existing code such that you delete thousands of lines of code, and your "LoC output" is negative for a time period. The Tale of Two Programmers is the epitome of such tales, IMO. [1]

That said, most of the time developers with roughly equal skill levels end up writing new code to implement new features. And LoC is about the best metric you can use. If you factor in expert developers writing fewer LoC to achieve the same goals, then the results of the paper are magnified, because if an expert's 1000 LoC is good for 2x the number of features as an average developer's 1000 LoC, but the number of bugs is comparable, then it's an even greater advantage to use the expert.

>No space given to future maintainability or bugs, only on raw productivity.

They do talk about bugs; they say in the summary:

"There are no significant quality benefits compared to a single programmer who uses static analysis and inspections."

And later on in the paper they compare delivered defects.

I'm easily in the "expert" end of the spectrum as a programmer; I've never seen a benefit in pair programming that would justify two salaries. I'm sorry, and I'm sure I'll be slammed for this, but pair programming is a crutch to boost programmer productivity of average or below average developers. "Average" developers would have to make 1/3 or less of my salary to make it worthwhile, economically.

In my opinion pair programming's only valid use is in training a junior developer (or allowing two junior developers to support each other, though in the paper the bugs delivered were far worse from two junior developers than from a single expert developer -- but you can't always hire an expert developer).

Otherwise it's a waste of money. If you can pull it off as a developer in some big company that has lots of money, by all means, have a blast. I thought HN was more about bootstrapping and getting code working as quickly as possible, and pair programming is not the way to accomplish that.

[1] http://mail.linux.ie/pipermail/social/1999-October/000483.ht...

That's a pretty good rationale. I'm not saying pair programming is good/bad, I don't have enough experience with it to be able to call it. But I do know management thinking when I see it. It's always riddled with assertions, lamp post search methodology, and clueless comparisons.

There is a spectrum of pairing possibilities. You can pair anywhere between 0 and 168 hours per week, in theory. I would not assume a linear relationship between that number and the value of pairing. :)

For me, a few hours of pairing a day has a lot of value. If I could, I would do that regularly. When I use it to focus on hard problems, it can improve my thinking, it improves knowledge transfer, and it builds team relationships.

The issue is less that LoC is universally a useless metric, and more that it's a metric whose nuances need to be accounted for carefully when used in analysis. The author gave no indication of such accounting, and so regardless of whether or not we would tend to agree with the conclusion, we should reject the analysis.

Really, the biggest problem is the presentation. The whole thing reads like a back of the envelope calculation turned blog post dressed up to look like an industry white paper. If this were just a blog post with a few more caveats thrown in, it wouldn't be nearly so bad.

It's still a Really Hard Problem.

How would you account for LoC nuances, exactly?

As I pointed out, the better the developer, the fewer LoC produced per "feature point" or however you want to measure it. So any analysis that doesn't take into account the nuances would be skewed against the expert developers.

This would mean that, if the study purported to prove that two average programmers were more productive than an expert, I would have to take the conclusion with a grain of salt (or reject the analysis).

But given that despite the disadvantage, experts were still shown to be more cost effective without pair programming? I don't see why we'd need to reject that conclusion.

To me, it reads like the conclusion and summary of a big statistical paper but ... there's no statistical paper. When I saw "Pair programming costs about 2.5 times as much as one programmer working alone" in the abstract I was really looking forward to seeing the data that led to that conclusion since the main problem with pairing discussions is the lack of data and surfeit of anecdote.

But there wasn't any useful data because the source, timing, measurement practices, and sample sizes for the grand summary tables were not mentioned at all. It's like reading an article that states very firmly that the moon is made up of precisely 89.24397% green cheese. It sure sounds very specific but where DID that number come from?

I hope the author publishes more details about how these numbers were calculated because we need more data about pairing but I don't really expect that he will.

More heat than light, really.

I stopped reading just after seeing LoC is used as metric.

I found the article confusing because it begins by noting the deficiencies of studies on pair programming and then presenting data from... where exactly? I couldn't tell.

And then:

It is obvious from table 3 that the best economic results do not come from pair programming but rather from hiring and using expert single programmers.

Well... sure... of course, and good luck with that! Meanwhile, in the real world, while you're waiting for your expert programmers to submit their resumes to your company, pair programming is a great way to bring new/junior devs up to speed. In fact, I think that may be one of its greatest values.

If you have two roughly equally skilled devs, it's possible that you'd be better served by working individually combined with good code reviews, though I don't think the article made much a case for that.

"Table 1 is extracted from a pair-programming calculator developed by the author."

One could claim that the author made up numbers to support the author's argument.

And hasn't pair programming been around at least since the 90's? doesn't seem that new to me.

"Object Oriented" programming has been around since the '50s. [1]

"Functional Programming" has as well. [2]

And the "Waterfall" development model also has its roots in the 1950s. [3]

"Extreme Programming" (1996, and a book in 1999) [4] was the first popular methodology to advocate "100% pair programming" as a major component. (I've read the book: They say that you have to use all of the practices, all the time, if you want "Extreme Programming" to work.)

Point is that, depending on how old you are, the late '90s can qualify as "recent."

[1] https://en.wikipedia.org/wiki/Object-oriented_programming#Hi...

[2] https://en.wikipedia.org/wiki/Functional_programming#History

[3] https://en.wikipedia.org/wiki/Waterfall_model

[4] https://en.wikipedia.org/wiki/Extreme_Programming#History

Line of code per hour...wow.

I am actually a fan of pair programming, I have seen and experienced it work - in an environment where you are designing and developing new things day in, day out - pair programming or as I like to call it...collective collaboration - is invaluable - it prevents following rabbit holes and single point of failures - and solutions get developed, much, much faster.

Of course for this to work you need a smart team of engineers and scientists who respect each other, who are confident in their ability but also recognize that two heads are better than one.

Anecdote time: I was working on a project where for a while I was the sole scientist/engineer - I spent a fair bit of my time bouncing ideas off a system architect who worked closely with my team. In a few months I was assigned an engineer who's knowledge of hardware absolutely flawed me, together we were able to knock down feature requests so fast it we became quite renowned in our little org...we hardly ever stopped talking during the day (this pissed off a couple of people to no end...but "you can't argue with working code and happy users" as my manager used to say)...I left that job a few months ago to explore the world a bit more but when I left the team was much bigger but the philosophy remained the same - talk, argue, yell, laugh, cover whiteboards and interrupt and genius happens.

YMMV, I've met developers who's ego could not adjust to that model or who "couldn't work in noisy environments" and it wasn't for them..but I always tr to seek out a noisy, dynamic team, because they are usually doing awesome things and having fun :)

Pairing benefits projects that still have uncertainty in design. Two people dealing with minutia just leaves one bouncing a ball off the wall while the other types.

I just looked at [1] which compares the productivity of a pair of programmers to that of two individual programmers. The paper claims an increase of 15% in man-hours while the author of this piece is claiming that a pair of programmers is 15% slower than a single programmer!

(Note: I've never used pair-programming and don't really care one way or the other. But this analysis seems terrible to me.)

[1] The Costs and Benefits of Pair Programming http://collaboration.csc.ncsu.edu/laurie/Papers/XPSardinia.P...

I find that pairing more during a kick-off / design phase of a project has been beneficial to my team, but I don't think pairing yields many benefits once design goals have been well-defined.

Isn't it more efficient to actually do the design before you start coding?

You must be new here.

Not entirely sure what you're implying here. Are you saying that a lot of people on HN are bad programmers or something? I'm certainly not a new programmer if that's what you mean, and I know a thing or two about rapid development of high quality code (having done it for about 30 years).

Would you recommend any books? That would help me a lot at this point. :)

I read this and it's still unclear to me whether the author conducted any experiments involving pair programming. If he did, why hasn't he described them in detail? If he didn't, where did his data come from?

I guess he didn't, and where the data comes from is "a pair-programming calculator developed by the author"—in other words, software he wrote that gives whatever he decided the answers should be, and whose code he isn't (at least in this piece) publishing.

Problems: 1. He quantifies productivity in lines of code. 2. By doing so, he only considers the development but not the maintenance cost of software. 3. He is not even internally consistent. He states pair programmers are 15% slower than individual programmers but then lists then as only creating 75% of the LOC an individual programmer does. 4. He pulled that number out of nowhere to begin with. The only referenced papers are his own. 5. The numbers given for errors are equally bogus. 6. At no point does he define his different tiers of programmer(novice, average, expert). 7. It would seem he assumes the same wage for all three tiers, which is completely unrealistic. 8. Point 7 being the case, the conclusion of the entire paper is that the best thing to do is hire a bunch of expert programmer to replace your average ones while paying them the same wage. Even thinking that you can identify with a high level of certainty who truly great programmers are is a rather lofty goal, and pg has an article or two talking about how difficult it can be. Add to that the fact that even google has come out recently and said that their methods of interviewing programmers has largely failed to identify those that would be successful at their company.

The two comments by Gerald Weinberg (at the very bottom of the page) make a good case for the advantages of pair programming.

I wonder if that's the same Gerald Weinberg I'm thinking of-(http://en.wikipedia.org/wiki/Gerald_Weinberg). I've read a few of his books. I am inclined to think 'yes'.

I'm pretty sure that you're right, and this is the Gerald Weinberg. (Or at least they both have "M" as their middle initials.) He's written quite a bit on programmer productivity.

A analysis done on based on LOC is purely laughable. I have done pair programming lot of times and some benefits are

1) For building a complex system in a short time you have to take lot of critical decisions under pressure implication of which will be felt even after the product is shipped. During pair programming your partner can stop you from making many such decision which actually saves LOT of lines of bad code. That is why this study is totally worthless

2) Pair programming is perhaps the best way to give training to someone new to the system. It is much more beneficial than the best of documents. It is like roaming around the city with a experienced guide than reading from a book

3) if you do pair programming with lots of experienced programmer you will actually learn to see things in s different perspective. You will start doing things in a different way.

Inspections are mentioned a lot. This is not a word I hear a lot of developers use, so I'm guessing this article is not written by one.

The cost of these has been completely externalized. Does the 85% reduction in defects come from a casual code review? Or are 3 senior developers doing Fagan inspections?

When an inspection reveals bugs, do we take into account the increased time to fix the bug because developers are informed about them after having forgotten some context?

Is static analysis incompatible with pairing?

Where does the 1.5 vs 1.6 defect rate come from? What about the rest of the numbers?

If a pair produces less LoC than someone soloing, couldn't this actually be a good thing? I know that the most productive coders I work with actually tend to delete more lines of code than they introduce.

I think the best way to compare individual programmer vs pair programmers is to look at it this way:

When you work yourself on implementing something, how will compare these two:

YOU (60% of your capacity) vs YOU (>90% of your capacity)

My philosophy is that pair programming is just like individual programming but just increase the output efficiency of one of the member in the pair by 30% or so (if the pair is a good pair). Thats it.

An individual programmer at his >90% capacity will outweigh (in terms of output/cost) two programmers at 60% capacity.

So, if you have to pair them to get a single programmer with 90% capacity, do it.

I don't think pair programming can be quantified. This article doesn't take into account the learning that happens while pair programming. The examples given of when pair programming does and doesn't work are irrelevant.

I agree.

This fails to account for: 1) The cost to train a novice developer & 2) The value if having two people equally acquainted with a code base (thereby reducing dependency on a particular employee)

Summary: 2 independent developers write more lines of code combined then 2 working as a pair. Thus pair programming is bad. How quaint.

... not exactly. he also examines the number of defects turned in by different pairs of programmers compared with single ones, and examines some of the financial cost of pair programming. Its a better read than you make it out to be.

"Pair programming is a new method", I could only get so far...

http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=913816...

Date of Conference: 2001

Does anyone honestly believe that a metric based on 'Lines of Code' has any validity at all?

I sure hope not, but have seen plenty of managers use it. I would guess refactoring would be really bad for those people. The author would probably think this guy is a horrible employee http://www.folklore.org/StoryView.py?project=Macintosh&story...

Is there an engineering profession where using more raw material to get some function is actually a good thing?

This comment thread sums it up very well:

http://namcookanalytics.com/high-costs-and-negative-value-of...

"New"!? It's been around for at least 15 years!

Thanks for posting this article, it's good to see someone try to quantify the benefits of a technique that divides the industry.

the only thing that sucks more than pair programming is bullshit research papers penned by condescending egotists.

Experimental results or GTFO.

very polemic.

Clearly measuring productivity in lines of code is the correct way to determine whether goals are achieved on schedule.

Clearly a lot of HNers aren't very proficient at reading.

Reply on news.ycombinator.com