Measuring developer productivity? A response to McKinsey | Hacker News Reader

Measuring developer productivity? A response to McKinsey

newsletter.pragmaticengineer.com

28 pointsshrikant15 comments

I am glad this discussion is being brought to the fore. In my experience, far too many managers are clueless about what engineering actually means. They see it like warehouses or factories where the number of units shipped == output. But in engineering, number of PRs or commits or lines of code are meaningless. In fact, setting these output targets take away time from engineers in doing things such as meaningful design choices, system health investigations, cross-team situations.

I see software engineering more as a team sport than as an individual sport. In team sports, each individual sacrifices some metrics for the good of the team.

Here is my rant about how soccer teams would perform if they were evaluated like the McKinsey-style nonsense becoming pervasive in the tech industry: https://medium.com/@NTDF9/if-soccer-managers-did-performance...

This was a great writeup. And shows that this management style is not just about programming, but any activity at all. The article mentions sales as having just such maddening performance metrics and just brushes past it as ok/obvious/normal while it's just as crazy as your football example.

I lol'd at this quote from the original McKinsey article:

> For example, one company found that its most talented developers were spending excessive time on noncoding activities such as design sessions or managing interdependencies across teams. In response, the company changed its operating model and clarified roles and responsibilities to enable those highest-value developers to do what they do best: code.

How dare our senior dev leads spend time _designing_, we need hands on keyboards!!!

That quote alone is why I am glad I did not read the article in full.

I was onboard with this until

> CEOs and CFOs are increasingly frustrated by [...] software engineering is too nuanced to measure, when sales teams have individual measurements and quotas to hit, as do recruitment teams in the number of positions to fill. The executive reasoning goes: if other groups can measure individual performance, it’s absurd that engineering cannot."

It's absurd to think that those other can be! THAT is the flaw!

Those measurements and quotas of sales/marketing/recruitment/etc are hugely corrupting! They lead to selling features that don't exist and cannot exist. They lead to bad hires. This same incorrect thinking leads directly to Teaching-to-the-test. To bureaucratic grant proposal systems. And most likely tons more that I just don't know about. Measuring the wrong thing is pervasive.

That the article thinks other fields can be measured without corruption totally undermines the entire article. This is exactly why McKinsey are stupid enough to think programmers can be measured: because they are not programmers. Exactly like how politicians think they can measure teachers: they are not teachers.

The article gives an example of how a recruiter is corrupted by metrics, leading to worse outcomes, so they're well aware of the problem you're describing. I imagine Kent Beck would say (and I'd agree with him) that your line of argument won't change anyone's minds, and will just lead you to being excluded from the conversation. There are some situations (and examples are listed in the article) where measurement is necessary, and the best you can do is hope to measure things that give helpful results.

Developer productivity is fundamentally unoperationalizable for all meaningful ways. The desire for which is a foolish consistency ie: the hobgoblin of little minds.

Every one of the listed decisions can be made effectively without an operationalization[1][2] of developer productivity and the sooner a playbook of methods takes over, the better.

1. https://en.wikipedia.org/wiki/Operationalization

2. Campbell,Norman Robert. Physics The Elements 1920 https://archive.org/details/physicstheelemen029733mbp

Productivity of artists can be measured if they are reduced to wall painters. The mind of a bureocrat is like a small dusty cubicle with labelled boxes and step-by-step guides pinned to the walls. There is a lightbulb in the cubicle, but it's very dim. When he needs to manage something, he carefully picks it apart and bins it into those little boxes, and maybe updates one of his step-by-step guides. Things that don't fit into his boxes or can't be managed by a step-by-step instruction cause him a vague sense of anxiety, so he throws those things away. He loves when everything is squarish, lookalike and predictable. They don't understand creativity the same way deaf men don't understand music. But this bureocrat type is very valuable in large organizations as they keep things in order.

> Productivity of artists can be measured if they are reduced to wall painters

Even wall painters are surprisingly hard to measure. Painting the wall without noticing issues with the underlying preparation will cause an unhappy customer at the end. Painting the wall with the color the customer has chosen without making sure the customer realizes that the color looks WAY darker on an entire wall compared to the small sample, will also lead to an unhappy worker.

It's very popular to think there are other professions that are easy to measure, and yet it's never your own productivity that can be measured.

It's madness that we aren't just saying that productivity of ANY KIND is damn hard to measure!

My company uses scrum to try to track engineering output. My boss looks at metrics like line counts. It is extremely stressful to be on a team of people who are all worried about losing their jobs and will accidentally (?) commit your changes to their MR. It's extremely stressful to spend all day doing research tracking down a weird issue in prod, deploying a hotfix finally, and realizing that all of that effort, which did not go towards anything you were supposed to be working on that sprint, but was nonetheless necessary, resulted in a one line change. 8 hours of your life, feeling highly stressed, maybe skipping lunch, results in fixing one typo somewhere. You think about how you'll get a bad performance review despite working hard.

I wish I was on a team of engineers who just did their work and didn't worry about gaming the system. I wish the engineers I worked with had enough time to be careful that such small mistakes, which are difficult to track down after the fact, did not make it to prod to cause our customers grief. I wish that management was competent enough to know that adding a missing quotation mark to the repo isn't just adding a missing quotation mark to the repo.

I assume you are actively searching for a new job?

I hope that place has exit interviews later so you can freely tell them on the way out how bad this is. But considering how bad it is, it would surprise me.

Today is actually my last day. If any sane companies have openings, please hit me up.

> metrics like line counts

Send your boss a link to "Negative 2000 Lines of Code"[1] and ask him if he thinks Bill was a shitty coder.

When I was scrum master for two teams I, too, tracked some metrics (for feedback and improvement, not for pay or career (ab)use).

While I did track the number of story points delivered and user stories closed per sprint, one of the things I was more interested in was how accurately the teams managed to hit their estimates (regardless of what they were). In that perspective, delivering too much too quickly was just as undesired as the opposite. We used this to refine the way that we did estimates, story refinement, and sprint planning.

Naturally these were team-based metrics, because software development is a team activity.

[1] https://www.folklore.org/StoryView.py?story=Negative_2000_Li...

> Naturally these were team-based metrics, because software development is a team activity.

Applying those same metrics to individuals is too tempting for a subpar manager. The data is right there!

I'm honestly glad that we're going to start doing this - it'll be great to have accountability within eng orgs

Reply on news.ycombinator.com