Using merge time is a terrible metric, perhaps worse so than deadlines because it can be more effectively weaponized. What would the incentive be for the developers other than to rush the code review process? Merges are not where you want to be rushing anything, but rather the opposite. If project deadlines are necessary, allowing code review its due time affords developers the ability to informally schedule things without sacrificing craftsmanship for what in reality is a vanity metric. Some code needs the be carefully considered and given time while other code doesn't necessarily need much review or worry at all, but no one can tell that by looking at mean time. If a developer is asked why some tasks had a longer than average mean time, then now they have to waste even more time by explaining themselves. In the worst case, the incentive to rush the review process results in more time wasted on bugs that could have been caught before they even had a chance to be merged.
Am I misunderstanding your view of how mean time to merge would be used?
The questions of whether features are appreciated by users, or which bugs should be fixed or not, or if a product is feature complete or needs more, are questions of business, and not developer, productivity and efficiency.
And regarding documentation, I consider that an integral part of code/software that can be judged similarly w.r.t. quality and impact, having its own features and bugs.
I think that qualitative metrics like satisfaction and collaboration could be helpful, especially when combined with traditional metrics like mean time to merge a code change. Taking my previous example of overworking or cutting corners to achieve high numbers, a qualitative metric for something like satisfaction might indicate a problem where a work output metric wouldn't.
But I think that any combination of metrics will be an oversimplification that could lead to problems if they are the only thing that matters. I'm not sure where the balance lies. I like that metrics can offer an objective view of performance and make it easy to spot trends. But I am wary of them oversimplifying things and dehumanising the team.
I think this depends on how strongly leadership/management tries to use this metric to change behavior, but in a company like mine where we have a lot of business dependencies, contracts etc which rely on predictions of throughput, this is an important metric for us to predict timelines. It's not used against a team or platform and it is primarily used internally by management and not so much engineering teams which I think is the right way to use it.
Absolutely. It’s the old descriptive vs normative saw. We should be very interested in measuring how productive we are. But how do we really know how productive we should be?
I think trying to answer that is hard to impossible for a developer, because you can always automate or abstract further, but the cost to do so marginally increases and you probably won’t know the full cost until it’s already realized, at which point, requirement and prediction are obviated.
I know I’ve had my fair share of negative experiences with a scrum team that wanted ever faster velocity and used burndown charts as a stick with no carrot in sight.
I could see how a satisfaction metric would counterbalance a work output metric. The team will probably feel less satisfied if they know that corners are cut or if they are having to work unsustainably.
I'm currently experimenting with using "Mode time" as I think it is less susceptible to data skew from outliers. See example below:
https://oss.gitsense.com/insights/github?p=days-open&q=days-...
For popular open-source projects that I used in the link above, the Mode time to merge is less than a day which is quite good in my opinion. And as you can expect, if you look at larger pull requests (>=10 file changes) the overall percentage drops by half as the link below shows:
https://oss.gitsense.com/insights/github?p=days-open&q=pull-...
I think what the link above shows is, you can't just willy-nilly use merge time to measure productivity, since there are a multiple variables at play.
Full disclosure: The link that I referenced is my tool