Delivery Lead Time in Practice

> Not fond of these metrics. Feels easy to gamify and hack. I can just push commit a when it's more mature, good I improved the metric, but reduced tinkering, collaboration with devs and product and probably released a subpar solution.

This is not gamed/hacked -- it's the metric working as designed. It is meant to measure purely how fast you can get commits into production. If you have found a way to do that faster, then the metric should increase to reflect that.

So there are two aspects of the problem: are you delivering fast, and are you delivering something valuable?

The delivery lead time doesn't even pretend to measure whether you're delivering something valuable. You need a different metric for that. And in your example, that would go down by your practise.

Author here. The targets you set for Delivery Lead Time and Deployment Frequency should be determined by your organisation and your team. Context matters. I've been at an organisation where upper management introduced the Accelerate/DORA targets verbatim. These numbers did not resonate at all with engineers and so saw little to no adoption.

Accelerate was published four years ago and understanding of the 'four DORA metrics' has matured. Štěpán Davidovič wrote a good piece Incident Metrics in SRE [1] where he pulls apart the Mean Time to Recovery Metric. I've written an overview of the DORA metrics and what to use instead - SLOs & SLIs [2]

[1] https://web.archive.org/web/20210917022148/https://static.go...

[2] https://isthisit.nz/posts/2022/state-of-the-dora-devops-metr...

External benchmarking is a tricky thing, especially with a number like this, which is derived from a self-reported survey. The important thing is that those teams perceive they can quickly deploy a change. Perhaps they do, at times, but as the author of the OP points out, that’s not really revealed in the source literature, and being able to isn’t the same thing as often or frequently flexing that muscle.

I would start with “can you even measure this”. Just getting something like this instrumented for your team is probably going to drive improvements.

Then you can put it on a Shewhart/PDCA Cycle driven primarily by the team itself through its retrospective and prioritization process. “Has this metric slipped? Do we feel like there are blocks in the deploy pipeline? Are those blocks more important than other work right now? What changes does the team feel most invested in?” Fostering that conversation is really important.

Grabbing these metrics, poorly measuring them, and then trying to hold a team accountable to some arbitrary, external, non-contextual measure is a sure way to kill morale and productivity, hinder future leadership efforts, and waste time and resources and non-priorities.

Of course, doing nothing will accomplish the same.

We do this kind of work with hospitals and their process metrics. Our measurements are always based on raw data, never self-reported metrics. Frequently, team’s self-reported perceptions prior to measurement is the most telling thing. When I worked in k-12 Ed, the same was true with teachers and their students: they perceived their students as great writers, but they were often much worse when measured quantitatively by normalized graders. What is important in both cases isn’t blindly pushing to improve those metrics, but developing an understanding of why that perceived difference exists, why the numbers are the way they are, and ultimately, whether and what kinds of differences you can make. In the case of hospitals, there are some units/teams where you don’t want high performance numbers, because it might lead to higher error rates. In Ed, there might be limits to what you can do as a single-subject teacher focused on a single year in a group of children’s lives. But you can understand each groups challenges and strengths a little better and maybe adapt accordingly.

Blindly pushing top-line metrics is not the same as blindly doing the same thing with no measurement or learning. Both are bad.