How to measure Lead Time for Change

DORA and Accelerate (Forsgren et al.) define “Lead Time for Change” as “the amount of time it takes a commit to get into production.”

By being specific about how and when you take the measurements, you can create a Deployment Lead Time metric that can help your platform team identify improvements to reduce Lead Time for Change across multiple teams.

Change = Deployments || Releases, but Deployments != Releases

Software changes are done in organisational events such as releases, or can happen frequently throughout the day such as deployments. However, releases often require collaboration with enabling teams such as marketing, legal, and customer operations to ensure a successful outcome, and are a form of organisational change events. Deployments are technical change events that do not require the same level of collaboration across the organisation. They can be done frequently throughout the day and with sufficient preparation, such as using feature flagging capabilities. They do not cause incidents that impact the availability or the user experience.

Release lead time, or cycle time will vary significantly depending on how the organisation has optimised for the flow of work, and significantly reducing the cycle lead time can be outside the scope of the interactions between a platform team and the product teams it works with. Deployment lead time however, can be optimised by interactions between a platform team and the stream-aligned product teams it works with.

Measuring deployment lead time will provide information on the common path to production across teams, whilst the measurement of cycle time will inform on the ways of working of the team and the other organisational activities that have to happen to release change to users.

Deployment lead time is good for comparing across teams without getting too involved in specific teams’ ways of working

If your platform team aims to optimise the path to production across many teams, prefer starting the Deployment Lead Time measurement from when the commit hits the main branch until that commit is deployed to production deployment (Commit D in the diagram).

By measuring when the work is ready to go to production, we gain accurate data on the process and pipelines required for the path to production that is easily comparable across teams and there is a reduction in bias for specific team’s ways of working like branching strategies, peer-review approach, testing strategy.

When measuring from the first commit of the branch (Commit A in the diagram), we’ll produce the team’s cycle time measurement. It’ll include the time it takes to produce the work, integrate that work with others, and peer-review it (if a separate stage).

The timing of the first commit being created can be easily gamed by engineers. Still, by avoiding measuring from the first commit for deployment lead time, we leave individual team preferences and ways of working alone and measure when the work from that team is ready for production.

Mean averages are good but do pay attention to your 50th, 90th, and 95th percentiles

Watch the median (50th percentile) to understand how you’re doing with most of your changes getting to production, and the long-tail percentiles (90th, 95th) to understand what happens when things are weirder than usual on their journey to production.

When making changes to production are happening quickly and safely, with a team that has a good understanding of how that software operates, you’ll find your long tail comes towards your median.

How to measure Lead Time for Change

There are many potential points in a typical developer’s workflow that you can use to measure how long it takes for a commit to get into production, be wary of accidentally measuring cycle time, pipeline time, or time to create value.

Instead, measure Deployment Lead Time to ensure your platform team can take action on the metric’s results and meaningfully impact it by changing the user experience of the product teams.