##  [Mastering DORA Metrics: A Practical Guide to Measuring and Improving Software Delivery](/index%2ephp/blog/mastering-dora-metrics-a-practical-guide-to-measuring-and-improving-software-delivery) 

    September 19, 2025 

  ![Mastering DORA Metrics: A Practical Guide to Measuring and Improving Software Delivery]()

6 min read 

- [freya fusion](/index%2ephp/blog-main-categories/freya-fusion)





 

## Measuring DORA Metrics

#### Data Collection: Manual vs. Automated

Automated data collection is the best way to avoid inaccurate metric which further leads to incorrect Business decisions. However, automation of every parameter involved in calculating a metric might not be possible due to constraints. In such cases, agree on a manual approach that is reasonable and aligned with Industry standards.

##### Lead Time:

Measured based on time taken by a feature to move from Business request to production. This can be measured by employing multiple tools at each step.

- **Automate testing & deployments** (CI/CD pipelines).
- **Reduce batch sizes** (smaller, more frequent changes).
- **Improve code review & approval processes**.
- **Monitor bottlenecks** (e.g., long QA cycles, manual approvals)

**Manual method:**

When **required tools/plugins** are not available for **automated** calculation of Lead time, a **manual** approach can be adopted. you can **manually** calculate **Lead Time** in Azure DevOps (ADO) or similar systems using **raw data extracts.** Here’s a step-by-step approach:

**Data Extraction**

- **Source:** ADO Work Item Query or API.

**Scope:**

- Filter for **Features/User Stories** marked as **Done** (i.e., deployed to Production).

**Extract:**

- **Created Date**(when the work item was requested).
- **Closed/Completed Date**(when marked "Done").

**Formula** for Feature/user story Lead time:

- **Lead Time** (per item) = Closed Date - Created Date

**Formula** for Project-Level Average

- **Avg Lead Time =** Sum(Lead Time per item) ÷ Total # of Items

**\* Note:** In cases where **planned features** are created **ahead** of time, **lead time** metric would **not** be a true reflection and in those cases, **Cycle time** can be used as an **alternative** metric.

##### Deployment Frequency:

- Count the number of deployments to production over a given period (e.g., per day, week, or month).

**Data Extraction:**

- Use **CI/CD tools** (e.g., GitHub Actions) to log count of deployments.

**Formula** for Deployment Frequency:

- Total number of **deployments**/Number of months

**Formula** for Project-Level Average:

- Number of **deployments** of the **project**/ Number of months

\***Note**: As per Industry definition, **Hotfix** deployments are **excluded** from count of deployments.

##### Change Failure Rate:

**What Counts as a "Failed Deployment"?**

- Rollbacks (reverting a release due to issues).
- Hotfixes (urgent patches post-deployment).

**Tracking Method:**

- Integrate with **CI/CD pipeline** deployment failures (e.g., Jenkins, GitHub Actions) to flag failures.
- Use **incident** management tools (e.g., ADO, Jira, ServiceNow).

 **Formula** for Change Failure Rate:

- Number of failed changes÷ Total number of Changes× 100.

**Manual Method:**

To calculate number of failed changes, count of **support incidents** that are categorized as “**Software Bug**” can be used. This **divided** by total number of **changes** released gives us CFR.

**Organizations** need to have the **incidents** classified **periodically** for the metric to be **accurate**.

##### Mean Time To Restore (MTTR):

**What Counts as "Recovery Time"?**

- **Start:** When the failure is detected (alert triggered).
- **End:** When the system is fully restored (e.g., rollback complete, hotfix deployed).

**Tracking Method:**

- **Incident management tools** (e.g., ADO, Jira).
- **Monitoring systems** (e.g., CloudWatch, Prometheus) to log resolution times.

**Formula** to calculate MTTR:

- Sum of Restoration Times ÷ Number of Failures

**Manual Method:**

Sum of **Restoration Times** can be obtained based on the **support incidents** that are classified as “**Software Bug”.** Difference between **“Created Date” & “Closed Date”** in days can be used as **numerator**.

Use **count of incidents** classified as “**Software Bug**” can be used as **denominator**.

#### Connecting ADO, CI/CD, and Monitoring Systems

Collecting these **metricsperiodically** and **optimizing** them help Organizations achieve **efficiency** in the way they **releasesoftware**.

Having **effective tools**, **plugins** and integrating them helps **automate** most of these tasks and generation of **metrics** and taking appropriate **decisions**.

**Azure DevOps(ADO)** provides required **metadata** for capturing the raw data for **calculating** these **metrics**. DevOps team needs careful **evaluation** considering the **futureBusiness** needs.

#### Visualization Tools and Dashboards

**Azure DevOps(ADO)** provides **dashboardcapabilities** with these **metadata** and widgets to capture some of these **metrics**. However, ensuring that metadata is **updated** for **every work item** is essential to generate **accurate metrics**.

Alternative tools like **Power BI, Grafana** can be employed for enhanced **visualization** and **reports** for **stakeholders**.

## Challenges and Considerations for Organizations

#### Data Collection & Usage of Tools

##### Data Integrity and Tool Limitations

**Challenges:**

- **Inconsistent Data Sources:** DevSecOps pipelines pull data from disparate tools (CI/CD systems, Scanners, incident trackers), leading to mismatched timelines, false positives, or gaps.
- **Manual Reporting Bias:** Human-curated metrics (e.g., incident reports) often lack standardization, skewing trends like **MTTR** (Mean Time to Recover).

**Solution:**

- **Enforce Traceability:** Use immutable metadata (e.g., Azure DevOps’ **pipeline IDs**, Git commits) to extract data related to code changes.
- **Validate with Cross-Tool Correlation:** Combine data from SonarQube (code quality), ADO to flag anomalies, Industry standards to validate metric data

##### Availability of Tools & Plugins

**Challenge:**

- Organizations may lack access to integrated DevSecOps tools (e.g., SAST/DAST scanners, deployment trackers) or struggle with licensing costs.

**Solution:**

- Leverage **Azure DevOps Marketplace** for plugins (e.g., *OWASP ZAP*, *SonarQube*).
- Use **open-source alternatives** where budgets are constrained.

##### Configuring ADO for Capturing Metadata

**Challenge:**

- ADO’s raw data (builds, releases, work items) requires proper tagging/linking to generate metrics like **Lead Time** or **Deployment Frequency**.

**Solution:**

- **Enforce mandatory fields** in work items and **periodic** updates
- Use **ADO Analytics Views** or **Power BI connectors** to query pipeline metadata.

##### Effective Updates on Work Item State Changes

**Challenge:**

- Manual state updates (e.g., Done → Approved) delay metrics like **Cycle Time**.

**Solution:**

- **Automate transitions** using ADO **webhooks** or **Azure Functions** (e.g., auto-update when PRs merge).
- Map states to **DevSecOps gates** (e.g., Security Review state before deployment).

##### Insufficient Data to Generate Metrics

**Challenge:**

- Sparse data (e.g., infrequent deployments, low count of incidents per project) skewing trends.

**Solution:**

- **Define minimum thresholds** (e.g., “Track only if >5 incidents of software bug category”).

##### Metrics That Are Inconclusive for Decisions

**Challenge:**

- Vanity metrics (e.g., “MTTR of 120 days”) don’t drive action.

**Solution:**

- Focus on **outcome-oriented metrics** backed by sufficient data

#### Organizational Resistance and Misuse

##### Organizational Resistance

**Challenges:**

- **Fear of Exposure:** Teams may resist metrics that highlight inefficiencies (e.g., high **Change Fail Rate** due to security rejections).
- **Tool Overload:** Introducing new DevSecOps tools, metadata to be captured & plugins (e.g., SAST scanners) can trigger pushback if perceived as disruptive/unnecessary.
- **Misaligned Incentives:** Leadership rewarding one metric over other. "Lead Time" alone may misguide

**Mitigation Strategies:**

- **Frame Metrics as Improvement Tools:**
    - Emphasize on how these metrics help Organization in the long run and to compare or measure performance
- **Pilot Metrics:**
    - Start with non-critical projects to demonstrate value before org-wide rollout and focus on metrics that offer Business value.

#### Organization long-term goals Vs Metric derived Results

**Challenge:**

- **Misalignment:** Short-term metric optimization (e.g., improving **Deployment Frequency**) may conflict with long-term goals (e.g., tech debt reduction, compliance maturity).
- **Vanity Metrics vs. Value Delivery:** Teams may prioritize metrics that look good over meaningful outcomes.

**Solution:**

**Cascade Metrics to Strategic Themes:**

- Map metrics (e.g., **Lead Time**) to business objectives (e.g., "Faster time-to-market for compliance-critical products").

**Challenge:**

**Metric Trap while defining roadmap**: Over-focusing on one metric (e.g., **MTTR**) may ignore systemic issues (e.g., inadequate test automation).

**Solution:**

**Weighted Metric Portfolios:**

- Balance the metrics with Organization’s long-term goals (e.g., **Deployment Frequency**), stability (**Change Fail Rate**), over Lead time

#### Balancing Metrics with Team Culture

##### The Cultural Risks of Over-Metrics

**Challenges:**

- **Fear of Measurement:** Teams may perceive metrics as surveillance, leading to stress and burnout.
- **Gaming the System:** Pressure to hit targets (e.g., **Deployment Frequency**) can incentivize shortcuts (e.g., skipping security scans).
- **Innovation Suffocation:** Excessive focus on metrics may discourage experimentation (e.g., fear of failing deployments affecting **Change Fail Rate**).

**Solution:**

- **Psychological Safety First:**
    - Emphasize that metrics are *diagnostic tools*, not performance evaluations.
    - Celebrate "learning from failures" (e.g., post-incident reviews that improve **MTTR**).
- **Team-Led Metric Design:**
    - Involve engineers in selecting metrics (e.g., let them choose between **Lead Time** or **Cycle Time** as their priority over **Change Failure Rate**).

#### Metrics Vs Team Capability & Training Needs

##### Capability Gaps Blocking Metrics

**Challenge:**

Teams lack skills to improve key metrics (e.g., slow **Lead Time** due to unfamiliarity with security tools)

**Solutions:**

- **Skill-Based Metric Segmentation:**
    - Example: Track **Lead Time** separately for teams adopting new SAST tools
- **Just-in-Time Training:**
    - Automate training triggers (e.g., if **Failed Pipeline Rate Due to Security** >15%, assign modular training).
- **Mentorship Metrics:**
    - Measure **% of Pair Programming Sessions** to boost knowledge sharing.

##### Metrics Ignoring Team Growth

**Challenge**:

- Overemphasis on output metrics (e.g., **Deployment Frequency**) undervalues capability building.

**Solutions:**

- **Balance Output and Growth Metrics:**
    - Pair **Deployment Frequency** with **% of Team Contributing to Pipeline Code**.
- **Career-Path Metrics:**
    - Example: Track **% of Engineers Leading Security Reviews** to encourage ownership.

## References

- Accelerate by Forsgren, Humble & Kim
- Azure DevOps Docs
- DevOps Research and Assessment (DORA) Reports