Measuring your DevSecOps journey

There are many opinions on DevOps, including what it means and how an organization should approach it. Is it just a buzzword, or does it describe a real cultural shift in an organization? The answers to these questions depend on the organization's experience, industry, and how successful the DevOps adoption is.

[ Explain DevOps in plain English ]

Common DevOps goals

The goals and objectives of any DevOps initiative are usually the same:

• Frequency:
  • Fast deployment (a straight-to-production philosophy)
  • Fast feedback from the user (fast feedback loops)
  • Fast time to market
• Stability:
  • Lower failure rate (learn quickly from failure and turn it into another opportunity)
  • High software uptime (the software-reliability engineers' error budget)

These goals align with the Four Keys identified by the DevOps Research & Assessment (DORA) team. The Four Keys are:

• Deployment frequency
• Lead time for changes
• Time to restore services
• Change failure rate

I use these keys as KPIs to measure where an organization is on their transformation journey.

DORA's 2019 Report provides examples about how DevOps elite performers compare against low performers. The elite performers usually have:

• Faster value delivery: They have a 106-times faster lead time (LT) from commit to deploy.
• Advanced stability and quality: They recover 2,604 times faster from incidents and have a seven-times lower change failure rate (CFR).
• Higher throughput: They deploy code 208 times more often.

Measuring the metrics

By picking the right metrics, you can make sure your DevOps transformation and acceleration are measurable and transparent with a modern security perspective that is tightly integrated. This is known as DevSecOps.

[ Related: Getting DevSecOps to production and beyond ]

DORA's four key metrics can also measure where you are on your DevSecOps journey. I split them into two main categories: delivery performance and stability.

Delivery performance metrics

Delivery performance metrics measure software delivery. There are two main metrics to determine delivery performance:

• Delivery lead time (DLT): This is the time from when developers start working on a request until it is available to the end users; that is, when code is committed to production.
• Deployment frequency (DF): This is how long it takes to deliver code changes, or how often customers deploy changes to production.

Stability metrics

Software stability metrics assess how resilient the software is during changes and runtime. I use the following metrics to collect this data:

• Mean time to restore (MTTR): This measures how long it takes to restore your product or service if you have an outage. The simplest way I've found to determine MTTR is by looking at software uptime by querying its health endpoints.
• Change fail rate (CFR): This measures how many deployments cause a failure in production. This shows the overall health of the software pipeline process and enhances it to capture errors earlier in the process.

[ Download the Enterprise automation in a DevOps world checklist. ]

Using a dashboard

These metrics are good starting points to illustrate the DevSecOps KPI journey. To make it easier to visualize, you can use a tool like Grafana to create a dashboard to show those metrics.

The following dashboard from the Four Keys GitHub repository displays the four metrics with daily systems data and a snapshot of the last 90 days.

Image

Conclusion

These metrics reflect the current state and health of a DevOps journey based on the DevSecOps maturity model. Your goal should be to keep evolving, enhancing, and improving end-users' experience with the delivered product or service. These metrics can help support these efforts.

[ Check out Red Hat's Portfolio Architecture Center for a wide variety of reference architectures you can use. ]