4 tips to get maximum value from engineering metrics

Engineering performance can be difficult to evaluate—even for engineers. Non-technical teams have key performance indicators (KPIs) like revenue, customer retention, and customer acquisition cost (CAC) that make it straightforward to measure their progress. The relationship between progress, goals, and objective data isn't always as clear in engineering.

Using metrics like DORA's 4 Keys—which measure deployment frequency, mean lead time for changes, mean time to recovery, and change failure rate—is a great starting point for understanding the health and performance of the software organization. Pairing these with other engineering metrics can help engineering leaders increase efficiency and improve visibility, accountability, and resource management.

[ Learn how IT modernization can help alleviate technical debt. ]

Below are four tips to get the most value from engineering metrics, whether you're using them for the first time or just want to fine-tune your approach.

1. Be transparent about how data is used

When engineering organizations use data to assess team health, understand allocations, and measure progress, the "how" is often the biggest source of fear. Developers may worry that their data will be used against them and that they will be held to certain numbers and penalized if they're not met. When everyone understands how data will be used and, when applicable, how to use it themselves, it can help alleviate many of these concerns.

To help address these worries, communicate exactly what data you're looking at, why it's useful, and how you're using it. Are you aggregating data at the team level or digging into individual metrics? Are you looking at big-picture metrics like pull request cycle time or granularly assessing every step of the software development lifecycle (SDLC)? Make sure to communicate your reasons for looking at data, such as improving goal-setting, grounding conversations in fact, and removing blockers.

2. Be thoughtful about metrics

There are various ways to ensure your metrics provide the information you need. Consider the following approaches.

a. Standardize your measurements 

A standard process for measuring metrics is essential for consistent insights. Your existing engineering tools, like Jira, contain a wealth of information. To get the most accurate metrics, use a tool that aggregates and processes information like incident and deployment data to make it easier to standardize your measurements rather than calculating metrics manually.

b. Decide which metrics matter

Measurements are productive only if you're looking at the right data. Choose metrics that are aligned with your team's goals, and consider how different data points might balance each other out to tell a complete story.

The first step is to establish team goals that align with business objectives. These goals may be developing new features, optimizing your underlying infrastructure, or improving efficiency. Ensure the goals are specific, measurable, and time-bound. From there, determine what metrics provide objective insight into how those goals progress. If your goal is improving efficiency, for example, the metric might be to reduce the average time to merge (the time between when a pull request is opened and when it's merged) by 10% quarter over quarter.

c. Consider complementary metrics

Look at complementary metrics to ensure your team is not over-optimizing to meet a goal at the expense of something else. If you're working on improving efficiency, for example, you may still want to keep an eye on quality metrics to ensure you're not sacrificing your standards for the sake of speed.

d. Involve your team

Include the people responsible for these goals in the conversation so they can contribute ideas and help set realistic milestones. Once you've finalized your goals and how to measure them, share them with stakeholders and adjacent teams who may influence their success. This creates a sense of shared ownership, support, and trust across teams.

3. Focus on continuous improvement

Engineering performance can be difficult to measure because "success" is multifaceted. When you create goals, standardize your process for measuring progress against them, and gather objective insights, it's easier to understand if your team has been successful. And, if you haven't been, a culture of continuous improvement creates space to learn why.

When metrics aren't trending in the right direction, treat it as an opportunity to learn more and help your team improve—not as justification for punitive action. Metrics can also be a great way to spot and recognize positive contributions to the team. Data makes it easier to learn from success and failure and work together to improve the process going forward.

[ For more insight on remediating troubled projects, see 20 tips to get an architecture project back on track. ] 

4. View data in context

Data cannot tell you everything you need to know about your team and how it's working. Placing all quantitative data in context by gathering additional qualitative information from your team is important. If you notice something that is not in line with your expectations, or if something is trending in a concerning direction, bring that information to a standup, 1-on-1, or team meeting to learn more. Use that data to start a conversation with your team and work with them to understand exactly what's going on—and address it.

Use metrics to understand engineering health

Leveraging engineering metrics gives leaders deeper insight into the performance of technical teams and individuals. Whether you start incorporating engineering metrics for the first time or want to extract more value from an existing process, it is crucial to prioritize transparency, continuous improvement, and contextualization. Remember that the process is iterative, so it's vital not to become fixated on identifying the perfect metrics to the extent that no measurements are taken.

[ Check out Red Hat Portfolio Architecture Center for a wide variety of reference architectures you can use. ]