As a leader, you must be a change agent.
You set directions, and you even define a strategy, maybe. But how do you know if you’re getting there?
How are your results? Are you moving fast enough? What have you indeed achieved in the last three months?
“If you can’t measure it, you can’t improve it.”
Peter Drucke
These are simple questions each leader should feel the need to answer. But it’s undoubtedly not straightforward. Many of us struggle, at least in some areas.
How many times, for instance, facing your boss, or your board, you need extra-work building and explaining measures to value your results, sustain a decision or an initiative?
How much of your team’s work is driven by facts and evidences?
Measures matter, but let alone mean nothing
Measures are raw data.
Good examples of IT measures are:
- total hours of an application “order_management” availability during last week
- number of incidents during last month
- number of calls received by the call center yesterday
- number of errors in API “get_cstomer_data” calls from 1pm to 2pm
- number of laptops your organization delivered to users in 2021
- our storage costs 1 Million dollars/year
As leaders, we need to derive a story from our measures to get insights and valuable information to set, maintain, or correct a path.
Metrics (or indicators) combine measures and become stories.
Good examples of stories we get by using IT metrics are:
- the availability to customers of the application “order_managent” was 98% last week (thus combining the availability measure above with the amount of total service time in a week for that application), actually lower than this year’s average by 12%, thanks to our continuous improvement solid practices
- critical incidents last month were 20%, with an increasing trend (+5%) from the previous month, due to the arising of repeated networking issues
- yesterday we had a peak of calls received by our call center: we got 50% more than the average daily call rate and 10% more than this year rises, mainly due to the new commercial proposal gone to the ads
- the API “get_customer_data” calls ended up with errors were 23% from 1 pm to 2 pm, versus an average of 5% after our engineering team implemented a workaround
- 100% of our laptops in 2021 were delivered within SLA
- 1 TB of storage now costs 1000$ a year, which is 30% less than last year thanks to the re-engineering project we completed last month.
Performance and Risk Metrics
Storytelling, for leaders, is usually about performance and risks.
Performance Indicators tells what you achieved, your velocity, your trends and much more.
Metrics providing warnings of exposure to risks, become Risk Indicators.
Looking at the above examples, we can easily argue that our laptop delivery indicator is a KPI. At the same time, the API error rate might be considered a KRI too, as well as the application availability one. A critical vulnerability indicator is a KRI.
Both matter: if you set a strategy or simply targets, KPIs should help to measure how (in terms of velocity and quality) your organization is getting there, while KRIs should help you to get in warnings that the risks you identified could get you off track, get a lower quality or need too much time.
Lagging and Leading Indicators
Both performance and risk indicators can be used to measure how we perform or risk (called lagging) and give us insight into future success predictions (called leading).
Back at the above examples, the cost per TB of storage is a lagging indicator, and so is the call center call rate, while the critical incident rate might be considered a leading one. This distinction can be subtle, as you can quickly see.
But the “core message” here is that you need to understand how you’ve gone and what might happen due to the data you’re collecting to adjust, change where necessary.
How to define the proper measures and indicators
As IT leaders, as we said, we must look for stories, which can tell whether or not we’re hitting the targets we or someone else (generally above our customers or us) defined.
As simple as that. Or as really complex as that.
Let’s take an example
Suppose your target for your core application availability this year is an increase of 10%.
The above-illustrated KPI on application availability is indeed good then. But is it sufficient? Probably not.
To improve service availability, you might need:
- to invest in the skills of your team, as they lack troubleshooting experience, which is badly impacting our MTTR (mean time to repair)
- to face are-engineering of your observability platform, as your don’t gather enough alerting and your MTTD (mean time to detect) is poor
- the service time we agreed with your business partners is H24x7, but your measures clearly indicate that from 7 am to 9 am and after 9 pm the application’s usage traffic drops by 95%, so you should re-negotiate it
- SW deployment on 75% of your core applications requires to put courtesy pages, thus making service unavailable to your customers; you should then measure how much planned unavailability you have and improve your deployment automation capabilities.
Now, the following question arises: should all the above secondary indicators be improved with equal priority?
Again, the answer is: not always.
A good prioritization driver is given by how much each of these indicators impacts your primary one.
For instance, you might be in a condition where even improving your MTTD, no measurable increase of applications availability would be obtained, as troubleshooting remains too weak, and that’s where you’re spending most of your time. Also, re-negotiating the service window might have a more significant impact too.
So, should you forget your MTTD? Of course not, but now you have prioritization drivers you can use to set or adjust your plan.
The above is quite a simple example, but it effectively explains how challenging reaching a target can be due to the several dimensions that should be considered and measured.
But you can dig into this complexity and find out there are ways to walk it through.
Summarizing: using indicators to take the (right) decisions
To sum up, I find this approach effective:
- define your goals. See my previous article for help on both defining and staying on top of goals
- look at the primary indicators, which can best summarize performance and risks on the achievement of your targets
- understand which secondary indicators might influence primarily your primary ones
- act on them, defining which actions would have the most impact on your primary ones; it should be better to be specific (for instance, improve MTTD by 12% in the next three months), but honestly, this can be challenging
- monitor your indicators’ trend regularly to introduce adjustments if needed.
Measurement culture
Even with the sharpest methodology to define your targets, make a sound plan and measure progress, your results might be impaired without a strong measurement culture in place with your teams.
A measurement culture leverages facts over the emotions and biases as decision tools.
We as leaders have to work hard to implement and cultivate this culture: rewards can be tremendous.
Conclusions
Being pragmatic, having clear goals and metrics to measure achievements and prevent risks, ruthlessly prioritize key actions, all within a solid measurement culture, are fundamental ingredients for a performance success recipe.