Though it’s crucial to communicate IT efficiency in terms that stakeholders can understand, IT performance metrics can be a pain to identify and consolidate. A business value dashboard can help — but not without a comprehensive metrics management strategy.
Our growing capacity for data collection has empowered business leaders to analyze the overwhelming multitude of factors that influence business outcomes. But while this rapid acceleration in technological advancement has made businesses more powerful, it also poses an unexpected problem: IT departments often have difficulty communicating the value of the service they provide.
Because IT departments tend to work on a varied range of tasks and disciplines, all of which require their own unique units of measurement, IT professionals sometimes struggle to consolidate performance information in terms that they can understand and act upon. Technological investments are based on a quantifiable notion of how well they can support revenue generation, and without tools that can illustrate that support to executives, justifying IT spend will always be an uphill battle.
Business value dashboards (BVDs) seek to address this problem by putting operational data in a business context, helping business leaders better understand and meet IT spending needs. An effective BVD will organize and display relevant metrics to business leaders and stakeholders in order to show exactly how investment in IT actively drives of revenue, rather than simply protecting against risk.
But before a BVD can accomplish any of this, the identification and consolidation of the relevant metrics is an extremely important and labor-intensive task that necessitates a holistic metric management strategy. The factors that define success to you must be defined and laid out at the most granular level possible, then weighted appropriately to create a complete picture of your organization and its needs. The four basic categories of metrics that must be created or selected are those that quantify factors relevant to people, process, technology, and resources.
People are perhaps the most complex variable of any business, but for obvious reasons it’s crucial to analyze the performance of employees. Because engaged employees are more productive and efficient, smart IT leaders will take steps to ensure that their direct reports are working in the positions that are best-suited to their individual capabilities.
People metrics are gathered from HR, Learning Management Systems, and IT-external management tools. Some standard people metrics that hardly merit explanation are skills, training, and certification — it’s obviously crucial to make sure that all employees are qualified for the positions they fill on paper. However, it’s also important for every organization to discern the more subtle factors that lead their top performers to success, and to identify metrics that relate to the company mission.
A solid indicator of your organization’s appeal to top talent is the time-to-fill metric. To calculate it, simply divide the total number of days for which jobs were open during a set period of time, then divide that day total by the total number of jobs which were open in that time period.
It’s vital to combine time-to-fill with another recruitment metric, quality-of-hire. To measure quality-of-hire, you’ll need a method for assessing the talent of each candidate, as well as a method for converting this data into some clear indication of financial impact. Both of these will depend on the goals of your organization, but regardless, talent evaluation should take place both pre-hire and 3-6 months after hiring (and it’s vital to make sure that both evaluations are internally consistent to ensure accurate ROI projection). Assign grades on the basis of performance, then calculate the grade average to determine quality of hire. You can track the effectiveness of new recruitment initiatives by tracking your quality of hire at different points in time (i.e. see if the grade average changes after a recruitment initiative).
Simple survey questions can be used to measure employee engagement, or how well each employee fits in his respective position and how confident she is in her ability to contribute to the company’s mission. Unengaged employees tend to undermine productivity in their departments and are generally more likely to miss work, while a Gallup survey showed that companies with highly engaged employees reported as much as 147% more in earnings per share.
Because process metrics tend to consist of technical data from specific groups within larger IT organizations, they are somewhat easier to identify than people metrics, but no less crucial to consider. Analysis of process metrics should be neither strictly technical nor group-specific; a well-defined metrics strategy spans across processes to translate operations-level metrics into figures that stakeholders can comprehend and act on. For instance, how has the deployment of a web application resulted (or not resulted) in a targeted revenue increase? If you’ve invested in ITIL or Six Sigma, how much are you getting back from the improved processes? How can you gauge how well they are continually improving?
But before consolidating process metrics into a “big picture” for stakeholders, you’ll need to hammer down some group-specific component metrics. Defect rate, or the rate at which a given process underperforms specifications, is always a good place to start. This metric can also be broken down into historical trend analysis to determine the effectiveness of testing, or categorized by defect severity for necessary context. Other areas to be considered are the speed with which IT delivers applications, the speed with which IT resolves problems, maximum server uptime, SLA, and OLA.
Technology metrics are similar to process metrics in that they cover a wide range of activities and sectors within IT. Nonetheless, you must establish an end-to-end view of how various technologies accomplish overarching business goals, highlighting problem areas along the way. Many of the technologies IT will be interested in measuring will be shared by multiple teams, so the goal of these metrics should be not only to quantify the benefits that new technology is delivering, but how those benefits are being allocated across departments.
Individual technology metrics should focus on efficiency, agility, and service quality as rendered by specific IT tools. Once a sufficient amount of data is available, however, business narratives can be built out of technology metrics.
For instance, if an application server has an outage, there are a few perspectives to consider. From the perspective of application monitors, the metric of interest would be the availability of applications on the server. From an infrastructure monitoring perspective, the metric of interest would be server downtime. These can be combined into a business value perspective, which shows the impact of outages on revenue and ability to deliver mission-critical services.
In this way, metrics can not only demonstrate how it prevents unnecessary expenditures, but exactly how much value a consistent ability to prevent those expenditures can add. If systems diversity is able to prevent an outage, then that success can be explained to stakeholders in terms of revenue loss avoidance, which will help to justify future IT spend on diversity and justify future spending to stakeholders.
Business stakeholders and IT managers should always understand how efficiently IT resources are being allocated. Different teams within IT often use different tools to measure efficiency, so the consolidation of resource metrics can be a labor-intensive task. Nevertheless, a comprehensive metrics strategy should specify how metrics are gathered, which metrics are chosen, how they’re consolidated, and how they’re published. For this reason, a catalog-based approach is necessary — this enables IT managers to define their ideal set of metrics for each process, function, and role, which will be consistent and interrelated even as underlying systems come and go.
Business Value Dashboards Consolidate and Translate IT Metrics
The four categories of metrics covered above fall under the umbrella of “below-the-line” metrics, which tend to be technical and designed for analysis by IT management and staff. “Above-the-line” metrics, by contrast, are not technical, and have direct business relevance (revenue, cost). A BVD will be composed almost entirely of above-the-line metrics, and when below-the-line metrics do appear, they’ll do so in relation to an overarching business metric — for instance, a BVD could compare the total number of website orders against order processing capacity given a certain period of time.
In combining these metrics, IT leadership can and should strive for top-to-bottom organizational transparency; business leadership rarely gets to see how systems interact with each other, so BVDs offer a rare opportunity to showcase process efficiency to upper management. Intelligent consolidation and presentation of IT metrics will anticipate questions from executives (“are these two systems operating together as efficiently as possible?”) and answer them before they’re asked.
Ultimately, BVDs are effective because they organize and present key IT performance metrics in narratives comprehensible to stakeholders and executives. In order to get the full ROI for your BVD, you’ll need an in-depth understanding of what business leaders actually need to see and act upon.