Part three in a series looking at how to measure the return on investment (ROI) of analytics. In some cases it will be possible to measure the outputs, but more often than not it's going to require some deep thinking about how to evaluate the outcomes.
The toughest part of ROI measurement is the measurement piece! In the last couple of columns I’ve been looking at some of the approaches to understanding how to evaluate the return on investment (ROI) in analytics, but how easy it is to actually to measure it?
I don’t think it is easy. As discussed in the first part of this series, measuring the investment piece is probably more straightforward than identifying and measuring the returns. Generally the accountants will tend to do that for you, as it’s mainly about measuring the costs of technology, services, and people. In fact, all too often they will be very transparent! That’s inherently often the problem in that the costs of analytics are transparent but the returns are relatively opaque. So is it possible to get more transparency into understanding the returns on analytics?
Last time I suggested that it’s useful to chunk the problem down and to start to think about the returns generated by analytics into the "tactical" returns and the "strategic" returns, tactical returns tending to be more immediate and the strategic returns tending to be longer term and perhaps less tangible.
With tactical investments designed to generate tactical returns, measurement can be easier and should be built into the business case for the investments in the first place. For example, investments in technology areas such as conversion rate optimization (CRO) are designed to produce specific results such as improvements in the conversion process and this is something that most businesses have been measuring for years. This has evolved from simple A/B testing through to more complex multi-variate testing and personalization approaches but the framework is broadly the same: The optimization process generates certain results, those results can be measured and can be compared against the total investment in that program of work.
As with all measurement processes, there is the need to be aware of the maxim "be careful what you measure because what you measure is what you will get." A focus on the measurement and the optimization of one metric can be at the expense of another one. So for example by investing in CRO technologies that drive tangible returns, are there other areas of the business that are being impacted, such as the overall customer experience perhaps?
A balance is needed between the immediate and measurable impact of analytics and the longer-term and more intangible aspects. The latter is certainly a more complex management and measurement problem and requires structured thinking. The goal is to understand the value that these investments in technologies, services, and people deliver to the business. This means trying to answer touch questions like: What is the business value of:
These types of strategic questions need more strategic measurement frameworks and also need to be addressed in the context of the analytics maturity of the organization and the trajectory that it has traveled in. Analytics functions are there to serve the organization and it’s only by being explicit as to how they serve that it can be possible to measure their impact. In a way this is no different from what analytics functions do on a day-to-day basis; they understand what the key performance indicators (KPIs) are and they put in place the analytics capabilities to provide the tracking, diagnostics, insight, and intelligence required to help the organization achieve those objectives. The question is how well they are doing that.
"Measuring the measurers" requires a systematic approach to defining, understanding, and evaluating the value delivered by analytics. The value delivery will be different for different organizations and will need to be contextualized and defined. For example, does the analytics function have the ability to directly influence the KPIs of the organization through its activities? If not, do internal customers and stakeholder groups believe that the analytics function adds value to the business?
The approach to addressing the first issue would be for the analytics function to evaluate the expected dollar value impact of each of its recommendations or actions and record the results as well. These results may take months to materialize and that is why a system or framework is required. Understanding the internal impact of an analytics function would require some type of survey approach that could be benchmarked over time.
As I said at the beginning, I don’t pretend this is easy and I don’t think there is a "one size fits all approach" to measuring the ROI of analytics. In some cases it will be possible to measure the outputs, but more often than not it’s going to require some deep thinking about how to evaluate the outcomes. But that’s what analysts do, right?
Neil Mason is SVP, Customer Engagement at iJento. He is responsible for providing iJento clients with the most valuable customer insights and business benefits from iJento's digital and multichannel customer intelligence solutions.
Neil has been at the forefront of marketing analytics for over 25 years. Prior to joining iJento, Neil was Consultancy Director at Foviance, the UK's leading user experience and analytics consultancy, heading up the user experience design, research, and digital analytics practices. For the last 12 years Neil has worked predominantly in digital channels both as a marketer and as a consultant, combining a strong blend of commercial and technical understanding in the application of consumer insight to help major brands improve digital marketing performance. During this time he also served as a Director of the Web Analytics Association (DAA) for two years and currently serves as a Director Emeritus of the DAA. Neil is also a frequent speaker at conferences and events.
Neil's expertise ranges from advanced analytical techniques such as segmentation, predictive analytics, and modelling through to quantitative and qualitative customer research. Neil has a BA in Engineering from Cambridge University and an MBA and a postgraduate diploma in business and economic forecasting.
US Consumer Device Preference Report
Traditionally desktops have shown to convert better than mobile devices however, 2015 might be a tipping point for mobile conversions! Download this report to find why mobile users are more important then ever.
E-Commerce Customer Lifecycle
Have you ever wondered what factors influence online spending or why shoppers abandon their cart? This data-rich infogram offers actionable insight into creating a more seamless online shopping experience across the multiple devices consumers are using.
September 9, 2015
12pm ET/9am PT
September 16, 2015
12pm ET/9am PT
September 23, 2015
12pm ET/ 9am PT