The toughest part of ROI measurement is the measurement piece! In the last couple of columns I’ve been looking at some of the approaches to understanding how to evaluate the return on investment (ROI) in analytics, but how easy it is to actually to measure it?
I don’t think it is easy. As discussed in the first part of this series, measuring the investment piece is probably more straightforward than identifying and measuring the returns. Generally the accountants will tend to do that for you, as it’s mainly about measuring the costs of technology, services, and people. In fact, all too often they will be very transparent! That’s inherently often the problem in that the costs of analytics are transparent but the returns are relatively opaque. So is it possible to get more transparency into understanding the returns on analytics?
Last time I suggested that it’s useful to chunk the problem down and to start to think about the returns generated by analytics into the “tactical” returns and the “strategic” returns, tactical returns tending to be more immediate and the strategic returns tending to be longer term and perhaps less tangible.
With tactical investments designed to generate tactical returns, measurement can be easier and should be built into the business case for the investments in the first place. For example, investments in technology areas such as conversion rate optimization (CRO) are designed to produce specific results such as improvements in the conversion process and this is something that most businesses have been measuring for years. This has evolved from simple A/B testing through to more complex multi-variate testing and personalization approaches but the framework is broadly the same: The optimization process generates certain results, those results can be measured and can be compared against the total investment in that program of work.
As with all measurement processes, there is the need to be aware of the maxim “be careful what you measure because what you measure is what you will get.” A focus on the measurement and the optimization of one metric can be at the expense of another one. So for example by investing in CRO technologies that drive tangible returns, are there other areas of the business that are being impacted, such as the overall customer experience perhaps?
A balance is needed between the immediate and measurable impact of analytics and the longer-term and more intangible aspects. The latter is certainly a more complex management and measurement problem and requires structured thinking. The goal is to understand the value that these investments in technologies, services, and people deliver to the business. This means trying to answer touch questions like: What is the business value of:
- Adding one more person to my analytics team?
- Having better data integration capabilities?
- Having better data visualization technologies?
- Engaging with this particular consulting business?
These types of strategic questions need more strategic measurement frameworks and also need to be addressed in the context of the analytics maturity of the organization and the trajectory that it has traveled in. Analytics functions are there to serve the organization and it’s only by being explicit as to how they serve that it can be possible to measure their impact. In a way this is no different from what analytics functions do on a day-to-day basis; they understand what the key performance indicators (KPIs) are and they put in place the analytics capabilities to provide the tracking, diagnostics, insight, and intelligence required to help the organization achieve those objectives. The question is how well they are doing that.
“Measuring the measurers” requires a systematic approach to defining, understanding, and evaluating the value delivered by analytics. The value delivery will be different for different organizations and will need to be contextualized and defined. For example, does the analytics function have the ability to directly influence the KPIs of the organization through its activities? If not, do internal customers and stakeholder groups believe that the analytics function adds value to the business?
The approach to addressing the first issue would be for the analytics function to evaluate the expected dollar value impact of each of its recommendations or actions and record the results as well. These results may take months to materialize and that is why a system or framework is required. Understanding the internal impact of an analytics function would require some type of survey approach that could be benchmarked over time.
As I said at the beginning, I don’t pretend this is easy and I don’t think there is a “one size fits all approach” to measuring the ROI of analytics. In some cases it will be possible to measure the outputs, but more often than not it’s going to require some deep thinking about how to evaluate the outcomes. But that’s what analysts do, right?
“You cannot succeed in analytics and marketing unless they are central to business operations and are helping business answer the questions that will drive dollars to the top or bottom line,” says Kerem Tomak, Sears Chief Digital Marketing & Analytics Officer.
The use of psychology in marketing and sales is not new, but it may be more useful than ever in an attention economy where time is precious and focus is rare. How can you tap into a demanding consumer to check whether there is an actual interest in your product?
According to a survey conducted as part of OnBrand Magazine's State of Branding Report 2017, marketers are well aware of the new technologies that are expected to be important to their brands in coming years, but the majority aren't rushing to invest in them before they're fully-baked.
Two weeks ago, Foursquare announced what could be the most important component of its data business: the Pilgrim SDK. So what does it do, and what does it mean for location-based marketing?