Do You Have the Right Framework for Measuring Success?

A conversation last week reminded me that while analytical technologies are developing at a pace, some of the problems they are looking to solve remain stubbornly the same. The person I was chatting to was lamenting that their organization was focused on measuring and reporting transactions, but beyond that had little understanding of what levers to pull to drive the business forward beyond the “classical” approach of conversion rate optimization.

We discussed the potential of using an additional (or even alternative) framework based around “customer optimization” or maximizing customer lifetime value. So rather than focusing on the conversion of website traffic to sales, focusing instead on the challenge of getting more people to buy more stuff. It’s amazing how many customer databases are littered with vast numbers of people who’ve only ever ordered once. What would performance look like if you only got a fraction of them to order twice? I’ve long thought that the art of marketing is to get people to do something twice; visit twice, click again, order one more time, etc.

From the conversation, though, it became clear that this customer-centric approach wasn’t in the DNA of the business and they were rooted in the site-centric approach to success. They thought about orders and average order value rather than customers and customer lifetime value. It would take a lot of work and a lot of persuasion to shift their perceptions how to measure success. She had her work cut out.

There’s the old saying, “Be careful what you measure, because what you measure is what you’ll get.” I think the development of the right key performance indicators (KPIs) for an organization is often a critical component of success in its own right, as it sets the tone and the agenda for the business. I’ve seen and worked in organizations that have got it right and those that have got it fantastically wrong. So what does good look like?

There are two components I think that are important:

  • KPIs have to make sense
  • KPIs have to make sense when looked at together

Meaningful KPIs

KPI development is no easy task, particularly in complex, multi-channel businesses. Even in pure-play online transactional environments it’s not necessarily easy, as my story earlier highlights. KPI development starts with good objectives but also the metrics themselves require careful consideration. For me the characteristics of good KPIs are that they should be:

  • Strategic measures of success (that’s the point really)
  • Actionable (is it possible to move the needle?)
  • Easy to understand (no complex ratios and indexes please)
  • Measurable (either directly or via proxy)

A Meaningful Framework

KPIs not only need to make sense individually but they also need to make sense collectively. This is what is meant by a measurement framework. The measurement framework organizes the KPIs into a coherent story about how the business expect to be successful in meeting its objectives. KPIs should be as complementary to each other as they possibly can. That’s not to say that there won’t be tensions from time to time. This approach to KPI development is looking at the business as a whole and involves taking almost a systems modelling approach to creating the right metrics to understand whether objectives are being met or not. Let’s look at an example.

A multi-channel business sells and services its technical products through stores, the call-center, and online. One of its key objectives is reduce the volume of calls to the call-center, particularly on the service side of the business. The strategy is to invest in online capabilities for customers to self-serve their accounts and troubleshoot any technical problems.

The overarching KPI for this objective is “calls to call center.” It meets most of the criteria of a good KPI outlined above. People get it, they can measure it, but it’s questionable how actionable it is. This metric is in reality the outcome of a lot of other factors, which need to be considered as part of the measurement framework for this objective. For example:

  • Are people aware that they can service their accounts online or find answers to their technical problems? If they don’t know that, then they’re still going to call.
  • Will people actually consider using online capabilities such as forums, FAQs, and chat to resolve customer service issues? Are their particular customer segments that are more likely than others? If people prefer to call, they’re going to need some compelling reason to do otherwise.
  • Are people actually using the online capabilities? Is awareness and interest actually translating into increased activity levels?
  • Do the online capabilities do the job? Are people able to complete tasks and resolve problems? If not, then they’re going to call…

Each of these components of the system will have its own set of KPIs that will be measured using different methodologies and instruments. However, together they add up to a coherent story about how the business is meeting its objective and importantly what areas it needs to focus on or which levers to pull, to get back on track if needed.

I’ve found it also helps to have a handy acronym to describe your framework. We called the one above “ACUP,” standing for Awareness, Consideration, Usage, and Performance. Sort of stuck in people’s minds.

Related reading

Flat business devices communication with cloud services isolated on the light blue background.