Measuring What Matters

  |  February 1, 2011   |  Comments

Consider this framework for measuring user experience.

We're not short of numbers are we? In our digital world, we generally have access to a vast amount of data from all sorts of different digital tracking and analysis systems. And that's just the site-centric data. Increasingly, we also have user-generated data such as that from voice of the customer surveys and remote user testing programs. So the problem's not how to get ahold of data anymore, it's knowing how to manage it and to structure it in a way that allows it to tell us a story.

This is why we need to develop measurement frameworks. Measurement frameworks are a way of structuring metrics and those all-important key performance indicators around the objectives of the business. The key thing about a measurement framework is that it's coherent and helps the business to understand not only the metrics themselves, but also the relationship between metrics.

I came across a good example recently in a paper from Google that outlines a framework it uses for measuring the user experience which is called HEART. HEART stands for happiness, engagement, adoption, retention, and task success. They use this framework to understand the user experience across their various different products. The important point is that the metrics that make up the framework come from a variety of different data sources.

Happiness, for example, is measured using attitudinal metrics gathered using survey data, adoption and retention may be measured using customer data, and task success by using remote user testing methodologies. The actual metrics used to measure the user experience vary from product to product depending on the nature of the product.

For example, the way that Gmail is measured is different to the way that Google Maps is measured, but the HEART framework brings the rigour and coherence to the individual metrics and usually describes what is being measured and to some extent why. It also always helps to have a good acronym!

To develop a framework, you need to go back to the objectives and determine you need to measure and why. In an e-commerce scenario, for example, a measurement framework may be based around the transactional environment looking at sales, order, average order value, number of customers, number of orders per customer, and so on. The framework would highlight the relationship between the different variables and how a change in one might or might not influence the other.

As an example, we were recently asked by a client to help them think about how to measure the success of various initiatives. In line with many companies, they are looking to serve more of their customer service needs using the online channel. One measurement perspective would be to focus on some of the core site-centric metrics such as visits to the customer service section, number of downloads, etc.

These metrics are certainly useful in terms of understanding what's actually happening on the site but are not necessarily indicative of whether the initiative overall is successful or not. To do that we needed to develop a wider framework that looked at the customer service section in the context of the overall customer journey.

The framework included measurements that looked at to what extent customers knew about some of the online capabilities, whether they would use them or not, and what they thought of the experience. The framework essentially tells the story about the migration of customer service from offline channels to online channels and allows the organization to understand where the focus areas need to be. Does the site need to improve or does the promotion of the offline capabilities need to increase? Such a framework can help the organization to understand the most effective way to hit the initiative's goals and objectives.

With data everywhere, it's easy not to be able to see the forest for the trees. Developing multi-sourced measurement frameworks is the way to add structure to your data and to focus on the metrics that matter.

ClickZ Live Toronto Twitter Canada MD Kirstine Stewart to Keynote Toronto
ClickZ Live Toronto (May 14-16) is a new event addressing the rapidly changing landscape that digital marketers face. The agenda focuses on customer engagement and attaining maximum ROI through online marketing efforts across paid, owned & earned media. Register now and save!


Neil Mason

Neil Mason is SVP, Customer Engagement at iJento. He is responsible for providing iJento clients with the most valuable customer insights and business benefits from iJento's digital and multichannel customer intelligence solutions.

Neil has been at the forefront of marketing analytics for over 25 years. Prior to joining iJento, Neil was Consultancy Director at Foviance, the UK's leading user experience and analytics consultancy, heading up the user experience design, research, and digital analytics practices. For the last 12 years Neil has worked predominantly in digital channels both as a marketer and as a consultant, combining a strong blend of commercial and technical understanding in the application of consumer insight to help major brands improve digital marketing performance. During this time he also served as a Director of the Web Analytics Association (DAA) for two years and currently serves as a Director Emeritus of the DAA. Neil is also a frequent speaker at conferences and events.

Neil's expertise ranges from advanced analytical techniques such as segmentation, predictive analytics, and modelling through to quantitative and qualitative customer research. Neil has a BA in Engineering from Cambridge University and an MBA and a postgraduate diploma in business and economic forecasting.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!



Featured White Papers

ion Interactive Marketing Apps for Landing Pages White Paper

Marketing Apps for Landing Pages White Paper
Marketing apps can elevate a formulaic landing page into a highly interactive user experience. Learn how to turn your static content into exciting marketing apps.

eMarketer: Redefining Mobile-Only Users: Millions Selectively Avoid the Desktop

Redefining 'Mobile-Only' Users: Millions Selectively Avoid the Desktop
A new breed of selective mobile-only consumers has emerged. What are the demos of these users and how and where can marketers reach them?


    • Contact Center Professional
      Contact Center Professional (TCC: The Contact Center) - Hunt ValleyLooking to join a workforce that prides themselves on being routine and keeping...
    • Recruitment and Team Building Ambassador
      Recruitment and Team Building Ambassador (Agora Inc.) - BaltimoreAgora,, continues to expand! In order to meet the needs of our...
    • Design and Publishing Specialist
      Design and Publishing Specialist (Bonner and Partners) - BaltimoreIf you’re a hungry self-starter, creative, organized and have an extreme...