Measuring What Matters

  |  February 1, 2011   |  Comments

Consider this framework for measuring user experience.

We're not short of numbers are we? In our digital world, we generally have access to a vast amount of data from all sorts of different digital tracking and analysis systems. And that's just the site-centric data. Increasingly, we also have user-generated data such as that from voice of the customer surveys and remote user testing programs. So the problem's not how to get ahold of data anymore, it's knowing how to manage it and to structure it in a way that allows it to tell us a story.

This is why we need to develop measurement frameworks. Measurement frameworks are a way of structuring metrics and those all-important key performance indicators around the objectives of the business. The key thing about a measurement framework is that it's coherent and helps the business to understand not only the metrics themselves, but also the relationship between metrics.

I came across a good example recently in a paper from Google that outlines a framework it uses for measuring the user experience which is called HEART. HEART stands for happiness, engagement, adoption, retention, and task success. They use this framework to understand the user experience across their various different products. The important point is that the metrics that make up the framework come from a variety of different data sources.

Happiness, for example, is measured using attitudinal metrics gathered using survey data, adoption and retention may be measured using customer data, and task success by using remote user testing methodologies. The actual metrics used to measure the user experience vary from product to product depending on the nature of the product.

For example, the way that Gmail is measured is different to the way that Google Maps is measured, but the HEART framework brings the rigour and coherence to the individual metrics and usually describes what is being measured and to some extent why. It also always helps to have a good acronym!

To develop a framework, you need to go back to the objectives and determine you need to measure and why. In an e-commerce scenario, for example, a measurement framework may be based around the transactional environment looking at sales, order, average order value, number of customers, number of orders per customer, and so on. The framework would highlight the relationship between the different variables and how a change in one might or might not influence the other.

As an example, we were recently asked by a client to help them think about how to measure the success of various initiatives. In line with many companies, they are looking to serve more of their customer service needs using the online channel. One measurement perspective would be to focus on some of the core site-centric metrics such as visits to the customer service section, number of downloads, etc.

These metrics are certainly useful in terms of understanding what's actually happening on the site but are not necessarily indicative of whether the initiative overall is successful or not. To do that we needed to develop a wider framework that looked at the customer service section in the context of the overall customer journey.

The framework included measurements that looked at to what extent customers knew about some of the online capabilities, whether they would use them or not, and what they thought of the experience. The framework essentially tells the story about the migration of customer service from offline channels to online channels and allows the organization to understand where the focus areas need to be. Does the site need to improve or does the promotion of the offline capabilities need to increase? Such a framework can help the organization to understand the most effective way to hit the initiative's goals and objectives.

With data everywhere, it's easy not to be able to see the forest for the trees. Developing multi-sourced measurement frameworks is the way to add structure to your data and to focus on the metrics that matter.

ClickZ Live San Francisco This Year's Premier Digital Marketing Event is #CZLSF
ClickZ Live San Francisco (Aug 11-14) brings together the industry's leading practitioners and marketing strategists to deliver 4 days of educational sessions and training workshops. From Data-Driven Marketing to Social, Mobile, Display, Search and Email, this year's comprehensive agenda will help you maximize your marketing efforts and ROI. Register today!

ABOUT THE AUTHOR

Neil Mason

Neil Mason is SVP, Customer Engagement at iJento. He is responsible for providing iJento clients with the most valuable customer insights and business benefits from iJento's digital and multichannel customer intelligence solutions.

Neil has been at the forefront of marketing analytics for over 25 years. Prior to joining iJento, Neil was Consultancy Director at Foviance, the UK's leading user experience and analytics consultancy, heading up the user experience design, research, and digital analytics practices. For the last 12 years Neil has worked predominantly in digital channels both as a marketer and as a consultant, combining a strong blend of commercial and technical understanding in the application of consumer insight to help major brands improve digital marketing performance. During this time he also served as a Director of the Web Analytics Association (DAA) for two years and currently serves as a Director Emeritus of the DAA. Neil is also a frequent speaker at conferences and events.

Neil's expertise ranges from advanced analytical techniques such as segmentation, predictive analytics, and modelling through to quantitative and qualitative customer research. Neil has a BA in Engineering from Cambridge University and an MBA and a postgraduate diploma in business and economic forecasting.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

Featured White Papers

BigDoor: The Marketers Guide to Customer Loyalty

The Marketer's Guide to Customer Loyalty
Customer loyalty is imperative to success, but fostering and maintaining loyalty takes a lot of work. This guide is here to help marketers build, execute, and maintain a successful loyalty initiative.

Marin Software: The Multiplier Effect of Integrating Search & Social Advertising

The Multiplier Effect of Integrating Search & Social Advertising
Latest research reveals 68% higher revenue per conversion for marketers who integrate their search & social advertising. In addition to the research results, this whitepaper also outlines 5 strategies and 15 tactics you can use to better integrate your search and social campaigns.

WEBINARS

    Information currently unavailable

Jobs

    • Internet Marketing Campaign Manager
      Internet Marketing Campaign Manager (Straight North, LLC) - Fort MillWe are looking for a talented Internet Marketing Campaign Manager to join the...
    • Online Marketing Coordinator
      Online Marketing Coordinator (NewMarket Health) - BaltimoreWant to learn marketing from the best minds in the business? NewMarket Health, a subsidiary...
    • Call Center Manager
      Call Center Manager (Common Sense Publishing) - Delray BeachWanted: Dynamic Call Center Manager with a Proven Track Record of Improving Response...