Building Effective Measurement Models

When deciding which measurement models to use, consider adopting aspects of the study of correlation, which may allow you to discover connections that may challenge your thinking on the critical drivers for your business.

Heading into the annual planning season (the end of 2014 is now closer than the beginning!), it’s useful to stop and consider the underlying models that guide business decisions. Even more than the models, it’s a great time to reflect on the basis for the models themselves. This matters, as the choice of how a model is derived fundamentally sets up success metrics, guiding how process improvements are made.

Most models begin with assumptions: a certain input leads, typically through a formula or other defined relationship, to some specific output. Increase ad spending…and awareness goes up, for example. At a deeper level, there is also a choice made that governs the basis for the model itself: is it based on a known (or believed to be known) mechanism that links actions with discrete results, or on an observed relationship between the two?

Models based on assumed linkages – called “mechanistic models” – are more common in business, probably because they are intuitively comfortable: they allow people to connect what they see with what they believe. It’s easy to believe that “with increased spend on advertising, as offer will get noticed more.” A mechanistic model for ad planning might therefore focus on budget and assume a relationship between awareness and spending, expressed through intermediary metrics like reach and frequency.

By comparison, models based on observed conditions and outcomes independent of assumed linkages – called “correlation models” – do away with the “why” and instead simply relate what ultimately happened to whatever else was happening at the time. These models require lots of data but they don’t rely on an understanding of any given process. Correlation models are therefore powerful tools in discovering relationships between action taken and results that may have been otherwise ignored.

Here’s an example: email open rates. Because we’re all consumers, we tend to think in terms of consumer-observables: taste, color, location, price, convenience, etc. So, when designing an email campaign, an investment in rich graphics to catch the attention of a recipient seems logical. After all, if the graphics are striking and catch attention, then it’s reasonable to assume a higher open rate.

Not so fast. Take a look at this email open rate data from MailChimp. MailChimp has email data, obviously, and lots of it. So, they can analyze all sorts of relationships: using correlation – the study of related events – they can find unusual but nonetheless robust relationships that are associated with (note: I did not say “cause”) higher open rates. Take a look at these email open rate charts, showing open rate (y-axis) presented against time of receipt (left) and day of the week (right).

openrate-graph

Now, back to the question of open rate and what drives it: Want to double your open rate? Send your email Tuesday, Wednesday, or Thursday so that it arrives in the afternoon of the time zone of your recipient. And you thought you needed an ad agency!

Of course, in real life it’s not always that simple (and yes, you really do need your agency!) But here’s the point: By combining mechanistic models with insights gleaned from correlation, you can improve your mechanistic model.

Who knows why open rates are higher in the afternoon? Who cares? They are. You might think (but you’d be wrong) that people would read email while having a coffee and “getting their day started” and so email rates would be higher then. But for whatever reason, they are highest midweek, afternoon.

Note as well that correlation does not imply causation: I wrote a specific section about this in the “Metrics and Measurement” section of my book, Social Customer Experience. You may want to dig deeper, and you’d certainly want to test this to make sure it applies to your business.

Regardless, as you continue refining your business models, consider adopting aspects of the study of correlation. Use this to discover apparent relationships that may challenge (or change!) your thinking on the critical drivers for your business. And for the time being send your email midweek, in the afternoon!

Subscribe to get your daily business insights

Whitepapers

US Mobile Streaming Behavior

Whitepaper | Mobile US Mobile Streaming Behavior

5y

US Mobile Streaming Behavior

Streaming has become a staple of US media-viewing habits. Streaming video, however, still comes with a variety of pesky frustrations that viewers are ...

View resource
Winning the Data Game: Digital Analytics Tactics for Media Groups

Whitepaper | Analyzing Customer Data Winning the Data Game: Digital Analytics Tactics for Media Groups

5y

Winning the Data Game: Digital Analytics Tactics f...

Data is the lifeblood of so many companies today. You need more of it, all of which at higher quality, and all the meanwhile being compliant with data...

View resource
Learning to win the talent war: how digital marketing can develop its people

Whitepaper | Digital Marketing Learning to win the talent war: how digital marketing can develop its people

2y

Learning to win the talent war: how digital market...

This report documents the findings of a Fireside chat held by ClickZ in the first quarter of 2022. It provides expert insight on how companies can ret...

View resource
Engagement To Empowerment - Winning in Today's Experience Economy

Report | Digital Transformation Engagement To Empowerment - Winning in Today's Experience Economy

4w

Engagement To Empowerment - Winning in Today's Exp...

Customers decide fast, influenced by only 2.5 touchpoints – globally! Make sure your brand shines in those critical moments. Read More...

View resource