Home  › Email › Email Marketing

Building Effective Measurement Models

  |  July 30, 2014   |  Comments

When deciding which measurement models to use, consider adopting aspects of the study of correlation, which may allow you to discover connections that may challenge your thinking on the critical drivers for your business.

Heading into the annual planning season (the end of 2014 is now closer than the beginning!), it's useful to stop and consider the underlying models that guide business decisions. Even more than the models, it's a great time to reflect on the basis for the models themselves. This matters, as the choice of how a model is derived fundamentally sets up success metrics, guiding how process improvements are made.

Most models begin with assumptions: a certain input leads, typically through a formula or other defined relationship, to some specific output. Increase ad spending...and awareness goes up, for example. At a deeper level, there is also a choice made that governs the basis for the model itself: is it based on a known (or believed to be known) mechanism that links actions with discrete results, or on an observed relationship between the two?

Models based on assumed linkages - called "mechanistic models" - are more common in business, probably because they are intuitively comfortable: they allow people to connect what they see with what they believe. It's easy to believe that "with increased spend on advertising, as offer will get noticed more." A mechanistic model for ad planning might therefore focus on budget and assume a relationship between awareness and spending, expressed through intermediary metrics like reach and frequency.

By comparison, models based on observed conditions and outcomes independent of assumed linkages - called "correlation models" - do away with the "why" and instead simply relate what ultimately happened to whatever else was happening at the time. These models require lots of data but they don't rely on an understanding of any given process. Correlation models are therefore powerful tools in discovering relationships between action taken and results that may have been otherwise ignored.

Here's an example: email open rates. Because we're all consumers, we tend to think in terms of consumer-observables: taste, color, location, price, convenience, etc. So, when designing an email campaign, an investment in rich graphics to catch the attention of a recipient seems logical. After all, if the graphics are striking and catch attention, then it's reasonable to assume a higher open rate.

Not so fast. Take a look at this email open rate data from MailChimp. MailChimp has email data, obviously, and lots of it. So, they can analyze all sorts of relationships: using correlation - the study of related events - they can find unusual but nonetheless robust relationships that are associated with (note: I did not say "cause") higher open rates. Take a look at these email open rate charts, showing open rate (y-axis) presented against time of receipt (left) and day of the week (right).


Now, back to the question of open rate and what drives it: Want to double your open rate? Send your email Tuesday, Wednesday, or Thursday so that it arrives in the afternoon of the time zone of your recipient. And you thought you needed an ad agency!

Of course, in real life it's not always that simple (and yes, you really do need your agency!) But here's the point: By combining mechanistic models with insights gleaned from correlation, you can improve your mechanistic model.

Who knows why open rates are higher in the afternoon? Who cares? They are. You might think (but you'd be wrong) that people would read email while having a coffee and "getting their day started" and so email rates would be higher then. But for whatever reason, they are highest midweek, afternoon.

Note as well that correlation does not imply causation: I wrote a specific section about this in the "Metrics and Measurement" section of my book, Social Customer Experience. You may want to dig deeper, and you'd certainly want to test this to make sure it applies to your business.

Regardless, as you continue refining your business models, consider adopting aspects of the study of correlation. Use this to discover apparent relationships that may challenge (or change!) your thinking on the critical drivers for your business. And for the time being send your email midweek, in the afternoon!


Dave Evans

Dave is the VP of social strategy at Lithium. Based in Austin, Dave is also the author of best-selling "Social Media Marketing: An Hour a Day," as well as "Social Media Marketing: The Next Generation of Business Engagement." Dave is a regular columnist for ClickZ, a frequent keynoter, and leads social technology and measurement workshops with the American Marketing Association as well as Social Media Executive Seminars, a C-level business training provider.

Dave has worked in social technology consulting and development around the world: with India's Publicis|2020media and its clients including the Bengaluru International Airport, Intel, Dell, United Brands, and Pepsico and with Austin's FG SQUARED and GSD&M| IdeaCity and clients including PGi, Southwest Airlines, AARP, Wal-Mart, and the PGA TOUR. Dave serves on the advisory boards for social technology startups including Palo Alto-based Friend2Friend and Mountain View-based Netbase and iGoals.

Prior, Dave was a co-founder of social customer care technology provider Social Dynamx, a product manager with Progressive Insurance, and a systems analyst with NASA| Jet Propulsion Labs. Dave co-founded Digital Voodoo, a web technology consultancy, in 1994. Dave holds a BS in physics and mathematics from the State University of New York/ Brockport and has served on the Advisory Board for ad:tech and the Measurement and Metrics Council with WOMMA.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Email newsletters delivered right to your inbox. Subscribe today!



Featured White Papers

2015 Holiday Email Guide

2015 Holiday Email Guide
The holidays are just around the corner. Download this whitepaper to find out how to create successful holiday email campaigns that drive engagement and revenue.

Three Ways to Make Your Big Data More Valuable

Three Ways to Make Your Big Data More Valuable
Big data holds a lot of promise for marketers, but are marketers ready to make the most of it to drive better business decisions and improve ROI? This study looks at the hidden challenges modern marketers face when trying to put big data to use.