Home  › Email › Email Marketing

Building Effective Measurement Models

  |  July 30, 2014   |  Comments

When deciding which measurement models to use, consider adopting aspects of the study of correlation, which may allow you to discover connections that may challenge your thinking on the critical drivers for your business.

Heading into the annual planning season (the end of 2014 is now closer than the beginning!), it's useful to stop and consider the underlying models that guide business decisions. Even more than the models, it's a great time to reflect on the basis for the models themselves. This matters, as the choice of how a model is derived fundamentally sets up success metrics, guiding how process improvements are made.

Most models begin with assumptions: a certain input leads, typically through a formula or other defined relationship, to some specific output. Increase ad spending...and awareness goes up, for example. At a deeper level, there is also a choice made that governs the basis for the model itself: is it based on a known (or believed to be known) mechanism that links actions with discrete results, or on an observed relationship between the two?

Models based on assumed linkages - called "mechanistic models" - are more common in business, probably because they are intuitively comfortable: they allow people to connect what they see with what they believe. It's easy to believe that "with increased spend on advertising, as offer will get noticed more." A mechanistic model for ad planning might therefore focus on budget and assume a relationship between awareness and spending, expressed through intermediary metrics like reach and frequency.

By comparison, models based on observed conditions and outcomes independent of assumed linkages - called "correlation models" - do away with the "why" and instead simply relate what ultimately happened to whatever else was happening at the time. These models require lots of data but they don't rely on an understanding of any given process. Correlation models are therefore powerful tools in discovering relationships between action taken and results that may have been otherwise ignored.

Here's an example: email open rates. Because we're all consumers, we tend to think in terms of consumer-observables: taste, color, location, price, convenience, etc. So, when designing an email campaign, an investment in rich graphics to catch the attention of a recipient seems logical. After all, if the graphics are striking and catch attention, then it's reasonable to assume a higher open rate.

Not so fast. Take a look at this email open rate data from MailChimp. MailChimp has email data, obviously, and lots of it. So, they can analyze all sorts of relationships: using correlation - the study of related events - they can find unusual but nonetheless robust relationships that are associated with (note: I did not say "cause") higher open rates. Take a look at these email open rate charts, showing open rate (y-axis) presented against time of receipt (left) and day of the week (right).


Now, back to the question of open rate and what drives it: Want to double your open rate? Send your email Tuesday, Wednesday, or Thursday so that it arrives in the afternoon of the time zone of your recipient. And you thought you needed an ad agency!

Of course, in real life it's not always that simple (and yes, you really do need your agency!) But here's the point: By combining mechanistic models with insights gleaned from correlation, you can improve your mechanistic model.

Who knows why open rates are higher in the afternoon? Who cares? They are. You might think (but you'd be wrong) that people would read email while having a coffee and "getting their day started" and so email rates would be higher then. But for whatever reason, they are highest midweek, afternoon.

Note as well that correlation does not imply causation: I wrote a specific section about this in the "Metrics and Measurement" section of my book, Social Customer Experience. You may want to dig deeper, and you'd certainly want to test this to make sure it applies to your business.

Regardless, as you continue refining your business models, consider adopting aspects of the study of correlation. Use this to discover apparent relationships that may challenge (or change!) your thinking on the critical drivers for your business. And for the time being send your email midweek, in the afternoon!


Dave Evans

Dave is the VP of social strategy at Lithium. Based in Austin, Dave is also the author of best-selling "Social Media Marketing: An Hour a Day," as well as "Social Media Marketing: The Next Generation of Business Engagement." Dave is a regular columnist for ClickZ, a frequent keynoter, and leads social technology and measurement workshops with the American Marketing Association as well as Social Media Executive Seminars, a C-level business training provider.

Dave has worked in social technology consulting and development around the world: with India's Publicis|2020media and its clients including the Bengaluru International Airport, Intel, Dell, United Brands, and Pepsico and with Austin's FG SQUARED and GSD&M| IdeaCity and clients including PGi, Southwest Airlines, AARP, Wal-Mart, and the PGA TOUR. Dave serves on the advisory boards for social technology startups including Palo Alto-based Friend2Friend and Mountain View-based Netbase and iGoals.

Prior, Dave was a co-founder of social customer care technology provider Social Dynamx, a product manager with Progressive Insurance, and a systems analyst with NASA| Jet Propulsion Labs. Dave co-founded Digital Voodoo, a web technology consultancy, in 1994. Dave holds a BS in physics and mathematics from the State University of New York/ Brockport and has served on the Advisory Board for ad:tech and the Measurement and Metrics Council with WOMMA.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Email newsletters delivered right to your inbox. Subscribe today!



Featured White Papers

US Consumer Device Preference Report

US Consumer Device Preference Report
Traditionally desktops have shown to convert better than mobile devices however, 2015 might be a tipping point for mobile conversions! Download this report to find why mobile users are more important then ever.

E-Commerce Customer Lifecycle

E-Commerce Customer Lifecycle
Have you ever wondered what factors influence online spending or why shoppers abandon their cart? This data-rich infogram offers actionable insight into creating a more seamless online shopping experience across the multiple devices consumers are using.




  • SEO Specialist
    SEO Specialist (Marcel Digital) - ChicagoSearch Engine Optimization (SEO) Specialist   Marcel Digital is an award winning digital marketing...
  • SEO / SEM Manager
    SEO / SEM Manager (CustomInk) - FairfaxAre you a friendly, motivated, and inquisitive individual? Are you a positive, out-going leader? Are you...
  • SEO Analyst
    SEO Analyst (XO Group) - New YorkSEO Analyst @ XO Group About this Job, You and Our Team: The XO Group SEO Team is looking for you, a passionate...