Where Have All the Control Groups Gone?

  |  September 4, 2012   |  Comments

In an age where the mindset around email marketing is quantity over quality, the idea of a "control" group has all but disappeared.

Email marketing is often the unsung hero of marketing portfolio success. Instead of praise and worship, email marketing gets a bad rap. Questions like "Is it dead?" "Does it work?" "Will it go on?" continue, everywhere…except among email marketers. Go figure, right?

What is comical is that it's common to find that the budgets of the naysayers, the exploration of new channels, is often funded by…revenue generated from email marketing. The biggest challenge that faces most email marketers (and marketers in general) is attribution. There is very little way to attribute the proper revenue consideration to email, mobile, social, apps - the list continues to grow. So I ask, "Where have all the control groups gone?"

In an age where the mindset around email marketing is quantity over quality, the idea of a "control" group has all but disappeared - which is unfortunate, because it's one of the single most telling methodologies to leverage in attribution modeling.

Where to Begin

First, you need to understand your entire marketing mix and map it out. There are going to be elements outside of your control, such as mass advertising opportunities that cannot be tracked to the individual, but it's important to understand that they exist and play a role in the communication of a message or brand to the customer. Categorize your efforts in groups of targeted and mass media to see what dials you have the opportunity to turn.

Finding the Audience

Once you have determined where you will be messaging, you need to then focus on the targeted efforts and begin adjusting the "who." It's imperative that you keep two things in mind. First, the "select" for the test must be random and representative of the entire mix. No segmenting geographically or taking an alphabetical data set and slicing it after every 10,000 customers - it needs to be truly random. Second, it needs to be statistically viable; each cell of the test needs to meet the requirements for a statistically viable sample size - often 10 percent works; depending on the size of your data set you may even be able to get away with a smaller percent - but better to be safe than sorry.

Determine the Mix

Now that you have the "who," you need to figure out exactly what they're going to get. It's suggested to execute these types of tests over a period of time, especially with attribution. A one-time proof does not a point make. But consistently exposing the same audience to a similar or like marketing mix will truly help you determine what impact each channel has on the likelihood to convert.

Let's take an example here. Say you have a database of 1 million subscribers that have proper permission to receive marketing messaging in all of your four available targetable channels: direct mail, email, SMS, and push-app notifications. The hypothesis is that email is the strongest contributor to the portfolio. To keep it manageable, break the audience into four categories: email only, email + direct mail, email + direct mail + SMS, and all four channels. This would allow you to measure the impact of adding the additional channels to communication flow. Clearly the messaging needs to be on point and the timing considerations are important - but now you have something to measure.

Be Diligent

As you get everything ready to go, be sure each audience is held out in its assigned category for the duration of the test and that each category has its own set of tracking tags and categorizations so that you can easily determine who is part of which data set. While tests like this can get complex to manage, your diligence and attention to detail through the process will certainly simplify matters.

It's important to remain the master of the test - there are likely going to be scenarios where someone wants to "just get a message out to everyone via all channels," but these exceptions will certainly corrupt your test, and ultimately the ability to determine attribution and lift by channels.

ClickZ Live Toronto Twitter Canada MD Kirstine Stewart to Keynote Toronto
ClickZ Live Toronto (May 14-16) is a new event addressing the rapidly changing landscape that digital marketers face. The agenda focuses on customer engagement and attaining maximum ROI through online marketing efforts across paid, owned & earned media. Register now and save!*
*Early Bird Rates expire April 17.


Kara Trivunovic

A 12-year email marketing veteran, Kara has been actively involved in programmatic email development, execution, and strategy in a variety of senior positions on the client, agency, and provider side. Most recently, Kara was founder and principal of The Email Advisor, a respected email marketing consultancy focusing on email strategy and channel optimization. Prior to launching The Email Advisor, Kara led strategic services for the email division of Premiere Global Services, where she worked with global organizations structuring a variety of custom email education programs, conceptualizing and implementing new and innovative email programs, optimizing contact strategies, and developing staffing and budget plans.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Email newsletters delivered right to your inbox. Subscribe today!



Featured White Papers

ion Interactive Marketing Apps for Landing Pages White Paper

Marketing Apps for Landing Pages White Paper
Marketing apps can elevate a formulaic landing page into a highly interactive user experience. Learn how to turn your static content into exciting marketing apps.

eMarketer: Redefining Mobile-Only Users: Millions Selectively Avoid the Desktop

Redefining 'Mobile-Only' Users: Millions Selectively Avoid the Desktop
A new breed of selective mobile-only consumers has emerged. What are the demos of these users and how and where can marketers reach them?


    • Contact Center Professional
      Contact Center Professional (TCC: The Contact Center) - Hunt ValleyLooking to join a workforce that prides themselves on being routine and keeping...
    • Recruitment and Team Building Ambassador
      Recruitment and Team Building Ambassador (Agora Inc.) - BaltimoreAgora, www.agora-inc.com, continues to expand! In order to meet the needs of our...
    • Design and Publishing Specialist
      Design and Publishing Specialist (Bonner and Partners) - BaltimoreIf you’re a hungry self-starter, creative, organized and have an extreme...