I've always been amazed that even really good baseball players can miss more than half the time and still be considered great. (I'd sure like those odds on my business projects, please!) E-mail marketing testing is similar. Hitting singles and doubles is solid progress and really important, but as marketers, we always want the home runs and are disappointed when our tests don't return huge ah-ha epiphanies. Since there are so many testable elements, and we test them individually, and only a few of them will result in dramatic findings; it can feel like your testing program is striking out. It's not.
There are two approaches to e-mail testing - for both B2B (define) and B2C (define) marketers. In an A/B or A/B/C test, you can go wide by using creative or segmentation approaches with multiple elements that are vastly different from each other. You may find that one test cell works significantly better! But you won't really know why. The differences are big and multi-element, so you don't know if it's the color or the offer or the tone or the image. The other choice is to go narrow - select testing options that are close to each other, with just one element different. You may not see a huge lift, but you will know that the lift can be assigned to that specific element.
In either case, always run the same test again at least once to validate the result. Then, use whatever learning you have to inform the next test. In this way, the singles and doubles really do add up. Even incremental results can turn into real revenue.
Testing essentials and best practices have been well documented and discussed. Be sure to limit the number of elements per test, test subject lines frequently, rinse and repeat to validate, have a very clear goal, and measure what you are testing. To that last point, if the test isn't about the call to action, then clicks might not be the right measurement. On the other hand, if clicks are all you can measure, be sure to set up tests that impact the call to action.
Here are three testing tricks to help you improve your batting average for higher and more consistent returns.
Use all the data at your disposal to inform your hypothesis:
Never assume that response patterns are static. Re-test all those "truths" that you and your brand managers "know" about subscribers. Often, what was true a year ago, or even three months ago, is not true today.
A/B testing is slow, and will take time, treasure, and commitment. I'm excited about the multivariate options for e-mail marketers now available via companies like 8Seconds and Sympact. These use different approaches, but basically update the image in real time to present unique experiences for individual subscribers. This brings the success of multivariant approaches that we know work well in landing pages to the e-mail message itself.
No more excuses. The only way to earn higher ROI (define) is to iterate your content and contact strategies until you learn what your subscribers - or segments of your subscribers - really want. Testing is the answer, and marketers who test well and consistently, earn up to 25 percent more than those who don't (according to Forrester Research, 2009). Please share your testing questions, ideas, and learnings below.
E-mail on autopilot or giving you new challenges every week? Take this five minute survey and we'll share the results back in a future column. Thank you!
Meet Your Favorite ClickZ Contributors
Many of ClickZ's leading expert contributors will be at ClickZ Live, the new online and digital marketing event kicking off in New York (March 31-April 3). Hear from the likes of: Jeremy Hull, Lisa Raehsler, Andrew Goodman, Bryan Eisenberg, Mathew Sweezey, Aaron Kahlow, Stephanie Miller, Simms Jenkins, Jeanne S. Jennings, Dave Hendricks and more!
Stephanie Miller is a relentless customer advocate and a champion for marketers creating memorable online experiences. A digital marketing expert, she helps responsible data-driven marketers connect with the people, resources, and ideas they need to optimize response and revenue. She speaks and writes regularly and leads many industry initiatives as VP, Member Relations and Chief Listening Officer at the Direct Marketing Association (www.the-dma.org). Feedback and column ideas most welcome, to smiller AT the-dma DOT org or @stephanieSAM.
March 19, 2014