Three Testing Tips to Increase Your E-mail Marketing ROI

I’ve always been amazed that even really good baseball players can miss more than half the time and still be considered great. (I’d sure like those odds on my business projects, please!) E-mail marketing testing is similar. Hitting singles and doubles is solid progress and really important, but as marketers, we always want the home runs and are disappointed when our tests don’t return huge ah-ha epiphanies. Since there are so many testable elements, and we test them individually, and only a few of them will result in dramatic findings; it can feel like your testing program is striking out. It’s not.

There are two approaches to e-mail testing – for both B2B (define) and B2C (define) marketers. In an A/B or A/B/C test, you can go wide by using creative or segmentation approaches with multiple elements that are vastly different from each other. You may find that one test cell works significantly better! But you won’t really know why. The differences are big and multi-element, so you don’t know if it’s the color or the offer or the tone or the image. The other choice is to go narrow – select testing options that are close to each other, with just one element different. You may not see a huge lift, but you will know that the lift can be assigned to that specific element.

In either case, always run the same test again at least once to validate the result. Then, use whatever learning you have to inform the next test. In this way, the singles and doubles really do add up. Even incremental results can turn into real revenue.

Testing essentials and best practices have been well documented and discussed. Be sure to limit the number of elements per test, test subject lines frequently, rinse and repeat to validate, have a very clear goal, and measure what you are testing. To that last point, if the test isn’t about the call to action, then clicks might not be the right measurement. On the other hand, if clicks are all you can measure, be sure to set up tests that impact the call to action.

Here are three testing tricks to help you improve your batting average for higher and more consistent returns.

• Develop educated testing hypotheses: Developing a good hypothesis is the most important – and most difficult – part of good testing. Your hypothesis is exactly what it sounds like – an educated guess at what will happen. An effective hypothesis is very specific and is something you can prove. For example, “Matching the subject line to the offer will increase click through.” “A boy in the image will generate more clicks than a product shot.” “IT managers will open/click more if we use an endorsement in the subject line, rather than a cost saving offer.”

  Use all the data at your disposal to inform your hypothesis:

  • Watch trends in e-mail response by age of file and type of e-mail. New subscribers often click but don’t convert, so test sending them more research and white papers.
  • Look at back data – which types of subject lines have worked well in the past, or which keywords perform best in search or postal mail? One software marketer found that using product names in the subject line performed less well then benefit statements.
  • Test the impact of secondary offers in promotions vs. transactional e-mails. A marketer to small businesses found that secondary offers in confirmation messages worked well, while they did not in promotional messages.
  • What are the logical merchandise up sells and cross sells to provide on the landing page? One retailer found that brands pull best for prospects, but generic categories with multiple brand options work best for customers.

  Never assume that response patterns are static. Re-test all those “truths” that you and your brand managers “know” about subscribers. Often, what was true a year ago, or even three months ago, is not true today.

• Segment: Unless your entire subscriber base is homogenous, testing results will dilute across the various segments. Thus, you can’t draw solid conclusions about the “why” or “how” behind your results. In particular, test prospect actions distinct from customers. Set up different tests for recent purchasers and the long inactive; men and women; and big spenders and first-time buyers. One of our B2B marketer clients found that recent buyers respond best to the personalized salutation, where the addition of first name made no difference for 120-day plus buyers.
• Technology is your friend: Work closely with your database wizards to set up strong segments for your test. Be sure you understand the persona and history of your control group, and use test cells that are statistically significant. If you don’t understand these statistical concepts, ensure that someone on the testing team is well versed. Otherwise, you can’t trust the results. If you have trouble duplicating a test result, then check your math as the likely culprit.

  A/B testing is slow, and will take time, treasure, and commitment. I’m excited about the multivariate options for e-mail marketers now available via companies like 8Seconds and Sympact. These use different approaches, but basically update the image in real time to present unique experiences for individual subscribers. This brings the success of multivariant approaches that we know work well in landing pages to the e-mail message itself.

No more excuses. The only way to earn higher ROI (define) is to iterate your content and contact strategies until you learn what your subscribers – or segments of your subscribers – really want. Testing is the answer, and marketers who test well and consistently, earn up to 25 percent more than those who don’t (according to Forrester Research, 2009). Please share your testing questions, ideas, and learnings below.

E-mail on autopilot or giving you new challenges every week? Take this five minute survey and we’ll share the results back in a future column. Thank you!