The Four Basic Email Campaign Tests

A major retailer recently introduced a new product. As part of the rollout, it decided email marketing was a suitable way to give customers an opportunity to purchase the item. Email, after all, is inexpensive compared to direct mail’s considerably higher costs. With this in mind, the company created an email message and blasted it to a 2 million name house list.

Yes, direct marketers, you read that correctly: It blasted the same message to 2 million people. That news would make traditional direct marketers shudder.

Why do so many email marketers not bother to test their offers?

Email marketers all too frequently ignore years of direct marketing lessons. They won’t or can’t or don’t test aspects of their messages. Email marketing firms complain frequently a lack of testing is the single most frustrating part of dealing with their clients. Marketers think, “Email is so inexpensive, why bother testing?”

Because email is very expensive indeed when its irrelevance drives customers away. Or when recipients delete your message at first sight. Or when your response rate is next to nothing.

Only a few years ago, offline direct marketers and their email counterparts were adversaries. Email, it was thought, would cannibalize traditional direct marketing. Because of this enmity, email marketers ignored the lessons of the well-established offline world. Well, email marketers have changed their tune and now look to their offline counterparts for lessons and best practices. Testing strategies must become part of the email marketing skill set.

Let’s contrast the retailer discussed above with Target Market magazine’s Direct Marketer of the Year. Brook Holmberg, the Christian Science Monitor’s circulation manager, has tested every conceivable aspect of direct mail campaigns. While many email marketers blast messages en masse, Holmberg tests:

  • Hard versus soft sells

  • Full-price versus discount offers
  • Different discount levels
  • Personalization
  • Follow-up mailings
  • Subscription length
  • Buy-one/get-one offers
  • Seasonality
  • Money-back guarantees
  • Mail format
  • Mail size
  • Free gift offer
  • Letter length

An impressive, thorough checklist has resulted in tests that provide response rates double that of the control group. Each mailing includes a test group that’s compared to the control group, so the newspaper’s marketers can constantly work to better response rates.

Many email marketers miss out on the insights gained from testing. They’re too focused on email’s low cost rather than the true benefits the medium affords. Email marketing’s biggest benefits are not the low cost of sending but the ability to quickly and inexpensively test and optimize offers. Direct mailers often have to wait up to two weeks to determine a test’s effectiveness. Email marketers have results in two days (without the additional costs of printing a mailing’s test run).

For those not taking advantage of message testing, a quick overview of the basics will improve your response rates:

  • Maintain a control message. To really understand if a change to an email actually affects response rates, it must be compared to a control group. Testing email is so easy that inexperienced marketers are tempted to test multiple facets of a message at once. Don’t. Stick to altering one part of the message, then compare it to the email with the best response so far. Testing more than one facet is unnecessarily complicated.

  • Test significant differences. Ensure what you’re testing differs significantly from the control group. For example, rather than slightly rewording your offer, try one hard sell and one soft sell. Or send one message with a discount and one without. Over time, alter the size of the discount.
  • Test formatting. Email service provider Silverpop’s recent study shows consumers receive a significant number of poorly formatted HTML messages because the sender didn’t test appearance in different email software. Unless you have a lab with a complete array of email clients, it’s best to have your service provider perform this function.
  • Start with the basics. I still receive email messages from large companies that leave the sender line, subject line, or both blank. This is the single best way to kill response. Imagine receiving in the mail a credit card solicitation in a plain white envelope with no return address. Marketers who are just beginning to test their messages should begin with altering subject lines. Do you personalize it? Include an offer? Is the offer dollars off or percent off? A hard or soft sell?

Testing needn’t be a complicated endeavor involving multiple cells testing different campaign aspects. A simple test of a single element will help improve your response rates. At worst, it will prove your current campaign works pretty well.

Related reading