E-Mail List Testing, Step by Step

Testing rented email lists’ value determines how effectively they generate new customers and how to model them to find similar successful ones to expand your email program. As in the postal world, continually testing, retesting, and rolling out on email lists in a prudent manner is a best practice.

Though the email environment is different, tried-and-true list-testing principles apply in the online world, too. Assume you have winning creative. Let’s focus on determining a winning list.

Every email list has its own set of values, its own persona, if you will. Depending on how the list was created, it will be more or less responsive to your offer.

A list’s persona is the result of the list members’ characteristics:

• Age
• Gender
• Location
• Interests
• Household income
• Education level

More important are other factors, including:

• How did people get on this list? Did they:
  • Enter a sweepstakes?
  • Sign up for a newsletter?
  • Purchase a product or service?
  • Request a free trial:
    • With a credit card?
    • Without a credit card?
• How long have they been on the list?
• Was the list created:
  • Online?
  • Offline?

   

• How much permission was granted?
  • Was the process opt-in or -out?
  • Can people easily unsubscribe themselves from the list?

   

• What was the offer used for each list?

It’s fairly well accepted sweepstakes-generated lists aren’t as responsive as others. Some percentage of list members entered the sweepstakes with no real interest in the product or service related to it.

Step 1: Track All Lists

Though it may sound elementary, it bears mentioning. Make sure you can track and source-code all leads and orders resulting from every list you use. The tracking code for each list must become a permanent part of the customer record. Whether that customer orders next week or next year, you can accurately attribute the order to the right list source.

Step 2: Compute Your Key Metric

Use a key metric to evaluate new customers from any source. It may be net present value, acquisition cost, or a host of others. For each list, compute this metric against previously established benchmarks.

It’s critical to find the right balance between initial and long-term results. Some lists may have a higher initial response rate, then fall off significantly. Others may start off with lower response, but repeat response behavior is significant. The key to developing an acquisition marketing plan is to keep one eye on initial performance, the other on retention rates.

Step 3: Take Advantage of Modeling Services

Many list brokers offer scoring and modeling services you may be able to apply to your lists. A rented email list could be matched against a known database to identify common data, such as household income, or vehicle and home ownership. Combined with your actual results, these models can point you to the right lists to test. Certain list owners directly provide response modeling services on their files.

Step 4: Step Up Winners, Retest Marginal, Analyze Losers

For lists that are clear winners, use a step-up process for the next mailing. If you used 50,000 names the first time, don’t jump to 5 million, especially if you rent the list and pay for the names. Instead, step up to 250,000. If results hold, step up to 500,000. Study winners’ characteristics, and look for other lists with similar personas to test.

For marginal lists, retest if results are within one standard deviation. If the customer acquisition cost is $25, retest lists where the acquisition costs were up to $49. Anything over $49, don’t retest. If the results fall within this framework, consider retesting offer, creative, price, or some combination thereof. Think about the list’s persona and the email message you sent. Try to figure out what you can do to enhance results.

For lists that are clear losers, first determine if something went wrong during execution. Were the correct offer, price, and creative message sent? Were the names delivered at an acceptable rate? Did spam affect the mailing? In some cases, a technological problem may have influenced results. Lacking that, if it didn’t work, move on and test other lists.

By testing, tweaking, and retesting, you may turn winners into screaming winners and marginal lists into winners as well. Don’t give up on a list because the initial metric isn’t where you want it to be, especially if you have a continuity or replenishing product or service where back-end results could tip the scales in the list’s favor.

There’s a wealth on information in the direct mail world about testing, models, regression analysis, and other tools and techniques. If you’re truly serious about making email work, do some research and apply what’s worked in the postal world.

Next, I’ll explore what you can do to aid customer retention. That’s really where the ballgame is won or lost.

Keep reading…

Want more email marketing information? ClickZ E-Mail Reference is an archive of all our email columns, organized by topic.