Home  › Email › Email Marketing

E-Mail List Testing, Step by Step

  |  April 4, 2005   |  Comments

If you must rent, here’s how to identify winning e-mail lists.

Testing rented email lists’ value determines how effectively they generate new customers and how to model them to find similar successful ones to expand your email program. As in the postal world, continually testing, retesting, and rolling out on email lists in a prudent manner is a best practice.

Though the email environment is different, tried-and-true list-testing principles apply in the online world, too. Assume you have winning creative. Let’s focus on determining a winning list.

Every email list has its own set of values, its own persona, if you will. Depending on how the list was created, it will be more or less responsive to your offer.

A list’s persona is the result of the list members’ characteristics:

  • Age

  • Gender

  • Location

  • Interests

  • Household income

  • Education level

More important are other factors, including:

  • How did people get on this list? Did they:

    • Enter a sweepstakes?

    • Sign up for a newsletter?

    • Purchase a product or service?

    • Request a free trial:

      • With a credit card?

      • Without a credit card?

  • How long have they been on the list?

  • Was the list created:

    • Online?

    • Offline?

  • How much permission was granted?

    • Was the process opt-in or -out?

    • Can people easily unsubscribe themselves from the list?

  • What was the offer used for each list?

It’s fairly well accepted sweepstakes-generated lists aren’t as responsive as others. Some percentage of list members entered the sweepstakes with no real interest in the product or service related to it.

Step 1: Track All Lists

Though it may sound elementary, it bears mentioning. Make sure you can track and source-code all leads and orders resulting from every list you use. The tracking code for each list must become a permanent part of the customer record. Whether that customer orders next week or next year, you can accurately attribute the order to the right list source.

Step 2: Compute Your Key Metric

Use a key metric to evaluate new customers from any source. It may be net present value, acquisition cost, or a host of others. For each list, compute this metric against previously established benchmarks.

It’s critical to find the right balance between initial and long-term results. Some lists may have a higher initial response rate, then fall off significantly. Others may start off with lower response, but repeat response behavior is significant. The key to developing an acquisition marketing plan is to keep one eye on initial performance, the other on retention rates.

Step 3: Take Advantage of Modeling Services

Many list brokers offer scoring and modeling services you may be able to apply to your lists. A rented email list could be matched against a known database to identify common data, such as household income, or vehicle and home ownership. Combined with your actual results, these models can point you to the right lists to test. Certain list owners directly provide response modeling services on their files.

Step 4: Step Up Winners, Retest Marginal, Analyze Losers

For lists that are clear winners, use a step-up process for the next mailing. If you used 50,000 names the first time, don’t jump to 5 million, especially if you rent the list and pay for the names. Instead, step up to 250,000. If results hold, step up to 500,000. Study winners’ characteristics, and look for other lists with similar personas to test.

For marginal lists, retest if results are within one standard deviation. If the customer acquisition cost is $25, retest lists where the acquisition costs were up to $49. Anything over $49, don’t retest. If the results fall within this framework, consider retesting offer, creative, price, or some combination thereof. Think about the list’s persona and the email message you sent. Try to figure out what you can do to enhance results.

For lists that are clear losers, first determine if something went wrong during execution. Were the correct offer, price, and creative message sent? Were the names delivered at an acceptable rate? Did spam affect the mailing? In some cases, a technological problem may have influenced results. Lacking that, if it didn’t work, move on and test other lists.

By testing, tweaking, and retesting, you may turn winners into screaming winners and marginal lists into winners as well. Don’t give up on a list because the initial metric isn’t where you want it to be, especially if you have a continuity or replenishing product or service where back-end results could tip the scales in the list’s favor.

There’s a wealth on information in the direct mail world about testing, models, regression analysis, and other tools and techniques. If you’re truly serious about making email work, do some research and apply what’s worked in the postal world.

Next, I’ll explore what you can do to aid customer retention. That’s really where the ballgame is won or lost.

Keep reading...

Want more email marketing information? ClickZ E-Mail Reference is an archive of all our email columns, organized by topic.

ClickZ Live New York Want to learn more?
Attend ClickZ Live New York March 30 - April 1. With over 15 years' experience delivering industry-leading events, ClickZ Live brings together over 60 expert speakers to offer an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!


Paul Soltoff

Paul Soltoff is the chief executive officer of SendTec, Inc., a direct marketing services company specializing in customer acquisition. SendTec combines extensive direct response experience with proprietary technologies to produce scalable results. Principal services include performance-based online marketing, offline direct response marketing and direct response television. SendTec represents advertising agencies and advertisers such as RealNetworks, AARP, Monster.com, AAA, Punch Software, MyPoints, Grey Worldwide, CosmetÍque Cosmetics, Columbia House, and Euro-Pro. Prior to starting SendTec, Paul was a founder and EVP of Saatchi and Saatchi's DRTV division in New York and has over 25 years of advertising, media and direct marketing experience.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Email newsletters delivered right to your inbox. Subscribe today!




Featured White Papers

A Buyer's Guide to Affiliate Management Software

A Buyer's Guide to Affiliate Management Software
Manage your performance marketing with the right solution. Choose a platform that will mutually empower advertisers and media partners!

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.


    • Website Optimizer - SEO, CRO, Analytics
      Website Optimizer - SEO, CRO, Analytics (Marcel Digital) - ChicagoMarcel Digital, an award winning interactive marketing agency established in 2003...
    • Director of Marketing
      Director of Marketing (Patron Technology) - New YorkDirector of Marketing We are seeking a Director of Marketing to manage and build our marketing...
    • Senior Interactive Producer
      Senior Interactive Producer (Ready Set Rocket) - New YorkWhat You'll Do As a member of our team, the Senior Producer reports directly to our...