Home  › Marketing › Strategies

Improving E-Mail Performance Through Testing

  |  February 28, 2005   |  Comments

Trial and error is a central part of marketing. Tips to make your trials a bit more scientific.

Too often, marketers cling to the same campaign elements and tactics without thoroughly testing email variables to determine what will drive performance higher. In fact a Jupiter Research study I recently concluded on the topic found that overall, about 60 percent of marketers don't test. Challenged by resource constraints, most of these non-testers simply feel they don't have the time to regularly test their campaigns.

Yet our research found the marketers who are testing on a regular basis (at least every other mailing) are almost twice as likely to have email conversion rates that exceed industry averages by one to two percent, as compared with marketers who don't test. Clearly, the effort of testing email variables (e.g. subject lines, content) is offset by better results. Testing is really the only way for a marketer to determine which element drives higher results. It's critical for optimizing email campaigns. To ease into testing, let's take a look at some of the rudimentary items to test.

Identify Marketing Goals. While this sounds very basic, I do find on occasion that email becomes a primary promotional tool simply because it's inexpensive. It is easy to get mired in the monotony of getting the email out and lose sight of specific campaign or mailing goals. Before you can conduct any testing you must first know what the desired outcome is.

Identify Consistent Audience Segments. One of the first elements to test, and an early deliverable that comes out of testing, is audience segmentation. Ensure your email solution can support multiple lists, and customer profiles can be easily divided into segments. Some attributes you'll want to test related to audience segmentation include demographic data (e.g. income, age, sex); behavior (e.g. open, click); purchase history (e.g. recency, frequency); acquisition source; attitudes and domain (e.g. AOL, Yahoo). I've discussed using behavior and attitudinal segmentation in previous columns; take a look at those for more ideas.

Experiment With Message Formats. While HTML is the preferred format for most marketers, anti-spam measures, including image and HTML blocking at many leading ISPs, underscore the importance of testing message format. Look for differences in delivery, open rate, clickthrough rate and conversion. Testing formats on a domain level and obtaining insight into the domain level performance is critical to really understand the impact the message format has on performance.

Tinker With Content. One area likely to have the largest impact on campaign performance is testing different creative elements. Experiment with the subject line, personalized campaign elements, number of products and/or offers presented, and message tone. Research I've conducted has found personalization can have a dramatic impact on email conversion rates. Determining if personalization is appropriate for your campaigns can only be achieved through testing.

Test Frequency. In order to determine your optimal mailing frequency or the impact of triggered lifecycle campaigns, test message frequency over time. My column on message frequency offers some ideas on alternate mailing intervals.

While testing is beneficial and all the above items are just some of the variables you can test, it's very important to adhere to the following basic testing guidelines to measure which element is driving the results.

  • Test only one variable at a time. Do this to determine which element is impacting performance test offers and subject lines, but don't test both at the same time.

  • Maintain a control group. To understand how the segments that are being tested behave over time, maintain a control group that doesn't receive any of the testing treatments.

  • Ensure tests occur on the same day. To minimize fluctuations in day-of-week response patterns, ensure all segments receive the tests on the same day of the week. That is, unless you're testing day of week mailing patterns. If so, constrict the test to that one variable.

  • Ensure tests are statically accurate. While the size of mailing test cells depends on your mailing list and testing method (A/B, Nth, etc.), it's recommended that each test cell can return at least 100 qualified responses. Based on response rates, this may require cells that contain as many as 10,000 to 15,000 names.

Trial and error is a central part of marketing. Hopefully, these suggestions will make your trials a bit more scientific. Good luck -- and let me know how it turns out.

I'm currently conducting a survey of email marketing executives. Please click here to participate. Thanks!

ClickZ Live New York Want to learn more?
Attend ClickZ Live New York March 30 - April 1. With over 15 years' experience delivering industry-leading events, ClickZ Live brings together over 60 expert speakers to offer an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!


David Daniels

For more than 20 years, David has been an industry proponent. Direct Magazine said David is "one of the most influential experts in email marketing, if not the most influential." Co-author of "Email Marketing An Hour A Day," David has held senior level positions at Forrester and JupiterResearch, Apple, Anthropologie, MacWarehouse, Proteam, and retailers that dotted the early days of CompuServe. David advises many industry organizations including the OTA, DMA, eec, and has been a contributor to the Weekend Today Show on NBC. Learn more about connected marketing and download free research with registration here. Follow David on Twitter @emaildaniels and learn more at www.relevancygroup.com.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Marketing newsletter delivered to you. Subscribe today!




Featured White Papers

A Buyer's Guide to Affiliate Management Software

A Buyer's Guide to Affiliate Management Software
Manage your performance marketing with the right solution. Choose a platform that will mutually empower advertisers and media partners!

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.


    • Website Optimizer - SEO, CRO, Analytics
      Website Optimizer - SEO, CRO, Analytics (Marcel Digital) - ChicagoMarcel Digital, an award winning interactive marketing agency established in 2003...
    • Director of Marketing
      Director of Marketing (Patron Technology) - New YorkDirector of Marketing We are seeking a Director of Marketing to manage and build our marketing...
    • Senior Interactive Producer
      Senior Interactive Producer (Ready Set Rocket) - New YorkWhat You'll Do As a member of our team, the Senior Producer reports directly to our...