Home  › Marketing › Strategies

Improving E-Mail Performance Through Testing

  |  February 28, 2005   |  Comments

Trial and error is a central part of marketing. Tips to make your trials a bit more scientific.

Too often, marketers cling to the same campaign elements and tactics without thoroughly testing email variables to determine what will drive performance higher. In fact a Jupiter Research study I recently concluded on the topic found that overall, about 60 percent of marketers don't test. Challenged by resource constraints, most of these non-testers simply feel they don't have the time to regularly test their campaigns.

Yet our research found the marketers who are testing on a regular basis (at least every other mailing) are almost twice as likely to have email conversion rates that exceed industry averages by one to two percent, as compared with marketers who don't test. Clearly, the effort of testing email variables (e.g. subject lines, content) is offset by better results. Testing is really the only way for a marketer to determine which element drives higher results. It's critical for optimizing email campaigns. To ease into testing, let's take a look at some of the rudimentary items to test.

Identify Marketing Goals. While this sounds very basic, I do find on occasion that email becomes a primary promotional tool simply because it's inexpensive. It is easy to get mired in the monotony of getting the email out and lose sight of specific campaign or mailing goals. Before you can conduct any testing you must first know what the desired outcome is.

Identify Consistent Audience Segments. One of the first elements to test, and an early deliverable that comes out of testing, is audience segmentation. Ensure your email solution can support multiple lists, and customer profiles can be easily divided into segments. Some attributes you'll want to test related to audience segmentation include demographic data (e.g. income, age, sex); behavior (e.g. open, click); purchase history (e.g. recency, frequency); acquisition source; attitudes and domain (e.g. AOL, Yahoo). I've discussed using behavior and attitudinal segmentation in previous columns; take a look at those for more ideas.

Experiment With Message Formats. While HTML is the preferred format for most marketers, anti-spam measures, including image and HTML blocking at many leading ISPs, underscore the importance of testing message format. Look for differences in delivery, open rate, clickthrough rate and conversion. Testing formats on a domain level and obtaining insight into the domain level performance is critical to really understand the impact the message format has on performance.

Tinker With Content. One area likely to have the largest impact on campaign performance is testing different creative elements. Experiment with the subject line, personalized campaign elements, number of products and/or offers presented, and message tone. Research I've conducted has found personalization can have a dramatic impact on email conversion rates. Determining if personalization is appropriate for your campaigns can only be achieved through testing.

Test Frequency. In order to determine your optimal mailing frequency or the impact of triggered lifecycle campaigns, test message frequency over time. My column on message frequency offers some ideas on alternate mailing intervals.

While testing is beneficial and all the above items are just some of the variables you can test, it's very important to adhere to the following basic testing guidelines to measure which element is driving the results.

  • Test only one variable at a time. Do this to determine which element is impacting performance test offers and subject lines, but don't test both at the same time.

  • Maintain a control group. To understand how the segments that are being tested behave over time, maintain a control group that doesn't receive any of the testing treatments.

  • Ensure tests occur on the same day. To minimize fluctuations in day-of-week response patterns, ensure all segments receive the tests on the same day of the week. That is, unless you're testing day of week mailing patterns. If so, constrict the test to that one variable.

  • Ensure tests are statically accurate. While the size of mailing test cells depends on your mailing list and testing method (A/B, Nth, etc.), it's recommended that each test cell can return at least 100 qualified responses. Based on response rates, this may require cells that contain as many as 10,000 to 15,000 names.

Trial and error is a central part of marketing. Hopefully, these suggestions will make your trials a bit more scientific. Good luck -- and let me know how it turns out.

I'm currently conducting a survey of email marketing executives. Please click here to participate. Thanks!

ClickZ Live San Francisco This Year's Premier Digital Marketing Event is #CZLSF
ClickZ Live San Francisco (Aug 11-14) brings together the industry's leading practitioners and marketing strategists to deliver 4 days of educational sessions and training workshops. From Data-Driven Marketing to Social, Mobile, Display, Search and Email, this year's comprehensive agenda will help you maximize your marketing efforts and ROI. Register today!

ABOUT THE AUTHOR

David Daniels

For more than 20 years, David has been an industry proponent. Direct Magazine said David is "one of the most influential experts in email marketing, if not the most influential." Co-author of "Email Marketing An Hour A Day," David has held senior level positions at Forrester and JupiterResearch, Apple, Anthropologie, MacWarehouse, Proteam, and retailers that dotted the early days of CompuServe. David advises many industry organizations including the OTA, DMA, eec, and has been a contributor to the Weekend Today Show on NBC. Learn more about connected marketing and download free research with registration here. Follow David on Twitter @emaildaniels and learn more at www.relevancygroup.com.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Marketing newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

Featured White Papers

BigDoor: The Marketers Guide to Customer Loyalty

The Marketer's Guide to Customer Loyalty
Customer loyalty is imperative to success, but fostering and maintaining loyalty takes a lot of work. This guide is here to help marketers build, execute, and maintain a successful loyalty initiative.

Marin Software: The Multiplier Effect of Integrating Search & Social Advertising

The Multiplier Effect of Integrating Search & Social Advertising
Latest research reveals 68% higher revenue per conversion for marketers who integrate their search & social advertising. In addition to the research results, this whitepaper also outlines 5 strategies and 15 tactics you can use to better integrate your search and social campaigns.

WEBINARS

    Information currently unavailable

Jobs

    • Interactive Product Manager
      Interactive Product Manager (Western Governors University) - Salt Lake CityWestern Governors University, one of the 20 largest universities...
    • SEO Senior Analyst
      SEO Senior Analyst (University of Phoenix (Apollo Education Group)) - San FranciscoSEO Senior Analyst   Position Summary...
    • SEM & Biddable Media Manager
      SEM & Biddable Media Manager (Kepler Group LLC) - New YorkAs an Optimization & Innovation Manager at Kepler Group, you will be on the bleeding...