Home  › Marketing › Strategies

Improving E-Mail Performance Through Testing

  |  February 28, 2005   |  Comments

Trial and error is a central part of marketing. Tips to make your trials a bit more scientific.

Too often, marketers cling to the same campaign elements and tactics without thoroughly testing email variables to determine what will drive performance higher. In fact a Jupiter Research study I recently concluded on the topic found that overall, about 60 percent of marketers don't test. Challenged by resource constraints, most of these non-testers simply feel they don't have the time to regularly test their campaigns.

Yet our research found the marketers who are testing on a regular basis (at least every other mailing) are almost twice as likely to have email conversion rates that exceed industry averages by one to two percent, as compared with marketers who don't test. Clearly, the effort of testing email variables (e.g. subject lines, content) is offset by better results. Testing is really the only way for a marketer to determine which element drives higher results. It's critical for optimizing email campaigns. To ease into testing, let's take a look at some of the rudimentary items to test.

Identify Marketing Goals. While this sounds very basic, I do find on occasion that email becomes a primary promotional tool simply because it's inexpensive. It is easy to get mired in the monotony of getting the email out and lose sight of specific campaign or mailing goals. Before you can conduct any testing you must first know what the desired outcome is.

Identify Consistent Audience Segments. One of the first elements to test, and an early deliverable that comes out of testing, is audience segmentation. Ensure your email solution can support multiple lists, and customer profiles can be easily divided into segments. Some attributes you'll want to test related to audience segmentation include demographic data (e.g. income, age, sex); behavior (e.g. open, click); purchase history (e.g. recency, frequency); acquisition source; attitudes and domain (e.g. AOL, Yahoo). I've discussed using behavior and attitudinal segmentation in previous columns; take a look at those for more ideas.

Experiment With Message Formats. While HTML is the preferred format for most marketers, anti-spam measures, including image and HTML blocking at many leading ISPs, underscore the importance of testing message format. Look for differences in delivery, open rate, clickthrough rate and conversion. Testing formats on a domain level and obtaining insight into the domain level performance is critical to really understand the impact the message format has on performance.

Tinker With Content. One area likely to have the largest impact on campaign performance is testing different creative elements. Experiment with the subject line, personalized campaign elements, number of products and/or offers presented, and message tone. Research I've conducted has found personalization can have a dramatic impact on email conversion rates. Determining if personalization is appropriate for your campaigns can only be achieved through testing.

Test Frequency. In order to determine your optimal mailing frequency or the impact of triggered lifecycle campaigns, test message frequency over time. My column on message frequency offers some ideas on alternate mailing intervals.

While testing is beneficial and all the above items are just some of the variables you can test, it's very important to adhere to the following basic testing guidelines to measure which element is driving the results.

  • Test only one variable at a time. Do this to determine which element is impacting performance test offers and subject lines, but don't test both at the same time.

  • Maintain a control group. To understand how the segments that are being tested behave over time, maintain a control group that doesn't receive any of the testing treatments.

  • Ensure tests occur on the same day. To minimize fluctuations in day-of-week response patterns, ensure all segments receive the tests on the same day of the week. That is, unless you're testing day of week mailing patterns. If so, constrict the test to that one variable.

  • Ensure tests are statically accurate. While the size of mailing test cells depends on your mailing list and testing method (A/B, Nth, etc.), it's recommended that each test cell can return at least 100 qualified responses. Based on response rates, this may require cells that contain as many as 10,000 to 15,000 names.

Trial and error is a central part of marketing. Hopefully, these suggestions will make your trials a bit more scientific. Good luck -- and let me know how it turns out.

I'm currently conducting a survey of email marketing executives. Please click here to participate. Thanks!

ClickZ Live Toronto On the heels of a fantastic event in New York City, ClickZ Live is taking the fun and learning to Toronto, June 23-25. With over 15 years' experience delivering industry-leading events, ClickZ Live offers an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!

ClickZ Live San Francisco Want to learn more? Join us at ClickZ Live San Francisco, Aug 10-12!
Educating marketers for over 15 years, ClickZ Live brings together industry thought leaders from the largest brands and agencies to deliver the most advanced, educational digital marketing agenda. Register today and save $500!


David Daniels

For more than 20 years, David has been an industry proponent. Direct Magazine said David is "one of the most influential experts in email marketing, if not the most influential." Co-author of "Email Marketing An Hour A Day," David has held senior level positions at Forrester and JupiterResearch, Apple, Anthropologie, MacWarehouse, Proteam, and retailers that dotted the early days of CompuServe. David advises many industry organizations including the OTA, DMA, eec, and has been a contributor to the Weekend Today Show on NBC. Learn more about connected marketing and download free research with registration here. Follow David on Twitter @emaildaniels and learn more at www.relevancygroup.com.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Marketing newsletter delivered to you. Subscribe today!



Featured White Papers

Gartner Magic Quadrant for Digital Commerce

Gartner Magic Quadrant for Digital Commerce
This Magic Quadrant examines leading digital commerce platforms that enable organizations to build digital commerce sites. These commerce platforms facilitate purchasing transactions over the Web, and support the creation and continuing development of an online relationship with a consumer.

Paid Search in the Mobile Era

Paid Search in the Mobile Era
Google reports that paid search ads are currently driving 40+ million calls per month. Cost per click is increasing, paid search budgets are growing, and mobile continues to dominate. It's time to revamp old search strategies, reimagine stale best practices, and add new layers data to your analytics.




    • GREAT Campaign Project Coordinator
      GREAT Campaign Project Coordinator (British Consulate-General, New York) - New YorkThe GREAT Britain Campaign is seeking an energetic and creative...
    • Paid Search Senior Account Manager
      Paid Search Senior Account Manager (Hanapin Marketing) - BloomingtonHanapin Marketing is hiring a strategic Paid Search Senior Account Manager...
    • Paid Search Account Manager
      Paid Search Account Manager (Hanapin Marketing) - BloomingtonHanapin Marketing is hiring an experienced Paid Search Account Manager to...