Home  › Marketing › Strategies

Improving E-Mail Performance Through Testing

  |  February 28, 2005   |  Comments

Trial and error is a central part of marketing. Tips to make your trials a bit more scientific.

Too often, marketers cling to the same campaign elements and tactics without thoroughly testing email variables to determine what will drive performance higher. In fact a Jupiter Research study I recently concluded on the topic found that overall, about 60 percent of marketers don't test. Challenged by resource constraints, most of these non-testers simply feel they don't have the time to regularly test their campaigns.

Yet our research found the marketers who are testing on a regular basis (at least every other mailing) are almost twice as likely to have email conversion rates that exceed industry averages by one to two percent, as compared with marketers who don't test. Clearly, the effort of testing email variables (e.g. subject lines, content) is offset by better results. Testing is really the only way for a marketer to determine which element drives higher results. It's critical for optimizing email campaigns. To ease into testing, let's take a look at some of the rudimentary items to test.

Identify Marketing Goals. While this sounds very basic, I do find on occasion that email becomes a primary promotional tool simply because it's inexpensive. It is easy to get mired in the monotony of getting the email out and lose sight of specific campaign or mailing goals. Before you can conduct any testing you must first know what the desired outcome is.

Identify Consistent Audience Segments. One of the first elements to test, and an early deliverable that comes out of testing, is audience segmentation. Ensure your email solution can support multiple lists, and customer profiles can be easily divided into segments. Some attributes you'll want to test related to audience segmentation include demographic data (e.g. income, age, sex); behavior (e.g. open, click); purchase history (e.g. recency, frequency); acquisition source; attitudes and domain (e.g. AOL, Yahoo). I've discussed using behavior and attitudinal segmentation in previous columns; take a look at those for more ideas.

Experiment With Message Formats. While HTML is the preferred format for most marketers, anti-spam measures, including image and HTML blocking at many leading ISPs, underscore the importance of testing message format. Look for differences in delivery, open rate, clickthrough rate and conversion. Testing formats on a domain level and obtaining insight into the domain level performance is critical to really understand the impact the message format has on performance.

Tinker With Content. One area likely to have the largest impact on campaign performance is testing different creative elements. Experiment with the subject line, personalized campaign elements, number of products and/or offers presented, and message tone. Research I've conducted has found personalization can have a dramatic impact on email conversion rates. Determining if personalization is appropriate for your campaigns can only be achieved through testing.

Test Frequency. In order to determine your optimal mailing frequency or the impact of triggered lifecycle campaigns, test message frequency over time. My column on message frequency offers some ideas on alternate mailing intervals.

While testing is beneficial and all the above items are just some of the variables you can test, it's very important to adhere to the following basic testing guidelines to measure which element is driving the results.

  • Test only one variable at a time. Do this to determine which element is impacting performance test offers and subject lines, but don't test both at the same time.

  • Maintain a control group. To understand how the segments that are being tested behave over time, maintain a control group that doesn't receive any of the testing treatments.

  • Ensure tests occur on the same day. To minimize fluctuations in day-of-week response patterns, ensure all segments receive the tests on the same day of the week. That is, unless you're testing day of week mailing patterns. If so, constrict the test to that one variable.

  • Ensure tests are statically accurate. While the size of mailing test cells depends on your mailing list and testing method (A/B, Nth, etc.), it's recommended that each test cell can return at least 100 qualified responses. Based on response rates, this may require cells that contain as many as 10,000 to 15,000 names.

Trial and error is a central part of marketing. Hopefully, these suggestions will make your trials a bit more scientific. Good luck -- and let me know how it turns out.

I'm currently conducting a survey of email marketing executives. Please click here to participate. Thanks!

ClickZ Live Chicago Learn Digital Marketing Insights From Leading Brands!
ClickZ Live Chicago (Nov 3-6) will deliver over 50 sessions across 10 individual tracks, including Data-Driven Marketing, Social, Mobile, Display, Search and Email. Check out the full agenda, or register and attend one of the best ClickZ events yet!

ABOUT THE AUTHOR

David Daniels

For more than 20 years, David has been an industry proponent. Direct Magazine said David is "one of the most influential experts in email marketing, if not the most influential." Co-author of "Email Marketing An Hour A Day," David has held senior level positions at Forrester and JupiterResearch, Apple, Anthropologie, MacWarehouse, Proteam, and retailers that dotted the early days of CompuServe. David advises many industry organizations including the OTA, DMA, eec, and has been a contributor to the Weekend Today Show on NBC. Learn more about connected marketing and download free research with registration here. Follow David on Twitter @emaildaniels and learn more at www.relevancygroup.com.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Marketing newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

UPCOMING TRAINING

Featured White Papers

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.

5 Ways to Personalize Beyond the Subject Line

5 Ways to Personalize Beyond the Subject Line
82 percent of shoppers say they would buy more items from a brand if the emails they sent were more personalized. This white paper offer five tactics that will personalize your email beyond the subject line and drive real business growth.

WEBINARS

    Information currently unavailable

Resources

Jobs

    • Executive Assistant
      Executive Assistant (Agora Inc. ) - BaltimoreAgora Inc., an international publishing company, located in the Mt. Vernon district of Baltimore, MD...
    • Paid Search Specialist
      Paid Search Specialist (Boathouse, Inc.) - Waltham  Boathouse is looking for a Paid Search Specialist to work as a part of the Digital Acquisition...
    • Paid Search / Search Engine Marketing (SEM, PPC) Specialist
      Paid Search / Search Engine Marketing (SEM, PPC) Specialist (HeBS Digital) - New YorkJOB TITLE:     Paid Search / Search Engine Marketing...