Home  › Email › Email Marketing

Improve E-Mail Performance With Testing, Part 1

  |  July 2, 2007   |  Comments

Ready to optimize your e-mail performance? Start testing! Part one of a series.

One of the easiest, least expensive, and most overlooked ways to make e-mail efforts more effective is testing. This is true no matter what type of e-mail you send, no matter what your e-mail business goals. Through testing I've help clients realize:

  • A 7 percent increase in opens by mailing a few hours earlier

  • A 50 percent increase in clicks by adding a single paragraph

  • A 75 percent increase in sales by moving the offer copy

With e-mail, small changes can make a big difference in results. But organizations looking to test and optimize e-mail face two challenges: figuring out what to test and maintaining an apples-to-apples test environment.

Testing Approaches

There are two different ways to approach testing, both valid. Which you use is determined by your circumstance and the results you're looking for.

If you want to get a read on how a single element will affect performance, change only that element for each test. If you want to improve performance and don't need to link the increase back to a single element, test multiple elements all at once.

Let's say your e-mail does very well overall; it meets or exceeds goals and industry benchmarks. You're just looking to get a little lift in your open rate. That's the time to test a single element.

But say your e-mail isn't doing so well. You're not meeting goals and are lagging way behind industry benchmarks. You may even be losing money every time you send. Then it's time for a major overhaul. Test a number of elements, all at once, to really pump up performance.

What to Test?

Once you've figured out whether you're testing a single element or a bunch, it's time to identify exactly what to change. You can test just about anything, the key is figuring out which tests will boost performance the most.

One of the easiest ways to get a boost is to test sending e-mail on different days and at different times. When I tested sending a client's e-mail newsletter at 12:30 p.m. versus 3:00 p.m., we saw a 7 percent increase in opens and a 10 percent increase in clicks from the earlier time slot. But don't just shift your sends to Mondays at 12:30 p.m. -- test it for yourself!

Proper test procedure is critical, particularly when you're doing day and time testing. Here are some tips:

  • Randomly split your list into two or more groups. Use one as your "control" or "business as usual" group; make the other ones test cells with the elements you're testing changed.

  • If you're testing day and time, you obviously can't send the e-mail all at once, but do send the groups as close together as you can to minimize outside factors:

    • When we tested sending at 12:30 p.m. versus sending at 3:00 p.m., we sent both e-mail messages on the same day.

    • If you're testing Tuesday versus Thursday, send the messages the same week so there's one day in between, not four.

  • When you're doing a day or time test, keep everything else the same to avoid having too many moving parts. If you figure out which days and times are best, you can focus on testing your product, offer, and creative elements.

I like to break the other tested elements into categories, based on which metric is most likely to move. If you're testing a single element, the key metric should have a domino effect on the actions that follow. A lift in open rates should translate into more clicks and more conversions. If you're testing multiple elements, this doesn't always hold. A lift in opens from one element may be offset by a decrease in clicks from another.

ElementsKey Metric
Sender and subject linesOpen rate
Anything in e-mail body (product, offer, creative)Click-to-open rate
Anything on landing page (product, offer, creative)Click-to-conversion rate

Next, I'll cover some tips for testing sender and subject lines, e-mail body content, and landing pages. In the meantime, figure out what you want to improve about your e-mail and which elements you should focus on to reach your goals.

Until next time,


Want more e-mail marketing information? ClickZ E-Mail Reference is an archive of all our e-mail columns, organized by topic.

ClickZ Live Chicago Join the Industry's Leading eCommerce & Direct Marketing Experts in Chicago
ClickZ Live Chicago (Nov 3-6) will deliver over 50 sessions across 4 days and 10 individual tracks, including Data-Driven Marketing, Social, Mobile, Display, Search and Email. Check out the full agenda and register by Friday, Oct 3 to take advantage of Early Bird Rates!


Jeanne Jennings

Jeanne Jennings is a 20 year veteran of the online/email marketing industry, having started her career with CompuServe in the late 1980s. As Vice President of Global Strategic Services for Alchemy Worx, Jennings helps organizations become more effective and more profitable online. Previously Jennings ran her own email marketing consultancy with a focus on strategy; clients included AARP, Hasbro, Scholastic, Verizon and Weight Watchers International. Want to learn more? Check out her blog.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Email newsletters delivered right to your inbox. Subscribe today!



Featured White Papers

IBM: Social Analytics - The Science Behind Social Media Marketing

IBM Social Analytics: The Science Behind Social Media Marketing
80% of internet users say they prefer to connect with brands via Facebook. 65% of social media users say they use it to learn more about brands, products and services. Learn about how to find more about customers' attitudes, preferences and buying habits from what they say on social media channels.

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising
If you're considering implementing a marketing attribution model to measure and optimize your programs, this paper is a great introduction. It also includes real-life tips from marketers who have successfully implemented attribution in their organizations.


    • Internet Marketing Campaign Manager
      Internet Marketing Campaign Manager (Straight North, LLC) - Downers GroveWe are looking for a talented Internet Marketing Campaign Manager...
    • Internet Marketing Specialist
      Internet Marketing Specialist (InteractRV) - DallasInternet Marketing Specialist InteractRV - Anywhere Telecommute, USA SEM | SEO | Content Creator...
    • Tier 1 Support Specialist
      Tier 1 Support Specialist (Agora Inc.) - BaltimoreThis position requires a highly motivated and multifaceted individual to contribute to and be...