Home  › Email › Email Marketing

Improve E-mail Performance With Testing, Part 2

  |  July 16, 2007   |  Comments

Ready to optimize your e-mail performance? Start testing! Part two of a series.

Part one of this series offers an overview on deciding what to test and some tips for developing apples-to-apples test scenarios. Today, we'll talk specifics about the most common things to test in e-mail marketing: sender lines, subject lines, and the body of the e-mail and the landing page.

Sender Lines

Always keep the actual sender line (me@mycompany.com) the same from send to send. This helps e-mail programs recognize you and deliver your e-mail to the inbox.

Only test the display sender line ("My Company," or "Me, My Company") sparingly. This is what recipients will use to recognize your e-mail; if it's constantly changing, they may not realize the e-mail is from you. But it is good to test:

  • Which of your company, brand, or product names (if you have more than one) is most effective

  • Whether adding a person's name to the sender line lifts opens

I don't recommend using sender lines that only contain a person's name. If that person changes positions or leaves your organization, you'll have to change the sender line and recipients may not recognize the new name. And if that person goes to a competitor, she can start sending e-mail and benefit from the fact your customers will recognize the name.

Subject Lines

Subject lines are a great place to test, just be sure whatever you use can be replicated for future sends. I like to construct different types of subject lines, then test those. For a promotional e-mail, for example, you might test subject lines based on one or more of the following:

  • Price

  • Savings/discount amount

  • Incentive

  • Product name

  • Product benefits

  • Offer end date

These are very basic and by no means inclusive. Your subject line can say anything (so long as it's an honest reflection of the e-mail content), so brainstorm a few ideas and go with that.

For an e-mail newsletter you might test such things as:

  • Article titles

  • Article topics

  • Article authors, if well-known

  • Promotions/offers

  • Benefits of reading the newsletter

I don't recommend testing the basics -- newsletter name, date -- because I've never seen it win against a subject line that's specific to the current issue of the e-mail newsletter. But that doesn't mean you shouldn't test it. Here, it's important to go with what works to get the highest open rate; whether you like it is secondary.

E-mail Body and Landing Page

Things you can test here fall into three general categories: product, offer, and creative. Here are some examples:

ProductOfferCreative
NameDiscount ($)Format
DescriptionDiscount (%)Copy
BenefitsIncentiveDesign
Austin30:51:17758
PriceOffer DeadlineColor
ImageCall to actionGraphics

It's fine to test just a subitem from each list. Examples:

  • Product test: with price vs. without price

  • Product test: benefit A vs. benefit B

  • Offer test: with incentive vs. without incentive

  • Offer test: 10 percent discount vs. 15 percent discount

  • Creative test: long copy vs. short copy

  • Creative test: blue vs. red as main color

Or you can really go wild and test more than one element at a time:

  • Does giving a product a different name, description, and benefits, with the same offer and creative treatment, drive more sales?

  • Testing an offer? Feel free to change the incentive, discount, and call to action, but keep the product and creative treatments the same.

  • Instead of just tweaking the color, develop a completely new creative treatment, with the same product and offer.

You can also change more than one of these categories. Try a new offer with new creative. Change the product name, description, and benefits. Just remember, if this version wins, you'll have to use it in its entirety to get the benefit to future mailings since you won't know how much each element contributed to the increase.

Proper test procedure is still required so results aren't skewed. In addition to the tips mentioned above:

  • Be sure to send all e-mail at the same time. You don't want the day or time of the send to influence results (unless that's what you're testing).

  • Make clear notes about what you're testing and which group is getting the "business as usual" version. This makes it easy to quickly determine a winner and implement what you've learned.

  • Wait at least 48 hours before declaring a winner. Early results aren't always accurate. Often, what's ahead after an hour doesn't maintain its momentum.

  • Make sure test groups are of significant quantity. You can do an analysis to determine statistically significant sample size and margin of error. Or you can use my rules of thumb:

    • Have at least 1,000 e-mail addresses per group; 5,000 is better if your list is over 10,000.

    • If your total list is smaller than 2,000, repeat the same test over a few sends until you have aggregate results for 2,000 e-mail addresses (1,000 per each group).

That's all there is to it. The more you test, the more you'll learn. That provides you ideas for other things to test. Try it yourself and let me know how it goes!

Until next time,

Jeanne

Want more e-mail marketing information? ClickZ E-Mail Reference is an archive of all our e-mail columns, organized by topic.

ClickZ Live Toronto Twitter Canada MD Kirstine Stewart to Keynote Toronto
ClickZ Live Toronto (May 14-16) is a new event addressing the rapidly changing landscape that digital marketers face. The agenda focuses on customer engagement and attaining maximum ROI through online marketing efforts across paid, owned & earned media. Register now and save!

ABOUT THE AUTHOR

Jeanne Jennings

Jeanne Jennings is a 20 year veteran of the online/email marketing industry, having started her career with CompuServe in the late 1980s. As Vice President of Global Strategic Services for Alchemy Worx, Jennings helps organizations become more effective and more profitable online. Previously Jennings ran her own email marketing consultancy with a focus on strategy; clients included AARP, Hasbro, Scholastic, Verizon and Weight Watchers International. Want to learn more? Check out her blog.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Email newsletters delivered right to your inbox. Subscribe today!

COMMENTS

UPCOMING EVENTS

Featured White Papers

ion Interactive 9 Strategies to Engage Your Consumers White Paper

9 Strategies to Engage Your Consumers
Online marketing apps are highly engaging - taking visitors on short, but effective, conversion-focused journeys. This white paper illustrates 9 strategies to engaging consumers through app-like experiences.

eMarketer: Redefining Mobile-Only Users: Millions Selectively Avoid the Desktop

Redefining 'Mobile-Only' Users: Millions Selectively Avoid the Desktop
A new breed of selective mobile-only consumers has emerged. What are the demos of these users and how and where can marketers reach them?

Resources

Jobs

    • Sr. Paid Search Manager
      Sr. Paid Search Manager (Bisk Education) - TampaCurrently seeking a Sr. Paid Search Manager in Tampa, FL for Bisk Education! Bisk Education is a...
    • Contact Center Professional
      Contact Center Professional (TCC: The Contact Center) - Hunt ValleyLooking to join a workforce that prides themselves on being routine and keeping...
    • Recruitment and Team Building Ambassador
      Recruitment and Team Building Ambassador (Agora Inc.) - BaltimoreAgora, www.agora-inc.com, continues to expand! In order to meet the needs of our...