Home  › Email › Email Marketing

Testing, Testing, Testing...

  |  November 28, 2007   |  Comments

Three tests to evaluate e-mail marketing effectiveness.

Everyone in e-mail marketing has probably sat through countless meetings where discussions go endlessly round about how to improve e-mail performance:

"Should we personalize the subject lines?"

"What's better: text e-mail or HTML?"

"What works better: short or long e-mail text?"

The only way to put these tired topics to rest once and for all, and to move on to more productive discussions, is to test.

As I mentioned in an earlier column, I'm going through this process with a client and it's very illuminating.

First we tested personalization, but it only produced a marginal improvement. Currently, we're testing text e-mail against HTML e-mail. In preliminary messages, HTML is clearly pulling out ahead in terms of the desired actions taken. Now we don't have to bat around those two discussion items any longer and can tackle more substantive tests that will hopefully make a more meaningful difference in revenue.

(Of course, you may find you get exactly the opposite results in your own testing. Sorry, you'll have to test all this yourself!)

What are other companies testing? I posed this question online and heard from Amy Bills, senior manager, field marketing, at Bulldog Solutions, about the company's recent forays into testing for its Webinar promotions and Marketing Watchdog Journal newsletter. She described three tests to consider to evaluate your e-mail marketing effectiveness.

Messaging and Price Points

Bills suggests doing a preliminary test to a subset of your list before rolling out an offer to your entire e-mail database.

Bulldog recently conducted a message test to see if there was price sensitivity for a specific package created for people looking for help with Webinar execution. The team created four e-mail messages with variations in message and price points for its lead-generation Webinars and sent them to subsets in the Bulldog community.

The results were conclusive: message B, combined with a lower price point, generated a 14.15 percent open rate, compared to 12.31 percent open rate for the least effective message and price point. Click-through rates (define) for the lower price point were more than three times that of the higher price point with the same message.

Needless to say, the results drove the messaging and price points for the wider initiative.

Highlighting New Features

While most people focus on subject lines when they conduct testing, the preview pane is closely related. MarketingSherpa says that 69 percent of people viewing e-mail at work use their preview panes. That means you should pay close attention to what appears in this little window of opportunity.

In the spring, Bulldog started adding video vignettes to its Marketing Watchdog Journal to complement the e-newsletter text. As part of the experiment, it started highlighting the video component in the preview pane. Open rates jumped 30 percent to 14.09 percent and CTRs nearly doubled to 2.20 percent.

It wasn't a fluke, since this trend continued to play out in subsequent e-newsletters. It also was helpful to Bulldog's marketing staff to be able to validate that the resources spent on video vignettes were paying off.

Fear vs. Opportunity

Fear and opportunity can be key drivers in event and Webinar marketing. How effective each driver will be often depends on the industry environment. It's worth testing to see how your audience responds to the two different hot buttons.

Bulldog tested two approaches for a recent Webinar. Subject line A incorporated urgency and opportunity: "Next Week: Make sure your lead-generation message is heard." Subject Line B incorporated fear: "Is your lead-generation a waste of time?"

The results: Subject line B, based on fear, generated a 16.06 percent open rate and a 1.52 percent CTR. Subject line A, based on urgency and opportunity, generated a 15.54 percent open rate and a 1.27 percent CTR.

While the difference wasn't striking, the data supported assertions by Bulldog's marketing VP that, with pressure mounting on CMOs to be accountable for their marketing spend, fear and uncertainty would be the primary motive for attending the Webinar.

Bulldog continues to test these two approaches and repeatedly sees that fear outperforms other motivational appeals. These findings are valuable beyond e-mail marketing, because this information also drives message decisions about how to write other copy for the Web and marketing collateral.

Bills adds one final word of encouragement. "Don't get discouraged if every test doesn't show a clear, dramatic result," she says. "It's more about consistent testing and looking at data as it accumulates -- and comparing it to your own metrics and industry benchmarks -- than about making snap judgments."

What are you testing in B2B e-mail these days? Let Karen know for future columns.

Want more e-mail marketing information? ClickZ E-Mail Reference is an archive of all our e-mail columns, organized by topic.

ClickZ Live New York Want to learn more?
Attend ClickZ Live New York March 30 - April 1. With over 15 years' experience delivering industry-leading events, ClickZ Live brings together over 60 expert speakers to offer an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!


Karen Gedney

Karen Gedney, an award-winning creative director and copywriter, shared her insights as a ClickZ Experts contributor from 2000 through 2009. She was known for her successful track record of achieving high e-mail response rates for Fortune 1000 companies and leading organizations. She died Nov. 16, 2010.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Email newsletters delivered right to your inbox. Subscribe today!




Featured White Papers

A Buyer's Guide to Affiliate Management Software

A Buyer's Guide to Affiliate Management Software
Manage your performance marketing with the right solution. Choose a platform that will mutually empower advertisers and media partners!

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.


    • Website Optimizer - SEO, CRO, Analytics
      Website Optimizer - SEO, CRO, Analytics (Marcel Digital) - ChicagoMarcel Digital, an award winning interactive marketing agency established in 2003...
    • Director of Marketing
      Director of Marketing (Patron Technology) - New YorkDirector of Marketing We are seeking a Director of Marketing to manage and build our marketing...
    • Senior Interactive Producer
      Senior Interactive Producer (Ready Set Rocket) - New YorkWhat You'll Do As a member of our team, the Senior Producer reports directly to our...