Home  › Email › Email Marketing

E-mail Testing: A Real-World Approach

  |  February 6, 2008   |  Comments

Six simple steps to testing your e-mail campaign, courtesy of SubscriberMail CEO Jordan Ayan.

I recently read "The Practical Guide to E-mail Marketing" by Jordan Ayan, CEO of SubscriberMail and frequent speaker at ClickZ's e-mail forums.

It's a great CliffsNotes introduction to e-mail marketing (only 55 pages total) that can help online marketing newbies quickly get up to speed.

While the whole guide is very helpful, the section on e-mail testing stands out. It covers the real-world thought process of anyone who's thinking about testing her e-mail program's effectiveness while providing a simple road map to follow.

I've excerpted the section because I think it outlines testing scenarios in the way people really think (or should be thinking) about testing.

Hand out this checklist next time you go into a meeting to discuss how to improve your e-mail marketing performance. It will help you guide the discussions more productively -- and probably hit upon the most profitable areas to test.

    Step 1: Ask a Question

    Start the testing process by asking a question. What are you hoping for? Determine a specific goal to accomplish rather than attempt multiple goals with one blanket approach. A series of small steps can be easy to test and analyze:

    • I'd like to have more people open my messages.

    • I'd like to have more people click through to my Web site.

    • I'd like to reengage with historically inactive people.

    • I'd like to have people click on a specific area, topic, or action.

    Step 2: Form a Theory

    Use your marketing experience and best practice knowledge to determine what aspects may make a difference in achieving the goal you've defined.

    • I think people may be bored with my current subject lines.

    • I think that the placement of the specific content may drive more people to action.

    • I think that people may not understand this is from my organization and therefore will not interact.

    • I think my calls to action need to be stronger.

    Step 3: Create the Test

    Set up your test, following best practices. Remember, you don't need to prove the obvious.

    To optimize opens, I am going to test (one per test):

    • From name

    • Best day to send

    • Subject line

    • Best time to send

    To optimize click-throughs, I'm going to test (one per test):

    • Creative/layout

    • Subject lines

    • Copy

    • From name

    • Calls to action

    To optimize conversions, I'm going to test (one per test):

    • Landing pages

    • Calls to action

    • Creative/layout

    • Subject lines

    • Copy

    • From name

    Step 4: Segment the List

    Choose the best list or segment to test, and split it (for that specific test):

    • I'm confident this list is the most appropriate to prove or disprove my theory.

    • My list is only large enough to do an A/B split.

    • My list is large enough that I can break it into a larger control and other smaller test segments.

    • My list is large enough that I can sample a percentage of my list to test.

    Step 5: Measure and Analyze Results

    Measure and analyze results to gain insight and prove or disprove theory. Accurately compile stats (to conversions). What does it all mean? Look beyond the numbers. Even small percentage differences can mean large gains in response rates:

    • My opens increased ___%.

    • My click-throughs changed __%.

    • My conversions changed __%.

    • Traffic to my Web site increased __%.

    • My click-throughs were more focused on specific area, topic, or action.

    • My click-throughs were spread out across areas, topics, or actions.

    • Sales calls increased __%.

    Step 6: Make Changes

    Commit to making at least one change in each campaign.

    • I need to change my from name.

    • I need to change my subject line.

    • I need to specific words.

    • I need to subject line format.

    • I need to add content.

    • I need to decrease content and simplify.

    • I need to increase clickable areas or clicks.

    • I need to highlight actionable items more.

    • I need to change copy.

    • I need to modify layout.

Let me know how this checklist works for you. You can be sure I'm passing it along to my clients.

Want more e-mail marketing information? ClickZ E-Mail Reference is an archive of all our e-mail columns, organized by topic.

ClickZ Live Toronto On the heels of a fantastic event in New York City, ClickZ Live is taking the fun and learning to Toronto, June 23-25. With over 15 years' experience delivering industry-leading events, ClickZ Live offers an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!

ClickZ Live San Francisco Want to learn more? Join us at ClickZ Live San Francisco, Aug 10-12!
Educating marketers for over 15 years, ClickZ Live brings together industry thought leaders from the largest brands and agencies to deliver the most advanced, educational digital marketing agenda. Register today and save $500!


Karen Gedney

Karen Gedney, an award-winning creative director and copywriter, shared her insights as a ClickZ Experts contributor from 2000 through 2009. She was known for her successful track record of achieving high e-mail response rates for Fortune 1000 companies and leading organizations. She died Nov. 16, 2010.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Email newsletters delivered right to your inbox. Subscribe today!



Featured White Papers

Gartner Magic Quadrant for Digital Commerce

Gartner Magic Quadrant for Digital Commerce
This Magic Quadrant examines leading digital commerce platforms that enable organizations to build digital commerce sites. These commerce platforms facilitate purchasing transactions over the Web, and support the creation and continuing development of an online relationship with a consumer.

Paid Search in the Mobile Era

Paid Search in the Mobile Era
Google reports that paid search ads are currently driving 40+ million calls per month. Cost per click is increasing, paid search budgets are growing, and mobile continues to dominate. It's time to revamp old search strategies, reimagine stale best practices, and add new layers data to your analytics.




    • GREAT Campaign Project Coordinator
      GREAT Campaign Project Coordinator (British Consulate-General, New York) - New YorkThe GREAT Britain Campaign is seeking an energetic and creative...
    • Paid Search Senior Account Manager
      Paid Search Senior Account Manager (Hanapin Marketing) - BloomingtonHanapin Marketing is hiring a strategic Paid Search Senior Account Manager...
    • Paid Search Account Manager
      Paid Search Account Manager (Hanapin Marketing) - BloomingtonHanapin Marketing is hiring an experienced Paid Search Account Manager to...