Home  › Email › Email Marketing

Strategic Testing: Tips for Success and Sources of Inspiration

  |  May 12, 2014   |  Comments

Testing is extremely important for a successful email campaign, so here are some tips to make sure you're testing in the right ways.

Last week I attended and spoke at Which Test Won: The Live Event in Atlanta (the European version of the show is taking place in London later this month). As the name suggests, the show is dedicated to testing to improve performance. If you know me, or if you've be a regular reader of my articles over the years, you'll know that I am a huge fan of testing. So this has been a fabulous experience!

Here are some tips on performance testing - both for those just starting out with A/B split or multivariate testing, or for those who have been doing it for a while.

1. Have a Solid Hypothesis

This has been one of my rallying cries for years - so it was so great to hear other speakers say it from the stage. Without a solid hypothesis, you're just guessing - and guessing is rarely as successful as having an informed point of view about what you're testing and why you think it will boost performance.
Sometimes the hypothesis is there, in your head, and you just need to massage it a bit and get it down on paper. Other times you're grasping - and there's no rhyme or reason to support your test. This latter situation is where you need to go back to the drawing board, come up with solid reasons why certain changes will boost performance, and change your test (see item three below, where I provide places to go for testing inspiration).

Here's an example: testing colors, let's say red versus green, for elements of your email. If you can not only explain why you're testing this but defend it, you have a solid hypothesis.

I did this type of color testing years ago when I worked in the stock advisory publishing industry. My solid hypothesis there was that "in the red" in relation to money was bad, while green was the color of money and was good. So I felt that making the calls to action and highlight color green, instead of red, would boost response because subliminally people would feel better about the chances that they would make money off our recommendations.

2. Technology Alone Won't Make Your Testing Program a Success

I'm a big fan of technology, but that alone won't make your testing program successful. It's the strategy behind your testing program that will make or break it, not what you use to implement it.

Garbage in, garbage out certainly applies here. If you don't have a sound hypothesis and you don't set up the test to get reliable, statistically significant results, you're wasting your time. It doesn't matter how sophisticated or expensive your technology solution is.

Don't get me wrong - I'm a fan of technology. My agency, Alchemy Worx, has a new Subject Line Analyzer Tool in beta. You input your idea for a subject line and it provides feedback based on past performance of your own email marketing as well as third-party email marketing specific to your industry. It's not something I see taking the place of traditional performance testing, but it can give you a leg up and help you fine-tune what you test to optimize results. If you'd like to learn more, feel free to email me.

3. Look for Inspiration From Other Marketers

So where can you get ideas for things to test that will move the needle and improve your bottom-line performance?

One of my favorite sources is WhichTestWon.com. They post a new test on their website each week and invite visitors to weigh in on which of the creatives they believe was more successful. Even better, the website will then tell you which test actually won and provide you quantitative details on the lift that was reported. Sign up for their email newsletter to get their tests delivered to your inbox on a regular basis. They also have a database of nearly 500 case studies you can peruse.

Not all the content here is about email marketing - but it's still a great place to go to see what others are testing and how it's working out for them. And a lot of the ideas here are very easy to adapt to email.

Another place I love for inspiration is MarketingSherpa.com. Their case studies are top-notch; they cover many channels but they have a newsletter dedicated to email marketing which is worth signing up for. But once again, don't limit yourself. See what people are testing and how it's doing - and then think about how you can adapt it to email.

Until next time,


ClickZ Live Toronto On the heels of a fantastic event in New York City, ClickZ Live is taking the fun and learning to Toronto, June 23-25. With over 15 years' experience delivering industry-leading events, ClickZ Live offers an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!

ClickZ Live San Francisco Want to learn more? Join us at ClickZ Live San Francisco, Aug 10-12!
Educating marketers for over 15 years, ClickZ Live brings together industry thought leaders from the largest brands and agencies to deliver the most advanced, educational digital marketing agenda. Register today and save $500!


Jeanne Jennings

Jeanne Jennings is a recognized expert in the email marketing industry and managing director of digital marketing for Digital Prism Advisors. She has more than 20 years of experience in the email and online marketing and product development world. Jeanne's direct-response approach to digital strategy, tactics, and creative direction helps organizations make their online marketing initiatives more effective and more profitable. Digital Prism Advisors helps established businesses unlock significant growth and revenue opportunities in the digital marketplace; our clients learn to develop and implement successful digital strategies, leveraging data and technology to better meet bottom line goals. Want to learn more? Check out Jeanne's blog and Digital Prisim Advisors.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Email newsletters delivered right to your inbox. Subscribe today!



Featured White Papers

Gartner Magic Quadrant for Digital Commerce

Gartner Magic Quadrant for Digital Commerce
This Magic Quadrant examines leading digital commerce platforms that enable organizations to build digital commerce sites. These commerce platforms facilitate purchasing transactions over the Web, and support the creation and continuing development of an online relationship with a consumer.

Paid Search in the Mobile Era

Paid Search in the Mobile Era
Google reports that paid search ads are currently driving 40+ million calls per month. Cost per click is increasing, paid search budgets are growing, and mobile continues to dominate. It's time to revamp old search strategies, reimagine stale best practices, and add new layers data to your analytics.




    • GREAT Campaign Project Coordinator
      GREAT Campaign Project Coordinator (British Consulate-General, New York) - New YorkThe GREAT Britain Campaign is seeking an energetic and creative...
    • Paid Search Senior Account Manager
      Paid Search Senior Account Manager (Hanapin Marketing) - BloomingtonHanapin Marketing is hiring a strategic Paid Search Senior Account Manager...
    • Paid Search Account Manager
      Paid Search Account Manager (Hanapin Marketing) - BloomingtonHanapin Marketing is hiring an experienced Paid Search Account Manager to...