Mix and Match Your Way to Failure

  |  June 8, 2010   |  Comments

Why using the best image with the best headline on your landing page doesn't always produce the best results.

A player who makes a team great is much more valuable than a great player.
-UCLA Coach John Wooden

When professional basketball players were first allowed to play in the Olympic Games, the United States assembled a "dream team" from the ranks of top NBA superstars. The expectation was that this high-powered assembly of top talent would walk all over their competition. However, the United States lost in the gold-medal match to Yugoslavia.

How could this have happened? Clearly the individual U.S. players were superior to their Yugoslav counterparts. But the Yugoslav squad had trained together and was used to playing by the slightly different rules of Olympic basketball. By contrast, the U.S. team was assembled shortly before the games and had not practiced very much. They had not jelled as a team.

Similarly, some landing page elements that you are testing may be superstars individually. But you should be looking for the combination of variables that performs best when presented together.

What is a variable interaction? Simply put, it is when the setting for one variable in your test positively or negatively influences the setting of another variable. If they have no effect on each other, they are said to be independent. In a positive interaction, two (or more) variables create a synergistic effect (yielding results that are greater than the sum of the parts). In a negative interaction, two (or more) variables undercut each other and cancel out some of the individual effects.

Let's look at a simple example. Let's assume that you are an auto dealer who sells both Ferraris and Volvos. Your goal is to sell cars and you want to test two different headlines and two different accompanying pictures. So there are a total of four possible versions based on your two variables.

If you believe that there are no interactions, then you must also believe that there is a "best" headline regardless of the accompanying picture, and that there is a "best" picture regardless of the headline used.

Clearly this is not the case. Each variable depends on the context in which it is seen. Figure 1 has a strong positive interaction (connecting the speed and power in the picture with the word "Fast" in the headline).


Figure 1 - Example ad: Picture A, Headline A

Figure 2 has a strongly negative interaction (making you think about the consequences of fast driving -"speed kills").


Figure 2 - Example ad: Picture B, Headline A

Figure 3 has a positive interaction (playing on the fear of accidents and highlighting Volvo's longstanding safety record).


Figure 3 - Example ad: Picture B, Headline B

So it's not the picture, and it's not the headline that determines the performance of the ad. It is their particular combination.

In online marketing, we want interactions. We want the picture to reinforce the headline, and the sales copy, and the offer, and the call-to-action. Similarly, we want to detect any parts of the landing page that are working at cross-purposes and undercutting the performance of other page elements. Our goal should be to find the best performing group of landing page elements.

Most landing page testing methods (such as A/B split testing and many forms of fractional factorial multivariate testing such as the Taguchi Method and Design of Experiments or DoE) assume that there are absolutely no interactions among your variables (that they are completely independent of each other). Obviously for online marketing this is an absurd assumption. Very strong interaction effects (often involving more than two variables) definitely exist, and in my experience are pretty common. So while you may be able to get some positive results by ignoring interactions, you will not be getting the best results. You will have stitched together a Frankenstein's monster made of workable pieces that do not make an effective whole. More importantly, you will have left money on the table during your landing page test.

So where can you look for interactions? In general, there is no way to guarantee that any subset of your testing elements does not interact. However, you should consider elements that are in physical proximity, or that are otherwise confounded with each other. For example, let's assume that you are testing a form and have chosen to test the call-to-action button color and text. Although these may seem independent, that is not the case. They both combine to create the specific presentation of the call-to-action; you should test for possible interactions.

Similarly, if you are testing different headlines followed by different sales copy, you should expect interactions. The headline is supposed to draw the visitor into reading further. If there is a disconnect between the headline and the following text, you can expect negative interactions. If they reinforce each other, you should expect positive synergies. So far I have primarily focused on interactions between two test elements. In fact, there are often strong interactions among several variables on a landing page. Think through the possible combinations of variables in your test for best results and use testing technology that allows you to find any interactions that exist.

ClickZ Live Chicago Join the Industry's Leading eCommerce & Direct Marketing Experts in Chicago
ClickZ Live Chicago (Nov 3-6) will deliver over 50 sessions across 4 days and 10 individual tracks, including Data-Driven Marketing, Social, Mobile, Display, Search and Email. Check out the full agenda and register by Friday, Oct 3 to take advantage of Early Bird Rates!

ABOUT THE AUTHOR

Tim Ash

Tim Ash is CEO of SiteTuners.com, a landing page optimization firm that offers conversion consulting, full-service guaranteed-improvement tests, and software tools to improve conversion rates. SiteTuners' AttentionWizard.com visual attention prediction tool can be used on a landing page screenshot or mock-up to quickly identify major conversion issues. He has worked with Google, Facebook, American Express, CBS, Sony Music, Universal Studios, Verizon Wireless, Texas Instruments, and Coach.

Tim is a highly-regarded presenter at SES, eMetrics, PPC Summit, Affiliate Summit, PubCon, Affiliate Conference, and LeadsCon. He is the chairperson of ConversionConference.com, the first conference focused on improving online conversions. A columnist for several publications including ClickZ, he's host of the weekly Landing Page Optimization show and podcast on WebmasterRadio.fm. His columns can be found in the Search Engine Watch archive.

He received his B.S. and M.S. during his Ph.D. studies at UC San Diego. Tim is the author of the bestselling book, "Landing Page Optimization."

Connect with Tim on Google+.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

Featured White Papers

IBM: Social Analytics - The Science Behind Social Media Marketing

IBM Social Analytics: The Science Behind Social Media Marketing
80% of internet users say they prefer to connect with brands via Facebook. 65% of social media users say they use it to learn more about brands, products and services. Learn about how to find more about customers' attitudes, preferences and buying habits from what they say on social media channels.

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising
If you're considering implementing a marketing attribution model to measure and optimize your programs, this paper is a great introduction. It also includes real-life tips from marketers who have successfully implemented attribution in their organizations.

Jobs

    • Tier 1 Support Specialist
      Tier 1 Support Specialist (Agora Inc.) - BaltimoreThis position requires a highly motivated and multifaceted individual to contribute to and be...
    • Recent Grads: Customer Service Representative
      Recent Grads: Customer Service Representative (Agora Financial) - BaltimoreAgora Financial, one of the nation's largest independent publishers...
    • Managing Editor
      Managing Editor (Common Sense Publishing) - BaltimoreWE’RE HIRING: WE NEED AN AMAZING EDITOR TO POLISH WORLD-CLASS CONTENT   The Palm...