When professional basketball players were first allowed to play in the Olympic Games, the United States assembled a "dream team" from the ranks of top NBA superstars. The expectation was that this high-powered assembly of top talent would walk all over their competition. However, the United States lost in the gold-medal match to Yugoslavia.
How could this have happened? Clearly the individual U.S. players were superior to their Yugoslav counterparts. But the Yugoslav squad had trained together and was used to playing by the slightly different rules of Olympic basketball. By contrast, the U.S. team was assembled shortly before the games and had not practiced very much. They had not jelled as a team.
Similarly, some landing page elements that you are testing may be superstars individually. But you should be looking for the combination of variables that performs best when presented together.
What is a variable interaction? Simply put, it is when the setting for one variable in your test positively or negatively influences the setting of another variable. If they have no effect on each other, they are said to be independent. In a positive interaction, two (or more) variables create a synergistic effect (yielding results that are greater than the sum of the parts). In a negative interaction, two (or more) variables undercut each other and cancel out some of the individual effects.
Let's look at a simple example. Let's assume that you are an auto dealer who sells both Ferraris and Volvos. Your goal is to sell cars and you want to test two different headlines and two different accompanying pictures. So there are a total of four possible versions based on your two variables.
If you believe that there are no interactions, then you must also believe that there is a "best" headline regardless of the accompanying picture, and that there is a "best" picture regardless of the headline used.
Clearly this is not the case. Each variable depends on the context in which it is seen. Figure 1 has a strong positive interaction (connecting the speed and power in the picture with the word "Fast" in the headline).
Figure 1 - Example ad: Picture A, Headline A
Figure 2 has a strongly negative interaction (making you think about the consequences of fast driving -"speed kills").
Figure 2 - Example ad: Picture B, Headline A
Figure 3 has a positive interaction (playing on the fear of accidents and highlighting Volvo's longstanding safety record).
Figure 3 - Example ad: Picture B, Headline B
So it's not the picture, and it's not the headline that determines the performance of the ad. It is their particular combination.
In online marketing, we want interactions. We want the picture to reinforce the headline, and the sales copy, and the offer, and the call-to-action. Similarly, we want to detect any parts of the landing page that are working at cross-purposes and undercutting the performance of other page elements. Our goal should be to find the best performing group of landing page elements.
Most landing page testing methods (such as A/B split testing and many forms of fractional factorial multivariate testing such as the Taguchi Method and Design of Experiments or DoE) assume that there are absolutely no interactions among your variables (that they are completely independent of each other). Obviously for online marketing this is an absurd assumption. Very strong interaction effects (often involving more than two variables) definitely exist, and in my experience are pretty common. So while you may be able to get some positive results by ignoring interactions, you will not be getting the best results. You will have stitched together a Frankenstein's monster made of workable pieces that do not make an effective whole. More importantly, you will have left money on the table during your landing page test.
So where can you look for interactions? In general, there is no way to guarantee that any subset of your testing elements does not interact. However, you should consider elements that are in physical proximity, or that are otherwise confounded with each other. For example, let's assume that you are testing a form and have chosen to test the call-to-action button color and text. Although these may seem independent, that is not the case. They both combine to create the specific presentation of the call-to-action; you should test for possible interactions.
Similarly, if you are testing different headlines followed by different sales copy, you should expect interactions. The headline is supposed to draw the visitor into reading further. If there is a disconnect between the headline and the following text, you can expect negative interactions. If they reinforce each other, you should expect positive synergies. So far I have primarily focused on interactions between two test elements. In fact, there are often strong interactions among several variables on a landing page. Think through the possible combinations of variables in your test for best results and use testing technology that allows you to find any interactions that exist.
Meet Your Favorite ClickZ Contributors
Many of ClickZ's leading expert contributors will be at ClickZ Live, the new online and digital marketing event kicking off in New York (March 31-April 3). Hear from the likes of: Jeremy Hull, Lisa Raehsler, Andrew Goodman, Bryan Eisenberg, Mathew Sweezey, Aaron Kahlow, Stephanie Miller, Simms Jenkins, Jeanne S. Jennings, Dave Hendricks and more!
Tim Ash is CEO of SiteTuners.com, a landing page optimization firm that offers conversion consulting, full-service guaranteed-improvement tests, and software tools to improve conversion rates. SiteTuners' AttentionWizard.com visual attention prediction tool can be used on a landing page screenshot or mock-up to quickly identify major conversion issues. He has worked with Google, Facebook, American Express, CBS, Sony Music, Universal Studios, Verizon Wireless, Texas Instruments, and Coach.
Tim is a highly-regarded presenter at SES, eMetrics, PPC Summit, Affiliate Summit, PubCon, Affiliate Conference, and LeadsCon. He is the chairperson of ConversionConference.com, the first conference focused on improving online conversions. A columnist for several publications including ClickZ, he's host of the weekly Landing Page Optimization show and podcast on WebmasterRadio.fm. His columns can be found in the Search Engine Watch archive.
He received his B.S. and M.S. during his Ph.D. studies at UC San Diego. Tim is the author of the bestselling book, "Landing Page Optimization."
Connect with Tim on Google+.
March 19, 2014