Increase Performance With Multivariate Testing

Online advertising is booming. The Interactive Advertising Bureau (IAB) reports online advertising costs at $9.6 billion for 2004, a 33 percent increase over last year. But skyrocketing customer acquisition costs aren’t matched by increased conversion. Forrester Research says conversion rates have dropped 32 percent over the past two years; over 97 percent of retail Web site visitor leave without becoming buyers.

Many of us have developed sophisticated tracking and optimization systems for online campaigns. But do we look beyond what happens at the advertising level? Do we look at what happens once the audience arrives on our Web site? If we do look, what do we do to ensure the conversion process is followed to completion?

There’s some really interesting work being done to increase conversion rates with multivariate testing. Multivariate testing allows you to analyze multiple variables at once and to isolate each variable’s individual effect.

This type of testing isn’t new. Direct marketing experts have used it for a long time. Now, more traditional and interactive agencies see value in this type of measurement and analysis.

Offermatica is making multivariate testing easy and offering it to the masses. This hosted testing and optimization service assists businesses with optimizing their online effectiveness. By creating zones on Web pages in which content can change, multiple page combinations are possible. Offermatica’s software interface also allows you to glean best-performing combinations from which you can adapt your site. Bottom line, the company enables marketers to rapidly test product and offer content to maximize selling effectiveness.

Let’s say you’ve got a campaign landing page on which you’d like to test multiple featured products and offers to see which combinations perform best. Multivariate testing is a great way to figure this out. With Offermatica, you can optimize not just the landing page but the whole user session as well. You can dramatically increase campaign performance in such areas as conversion rate, revenue per visitor, and gross margin.

Interested in implementing multivariate testing in your campaigns? Consider the following thoughts from Offermatica:

  • Never eat anything bigger than your head. Identify a small number (2-3) of higher value and higher volume segments and design tests for them. A smaller number of tests allows for more careful thought about why the results are the way they are. It is very common that ideas that are “sure things” are duds and strange things matter.

  • Prepare for the “Costanza effect.” There was an episode of “Seinfeld” where George Costanza decided to do the exact opposite of what he would normally do. As a result he gets a promotion, wins the girl… In testing, the “Costanza effect” dictates that tests will often produce results that are the exact opposite of what you were absolutely sure they would be.
  • 99 bottles of beer on the wall, 99 bottles of beer… The best results rarely come from the first test. Even when we bring all of our best experience to a focused experiment design, the real wins come from the second experiment that is optimized by the results of the first.

You can also apply multivariate testing to search text listings and display advertising to optimize creative. You can test colors, headlines, offers, images, and so on.

I spoke with James Rouche, founder and president of Offermatica, about multivariate testing. He offered these thoughts:

  • The solution must be easy to use. It needs to be something that can be leveraged by a small marketing group.

  • It must be powerful. The tool has to be smart enough to tell you when something’s statistically significant.
  • Know this will become the standard in marketing. If you don’t apply this type of testing, you’re irresponsible with your money.

I wholeheartedly agree. Yet it is a complex topic. Research it for yourself, and let’s get a dialogue going.

Related reading