Conversion Testing: How Dependable Is Your Traffic?

  |  October 12, 2010   |  Comments

When considering the suitability of your traffic mix for conversion testing, watch for three important characteristics: recurring, controllable, and stable.

Your audience is like a continual snowfall of visitors landing on your site – no two snowflake crystals are exactly alike. Streams of diverse people visit your site from a variety of traffic sources, all with different backgrounds and intents.

What's worse; each individual visitor may act differently under different circumstances and conditions. Time of day is just one factor that can have very strong behavioral effects. Someone browsing surreptitiously at work will spend less time on your site during work hours than when browsing at home during the evening.

Likewise, weekend behavior is different than that of the workweek. There are well-known differences in direct response e-mail conversion rates depending on the day of the week. But the "best" days may vary dramatically based on the audience and the offer in question. So, day of week turns out to be another unknown variable that must be considered in your conversion testing.

These variations in visitor segments and behavioral fluctuations can add to the complexity of conversion testing and tuning. Some websites have such extreme audience changes and vicious seasonality factors that conversion tuning is simply not possible. The predicted best answer will simply not hold up over time.

Consider, for example, the seasonal fluctuations of a flower-delivery site. In the last two to three weeks before Mother's day, the site will experience a disproportionately high conversion rate as compared to the rest of the year. In late April and early May, the traffic will be comprised of planners who are looking for the best arrangement at the lowest price. But in the final week before the holiday, the audience shifts to the procrastinators, who are less price-conscious and not likely to be particular about selection.

Even a seemingly homogeneous audience may display seasonal changes in Web browsing and online purchase behavior that could be completely unrelated to your product or service. For example, in the winter people from colder climates will tend to vacation in warmer places, and in the summer this pattern reverses. As a result, travel-related sites may experience broad changes in audience mix and intent that can make it difficult to conduct certain types of conversion testing.

Before planning any type of conversion testing, consider the impact that fluctuations in your traffic mix might have. Some variables can be mitigated by running conversion tests in whole-week units. But longer-term variations such as external events or seasonality can be much harder to deal with.

When considering the suitability of your traffic mix for conversion testing, watch for three important characteristics: recurring, controllable, and stable.


For optimal testing, your traffic should come from a replenishable resource. For example, PPC or banner ad traffic is essentially endless - you can get more of it as long as you are willing to pay. This supply of new visitors is important because you typically want to run conversion tests on people who have not been exposed to your company or website before. It's alright to have a high percentage of repeat visitors in your test, as long as the mix of visitors does not change and represents a roughly constant percentage of your traffic. However, this is different from the traffic coming from a single e-mail drop to a finite in-house list. E-mail campaigns come under the category of direct marketing and have their own methods for testing and improving effectiveness. However, nonrecurring traffic sources like e-mail have many drawbacks for traditional landing page optimization and must be used cautiously (if at all).


It's easy to control paid search and other online media buys. But other traffic sources are not under your command. For example, organic SEO depends on changes in the ranking algorithms of the search engines, and you cannot predict what mix of currently high-ranking keywords your traffic will arrive from. You also cannot control the context in which your site was seen (i.e., the text of the search result that was shown alongside your link). Nor can you directly specify the pages on which the traffic will land. However, SEO traffic can still be used for tests if it has a record of being historically stable in terms of the volume and mix of landing pages.


Even if your traffic is recurring and controllable, it may not be stable. For example, you may see a periodic traffic spike as one of your marketing partners pushes a special recurring campaign that drives visitors to your site. Or the turnover in the composition of your affiliate program results in a rapidly changing traffic mix. SEO traffic can also disappear overnight as the ranking algorithms are adjusted.

An old computer programming acronym is GIGO - "Garbage In, Garbage Out." This is true for landing page testing. If you can't rely on clean traffic sources for your landing page test, the results are likely to be highly suspect or downright wrong.



Tim Ash

Tim Ash is CEO of, a landing page optimization firm that offers conversion consulting, full-service guaranteed-improvement tests, and software tools to improve conversion rates. SiteTuners' visual attention prediction tool can be used on a landing page screenshot or mock-up to quickly identify major conversion issues. He has worked with Google, Facebook, American Express, CBS, Sony Music, Universal Studios, Verizon Wireless, Texas Instruments, and Coach.

Tim is a highly-regarded presenter at SES, eMetrics, PPC Summit, Affiliate Summit, PubCon, Affiliate Conference, and LeadsCon. He is the chairperson of, the first conference focused on improving online conversions. A columnist for several publications including ClickZ, he's host of the weekly Landing Page Optimization show and podcast on His columns can be found in the Search Engine Watch archive.

He received his B.S. and M.S. during his Ph.D. studies at UC San Diego. Tim is the author of the bestselling book, "Landing Page Optimization."

Connect with Tim on Google+.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!



Featured White Papers

2015 Holiday Email Guide

2015 Holiday Email Guide
The holidays are just around the corner. Download this whitepaper to find out how to create successful holiday email campaigns that drive engagement and revenue.

Three Ways to Make Your Big Data More Valuable

Three Ways to Make Your Big Data More Valuable
Big data holds a lot of promise for marketers, but are marketers ready to make the most of it to drive better business decisions and improve ROI? This study looks at the hidden challenges modern marketers face when trying to put big data to use.