Conversion Testing: How Dependable Is Your Traffic?

Your audience is like a continual snowfall of visitors landing on your site – no two snowflake crystals are exactly alike. Streams of diverse people visit your site from a variety of traffic sources, all with different backgrounds and intents.

What’s worse; each individual visitor may act differently under different circumstances and conditions. Time of day is just one factor that can have very strong behavioral effects. Someone browsing surreptitiously at work will spend less time on your site during work hours than when browsing at home during the evening.

Likewise, weekend behavior is different than that of the workweek. There are well-known differences in direct response e-mail conversion rates depending on the day of the week. But the “best” days may vary dramatically based on the audience and the offer in question. So, day of week turns out to be another unknown variable that must be considered in your conversion testing.

These variations in visitor segments and behavioral fluctuations can add to the complexity of conversion testing and tuning. Some websites have such extreme audience changes and vicious seasonality factors that conversion tuning is simply not possible. The predicted best answer will simply not hold up over time.

Consider, for example, the seasonal fluctuations of a flower-delivery site. In the last two to three weeks before Mother’s day, the site will experience a disproportionately high conversion rate as compared to the rest of the year. In late April and early May, the traffic will be comprised of planners who are looking for the best arrangement at the lowest price. But in the final week before the holiday, the audience shifts to the procrastinators, who are less price-conscious and not likely to be particular about selection.

Even a seemingly homogeneous audience may display seasonal changes in Web browsing and online purchase behavior that could be completely unrelated to your product or service. For example, in the winter people from colder climates will tend to vacation in warmer places, and in the summer this pattern reverses. As a result, travel-related sites may experience broad changes in audience mix and intent that can make it difficult to conduct certain types of conversion testing.

Before planning any type of conversion testing, consider the impact that fluctuations in your traffic mix might have. Some variables can be mitigated by running conversion tests in whole-week units. But longer-term variations such as external events or seasonality can be much harder to deal with.

When considering the suitability of your traffic mix for conversion testing, watch for three important characteristics: recurring, controllable, and stable.

Recurring

For optimal testing, your traffic should come from a replenishable resource. For example, PPC or banner ad traffic is essentially endless – you can get more of it as long as you are willing to pay. This supply of new visitors is important because you typically want to run conversion tests on people who have not been exposed to your company or website before. It’s alright to have a high percentage of repeat visitors in your test, as long as the mix of visitors does not change and represents a roughly constant percentage of your traffic. However, this is different from the traffic coming from a single e-mail drop to a finite in-house list. E-mail campaigns come under the category of direct marketing and have their own methods for testing and improving effectiveness. However, nonrecurring traffic sources like e-mail have many drawbacks for traditional landing page optimization and must be used cautiously (if at all).

Controllable

It’s easy to control paid search and other online media buys. But other traffic sources are not under your command. For example, organic SEO depends on changes in the ranking algorithms of the search engines, and you cannot predict what mix of currently high-ranking keywords your traffic will arrive from. You also cannot control the context in which your site was seen (i.e., the text of the search result that was shown alongside your link). Nor can you directly specify the pages on which the traffic will land. However, SEO traffic can still be used for tests if it has a record of being historically stable in terms of the volume and mix of landing pages.

Stable

Even if your traffic is recurring and controllable, it may not be stable. For example, you may see a periodic traffic spike as one of your marketing partners pushes a special recurring campaign that drives visitors to your site. Or the turnover in the composition of your affiliate program results in a rapidly changing traffic mix. SEO traffic can also disappear overnight as the ranking algorithms are adjusted.

An old computer programming acronym is GIGO – “Garbage In, Garbage Out.” This is true for landing page testing. If you can’t rely on clean traffic sources for your landing page test, the results are likely to be highly suspect or downright wrong.