If your data rate is too low, your website experiences seasonal spikes in traffic, or your website has major issues, you may be doomed to failure.
In my last column, I wrote about some of the ridiculous excuses we often hear from companies for not starting a conversion optimization program. You should obviously try to improve your conversion rate. But in many people's minds there is a mistaken perception that landing page testing is the only tool available. The fact is, there are many situations where testing is just plain wrong, and probably doomed to failure:
If Your Data Rate Is Too Low
Say your site is fairly new and you're only seeing a few conversions per day. You're beginning to panic because you have a low conversion rate and your marketing person tells you that you should be testing so you can optimize for conversions. Unfortunately, this is one situation where your tests would most likely be inconclusive.
Testing is built on probability and statistics. So we recommend at least 10 conversion actions per day in order to conduct even a basic A/B split test. Below that level you will have too much "fuzz" and variability in your answer and will probably be unable to reach conclusive results. Or your test can stretch out over many months of data collection, leading to problems with seasonal changes in your audience's behavior.
One way to deal with a low data rate is to change what you consider to be the conversion action. If you don't have enough data to test the bottom-of-the funnel macro conversion (e.g., the sales in an e-commerce setting), go up the conversion path to find more granular and frequently appearing micro conversions, which can still ultimately impact the bottom line. For example, you can measure the number of adds-to-cart instead. These happen more frequently, and serve as a proxy for the actual down-funnel conversion that you want to measure (completed sales at the end of the checkout).
If Your Website Experiences Seasonal Spikes in Traffic
It's a month before Mother's Day, you're in the flower business, and your test-happy team wants to dust off one of those wish list items that's been sitting in the inbox for months, starting with an important landing page on your site. Just say "no"!
Testing is based on the assumption that you have a homogenous group coming to your site. If you can predict how a subset of that group acts, you can generalize that to how the rest of the users are going to act. But businesses that face seasonal shopping trends have a very different traffic mix during peak season.
For example, if you're in the flower business and a month out from Mother's Day new visitors to your site are likely price shopping, and that's the only criteria they care about: your price. These are the people who plan ahead (and probably buy their Christmas present in July as well).
Two days out from Mother's Day, the visitors who show up to your site are in a panic. These visitors are looking for express shipping and they do not care about paying extra for the flowers or the shipping. They just want to make sure that Mom gets them on time. These procrastinators (myself included) are part of a very different group.
The point is, visitors in the months leading up to the holiday are going to behave differently. If you tested for price sensitivity two months out, you'd see there was a lot of it; if you tested days out, you wouldn't see much because visitors care more about availability.
Very spiky seasonality complicates your tests. To adjust for this, you might tune your CRO for the short-term seasonal period - and only apply your findings within the same period the following year. Or you might conversely filter out any short-term peak-season impacts and treat them as anomalies, and tune for the baseline behavior throughout the rest of the off-peak year.
Here are some tips for dealing with seasonal business and testing:
If Your Website Has Major Issues
You've been testing and testing and not getting any conclusive results. You're trying to pinpoint the problem but can't seem to figure out what's going wrong. But the problem may be bigger than any one of your tests. The problem is your whole website experience.
If your site has major fundamental problems with its navigation, design, or usability, testing piece by piece is very inefficient because what the site really needs is to be rebuilt and redesigned from the ground up.
If that sounds extreme, think about the fact that conversions don't just happen in a bubble. You want your site to offer a seamless experience for users overall. If you have a crappy site, putting Band-Aids on landing pages here and there is not the answer.
When visitors are meandering around your site without clear purpose or direction, commonly using the back button, or relying on the search function over navigation, you probably have a website problem.
The overall goals of web usability are:
Your site's major issues can be remedied by focusing on three key areas:
So the next time someone tells you that you need to test that, keep in mind not all environments are sufficiently testable. If you work to understand the big picture, then you can often find alternatives to test that lead to a shared goal, or you can figure out how to best deal with data that can throw your results.
Image on home page via Shutterstock.
Join the Industry's Leading eCommerce & Direct Marketing Experts in Chicago
ClickZ Live Chicago (Nov 3-6) will deliver over 50 sessions across 4 days and 10 individual tracks, including Data-Driven Marketing, Social, Mobile, Display, Search and Email. Check out the full agenda and register by Friday, Oct 3 to take advantage of Early Bird Rates!
Tim Ash is CEO of SiteTuners.com, a landing page optimization firm that offers conversion consulting, full-service guaranteed-improvement tests, and software tools to improve conversion rates. SiteTuners' AttentionWizard.com visual attention prediction tool can be used on a landing page screenshot or mock-up to quickly identify major conversion issues. He has worked with Google, Facebook, American Express, CBS, Sony Music, Universal Studios, Verizon Wireless, Texas Instruments, and Coach.
Tim is a highly-regarded presenter at SES, eMetrics, PPC Summit, Affiliate Summit, PubCon, Affiliate Conference, and LeadsCon. He is the chairperson of ConversionConference.com, the first conference focused on improving online conversions. A columnist for several publications including ClickZ, he's host of the weekly Landing Page Optimization show and podcast on WebmasterRadio.fm. His columns can be found in the Search Engine Watch archive.
He received his B.S. and M.S. during his Ph.D. studies at UC San Diego. Tim is the author of the bestselling book, "Landing Page Optimization."
Connect with Tim on Google+.
IBM Social Analytics: The Science Behind Social Media Marketing
80% of internet users say they prefer to connect with brands via Facebook. 65% of social media users say they use it to learn more about brands, products and services. Learn about how to find more about customers' attitudes, preferences and buying habits from what they say on social media channels.
An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising
If you're considering implementing a marketing attribution model to measure and optimize your programs, this paper is a great introduction. It also includes real-life tips from marketers who have successfully implemented attribution in their organizations.
September 17, 2014
September 23, 2014
September 30, 2014
1:00pm ET/10:00am PT