People preach that using testing to tune and optimize Web sites is easy. They say the tools are out there and aren't terribly difficult to implement. Yet, there are some clear problems that occur if not approached the right way.
Last time, I recapped some trends from the Omniture Summit, including the move by more companies to optimize their Web sites. Tuning and improving site performance through testing (A/B, multivariate, etc) is positive because there's often a lot of low-hanging fruit.
At the summit, many conversations revolved around testing. A relatively small number of companies say it's a regular occurrence within their business, but nearly everyone says they're starting to experiment with it or plan to very soon.
Testing Factors to Consider
Let's assume a company invests (dollars and resources) in a testing tool, whether it's Omniture Test & Target, the free Google Optimizer, or other tools, and gets it implemented properly. Let's go one step further and say these companies set the tests up correctly with the correct measurements of success, statistical significance, segmentation and the like. So far so good, right?
These may seem like basics, but when most companies start experimenting with testing they spend a lot of time struggling through these few basic things. Dedicated companies will figure these things out over time, but it can be painful and time consuming.
But that isn't where most companies fail. Most companies really struggle and leave money on the table by failing to understand the difference between treating site testing as a project versus a program. Ideally, companies want to "programize" their site testing -- this means thinking of it holistically and as an ongoing initiative instead of just a series of one-off tests.
Most companies start by looking at low-hanging fruit -- things they believe might be weak areas, that customers have complained about, or what an executive doesn't like. These are all natural places to start with and often will show some return.
But as time goes on, low-hanging opportunities become fewer and farther between and less fruitful in terms of their impact. Companies begin to struggle. One of two things happens: the desire, focus, and willingness to dedicate resources starts to fizzle out or they get smart and realize they need to look at optimization differently.
Building Testing into a Program
What does it mean to move from a project view to a program view? It means looking at testing holistically and as an ongoing initiative, not just testing something, picking the winner based on the results of one or two measurements of success, and then moving onto the next thing. It means taking lessons learned from each test, understanding the why behind tests, running follow up tests, and many more things.
Here are a few things that make a successful "programized testing" initiative:
Again, site optimization is a key trend we're seeing. Testing is taking off as the economy slows and as businesses must get more out of their sites with fewer resources and less money.
Take the time to really do it right. Low-hanging fruit won't last forever -- and the potential upside is much greater when you programize the process. Shoot me a note and let me know what you have done within your company to address many of these things.
As the Chief Performance Marketing Officer for POSSIBLE, Jason supports the agency's global Marketing Sciences and Media Services programs.
His primary role is to help POSSIBLE teams and clients use data to craft digital strategies that attract, convert, and retain customers - maximizing ongoing ROI across paid, earned, and owned channels. He believes that brands can better serve their customers by understanding audience behavior, and that messaging should be targeted to individual customers through the use of testing, behavioral targeting, and CRM initiatives.
Jason has written extensively about digital analytics, optimization and digital strategy, including an ongoing column at ClickZ.com. He is the co-author of "Actionable Web Analytics: Using Data to Make Smart Business Decisions," which is one of the leading texts in the field of digital analytics. His client roster includes Microsoft, Nike, Nokia, Dell, Ford, Sony, PayPal/eBay, P&G, Alcoa, Expedia, Mazda, Intel, and Motorola, and more. Jason is a frequent speaker at conferences and seminars around the world ranging from the Cannes Lions, Adobe Omniture Summits, eMetrics, SES, ad:tech, BazaarVoice, and many other WPP events.
Follow him on Twitter @JasonBurby.
May 22, 2013
1:00pm ET / 10:00am PT
June 5, 2013
1:00pm ET / 10:00am PT