What to Consider When Choosing a Testing Platform

• Tweet
• Buffer

Previously, I wrote about five steps you can take to start testing and how to grow a successful testing program. Once you've decided to take the plunge into continual optimization, the next step is choosing a testing platform to run your A/B and multivariate tests. With lots of options and maturing vendor solutions, companies in Asia can bypass the hurdles that Western markets have faced over the last five years and start taking advantage of more complex testing strategies right away.

Implementation - the Big Hurdle to Agile Testing

One of the biggest hurdles testing programs have faced is around technical implementation. Simply put, testing doesn't just start and stop with a good idea. There are multiple steps to getting a test off the ground and live on a site. Too often, test ideas lose momentum when the actual implementation needs to take place because developers typically have other responsibilities. So, if testing requires constant code changes, you'll notice a big delay every time you want to run a campaign.

However, if you're only planning to run a few tests per year, don't immediately close the door on tools that require test-specific implementation. The greater point is that if you don't want implementation to be an issue, you should:

• Map out how many tests you foresee running each year and across which channels.
• Involve developers early in your vendor selection so they understand their role and have an input on which implementation would work best.
• Understand how conversions are tracked and if you need integration with your current web analytics solution.

The good news is that many testing vendors have realized that implementation is a big factor to how their tools are used and have begun structuring their interfaces and support systems with the average digital marketer in mind.

Other Factors to Consider

Technical implementation isn't the only thing to consider when you're looking for a testing platform. To make the best choice for your business, you'll also want to understand the following:

• Price. Of course, the ongoing price of the tool itself will be a big factor in your decision. With different pricing models, you'll need to have a clear idea on how many tests you want to run as well as the average traffic volume your site receives.
• Test types. Understand which test types the tool can accommodate. A/B and multivariate testing are the obvious two, but you should also think about how a tool runs multivariate campaigns (full or partial factorial) as well as test automation.
• Channels. Testing doesn't need to be limited to your site or landing pages. With the rise of omni-channel marketing, testing a seamless message across channels will become even more important. If you're just getting started, I recommend sticking to the basics. However, it's good to understand the full range of channels each tool can test on so you're optimizing the areas that matter most to your business.
• Segmentation. Many tools offer basic segmentation capabilities to look at the key groups that interact with your content, but be sure to gauge how to create more sophisticated segments that truly relate to your business.
• Targeting. Once you recognize a difference in performance between groups, you'll want to leverage the data to increase conversions. Be sure your tool can target these groups and has the flexibility to assign different amounts of traffic to your creatives.
• Reporting. Testing platforms will be able to tell you when confidence is reached and the difference between your control and challengers, but think about other ways you will want to break down and integrate this data with the tools you already have.
• Management. Finally, you'll want to understand the different options you have for access, training, test strategy, and management. Some platforms offer full-service solutions while others require you to run this in-house or through an agency partner.

How to Get Started

1. Create a team of stakeholders who will help in the vendor evaluation and outline their specific requirements.
2. Do the research on what solutions are available. Below are two good resources to help you understand current options:
   • Adobe download of The Forrester Wave™: Online Testing Platforms, Q1 2013 report
     • http://success.adobe.com/assets/en/downloads/forrester/30317.The_Forrester_Wave_Online_2013.pdf
   • Comparison chart on testing software by conversion rate experts
     • http://www.conversion-rate-experts.com/split-testing-software/
3. Develop a smaller list of companies you want to receive demos from.
4. Evaluate the strengths and weaknesses of each solution and decide on the top two candidates.
5. Test them out! Most solutions offer a free trial so that you can understand how it will function within your business. I encourage you to take this time to push the boundaries to gain a better understanding on each solution's limitations.
6. Choose a solution and start optimizing your marketing efforts!

The Future of Testing

Testing will become more of a core component of companies' overall marketing strategies. As a result, tools will continue to evolve quickly. Already we are seeing big changes with leaders in the market to integrate their specific solutions into full optimization suites that will hopefully begin to crack the question of, "What do we do with all this big data?"

We will also see the progression of more sophisticated testing scenarios to keep up with the rise of the omni-channel marketing that spans pages, domains, devices, and locations.

Ultimately, over time we should see a more personalized experience that's not just based on a wide set of segmentation factors, but the specific needs and behaviors of every individual user.

Image on home page via Shutterstock.