What to Consider When Choosing a Testing Platform

  |  July 2, 2013   |  Comments   |  

Companies in Asia can bypass the hurdles that Western markets have faced over the last five years and start taking advantage of more complex testing strategies right away. Here's how.

Previously, I wrote about five steps you can take to start testing and how to grow a successful testing program. Once you've decided to take the plunge into continual optimization, the next step is choosing a testing platform to run your A/B and multivariate tests. With lots of options and maturing vendor solutions, companies in Asia can bypass the hurdles that Western markets have faced over the last five years and start taking advantage of more complex testing strategies right away.

Implementation - the Big Hurdle to Agile Testing

One of the biggest hurdles testing programs have faced is around technical implementation. Simply put, testing doesn't just start and stop with a good idea. There are multiple steps to getting a test off the ground and live on a site. Too often, test ideas lose momentum when the actual implementation needs to take place because developers typically have other responsibilities. So, if testing requires constant code changes, you'll notice a big delay every time you want to run a campaign.

However, if you're only planning to run a few tests per year, don't immediately close the door on tools that require test-specific implementation. The greater point is that if you don't want implementation to be an issue, you should:

  • Map out how many tests you foresee running each year and across which channels.
  • Involve developers early in your vendor selection so they understand their role and have an input on which implementation would work best.
  • Understand how conversions are tracked and if you need integration with your current web analytics solution.

The good news is that many testing vendors have realized that implementation is a big factor to how their tools are used and have begun structuring their interfaces and support systems with the average digital marketer in mind.

Other Factors to Consider

Technical implementation isn't the only thing to consider when you're looking for a testing platform. To make the best choice for your business, you'll also want to understand the following:

  • Price. Of course, the ongoing price of the tool itself will be a big factor in your decision. With different pricing models, you'll need to have a clear idea on how many tests you want to run as well as the average traffic volume your site receives.
  • Test types. Understand which test types the tool can accommodate. A/B and multivariate testing are the obvious two, but you should also think about how a tool runs multivariate campaigns (full or partial factorial) as well as test automation.
  • Channels. Testing doesn't need to be limited to your site or landing pages. With the rise of omni-channel marketing, testing a seamless message across channels will become even more important. If you're just getting started, I recommend sticking to the basics. However, it's good to understand the full range of channels each tool can test on so you're optimizing the areas that matter most to your business.
  • Segmentation. Many tools offer basic segmentation capabilities to look at the key groups that interact with your content, but be sure to gauge how to create more sophisticated segments that truly relate to your business.
  • Targeting. Once you recognize a difference in performance between groups, you'll want to leverage the data to increase conversions. Be sure your tool can target these groups and has the flexibility to assign different amounts of traffic to your creatives.
  • Reporting. Testing platforms will be able to tell you when confidence is reached and the difference between your control and challengers, but think about other ways you will want to break down and integrate this data with the tools you already have.
  • Management. Finally, you'll want to understand the different options you have for access, training, test strategy, and management. Some platforms offer full-service solutions while others require you to run this in-house or through an agency partner.

How to Get Started

  1. Create a team of stakeholders who will help in the vendor evaluation and outline their specific requirements.
  2. Do the research on what solutions are available. Below are two good resources to help you understand current options:
  3. Develop a smaller list of companies you want to receive demos from.
  4. Evaluate the strengths and weaknesses of each solution and decide on the top two candidates.
  5. Test them out! Most solutions offer a free trial so that you can understand how it will function within your business. I encourage you to take this time to push the boundaries to gain a better understanding on each solution's limitations.
  6. Choose a solution and start optimizing your marketing efforts!

The Future of Testing

Testing will become more of a core component of companies' overall marketing strategies. As a result, tools will continue to evolve quickly. Already we are seeing big changes with leaders in the market to integrate their specific solutions into full optimization suites that will hopefully begin to crack the question of, "What do we do with all this big data?"

We will also see the progression of more sophisticated testing scenarios to keep up with the rise of the omni-channel marketing that spans pages, domains, devices, and locations.

Ultimately, over time we should see a more personalized experience that's not just based on a wide set of segmentation factors, but the specific needs and behaviors of every individual user.

Image on home page via Shutterstock.


Subscribe to ClickZ Asia to receive your weekly dose of the region's latest trends, tips, and insights straight to your inbox.

ClickZ Live Toronto On the heels of a fantastic event in New York City, ClickZ Live is taking the fun and learning to Toronto, June 23-25. With over 15 years' experience delivering industry-leading events, ClickZ Live offers an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!

ClickZ Live San Francisco Want to learn more? Join us at ClickZ Live San Francisco, Aug 10-12!
Educating marketers for over 15 years, ClickZ Live brings together industry thought leaders from the largest brands and agencies to deliver the most advanced, educational digital marketing agenda. Register today and save $500!


Breanne Maliniak

Breanne Maliniak is a user experience and conversion rate optimization consultant. With over 5 years of experience in the digital space, her passion is balancing business and consumer needs to improve her clients’ profitability. With years of consulting for clients of all sizes in B2B and B2C industries, she firmly believes that a company’s success relies on happy customers and works on helping organizations discover and connect user experience to every facet of their business. By doing extensive research, Breanne’s passion for creating targeted data-driven optimization strategies has led to successful projects with marquee clients such as SAP, Four Seasons Hotels & Resorts, Yves Saint Laurent, Hewlett-Packard and Aberdeen Asset Management. In 2010, Breanne was recognized with the HSMAI Adrian Award for the Best Search Marketing Strategy.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!



Featured White Papers

Gartner Magic Quadrant for Digital Commerce

Gartner Magic Quadrant for Digital Commerce
This Magic Quadrant examines leading digital commerce platforms that enable organizations to build digital commerce sites. These commerce platforms facilitate purchasing transactions over the Web, and support the creation and continuing development of an online relationship with a consumer.

Paid Search in the Mobile Era

Paid Search in the Mobile Era
Google reports that paid search ads are currently driving 40+ million calls per month. Cost per click is increasing, paid search budgets are growing, and mobile continues to dominate. It's time to revamp old search strategies, reimagine stale best practices, and add new layers data to your analytics.




    • GREAT Campaign Project Coordinator
      GREAT Campaign Project Coordinator (British Consulate-General, New York) - New YorkThe GREAT Britain Campaign is seeking an energetic and creative...
    • Paid Search Senior Account Manager
      Paid Search Senior Account Manager (Hanapin Marketing) - BloomingtonHanapin Marketing is hiring a strategic Paid Search Senior Account Manager...
    • Paid Search Account Manager
      Paid Search Account Manager (Hanapin Marketing) - BloomingtonHanapin Marketing is hiring an experienced Paid Search Account Manager to...