Test or Die

  |  February 26, 2010   |  Comments

When you finally decide to start testing, consider these other factors before selecting a tool.

Anyone who has read my "69 Free or Low Cost Tools to Improve Your Website" post knows that I love tools. This past week I was excited to see a new list of tools with the just published "Which Multivariate?" a multivariate testing tool comparison guide. I'm happy they beat me to the punch in putting this list of tools together, because many people have asked me about the various testing platforms and tools, and I haven't had a chance to get to it.

The Critical Parts of the Testing Equation

When it comes to a choice of testing tools, you can see there is a variety of testing software available other than Google's Website Optimizer and Omniture's Test and Target. When you finally decide to start testing, the tool should not be the first part of the equation. It's like being an emergency room doctor and having lots of crutches around and trying to use a crutch to solve every one of the issues you face. It just won't solve the problem.

  1. You first need to understand the business goals and problems you are trying to achieve.

  2. Then you must choose the right tools for that job. To select the right one, you have to weigh your needs and your company's level of sophistication when it comes to running experiments against price, and keep in mind that the tool you start out with today may not be a good fit to solve all the challenges. So flexibility is key.

  3. The last critical part of the equation is: do you have the knowledge internally to decide what to test (things that really will move the needle) and do you have the resources to get the tests executed?

As you can see steps one and three are the most important to your success, the tool is just a tool.

What to Do Is More Important Than How You Do It

A student of mine, Philip Anderson, and I were chatting about this when he reminded me of this classic story. Ever heard the story of the giant ship engine that failed? The ship's owners tried one expert after another, but none of them could figure out how to fix the engine. Then they brought in an old man who had been fixing ships since he was a youngster. He carried a large bag of tools with him, and when he arrived, he immediately went to work. He inspected the engine very carefully, top to bottom.

Two of the ship's owners were there, watching this man, hoping he would know what to do. When he went down to the engine room he felt the pipes with his hands, took out a stethoscope, listened to the pumps, and finally placed one of his hands on one of the gauges with his eyes closed. After looking things over, the old man reached into his bag and pulled out a small hammer. He gently tapped something. Instantly, the engine lurched into life. He carefully put his hammer away. The engine was fixed! A week later, the owners received a bill from the old man for $10,000.

"What?!" the owners exclaimed. "He hardly did anything!"

So they wrote the old man a note saying, "Please send us an itemized bill."

The man sent a bill that read:

Tapping with a hammer: $2.00

Knowing where to tap: $9,998.00

Effort is important, but knowing where to make an effort makes all the difference.

Can You Get It Done?

Once again, to mitigate your risk and achieve the best conversion rate lift, you want to work with someone who has a demonstrated track record of using a variety of tools: reputable, on-demand testing tools that don't rely heavily on software license revenues and will allow you flexibility to change the software as your needs evolve and your testing needs become more complex. As important as knowing what to do is, it does no good if you can't do anything about it.

You must have resources in house or else hire an independent firm or contractors to either plan, set up, or execute the variations and creative for your tests. They should be able to fill the gaps you have internally in order for you to continue improving your business goals continuously. Testing is not a one-time event you check off the list. It needs to be part of your everyday culture to succeed.

I heard Guy Kawasaki say, "Execution is not an event - a one-time push toward achieving goals. Rather, it is a way of life." The reason testing is so critical today is because the other business constant - change - is occurring at a pace much faster than ever before. If you can't develop the internal metabolism to test, change, and execute rapidly you are going to be left behind like the horse and buggy makers were by Ford. In the academic world they say "publish or perish," but in today's marketing world it is "test or die!"

ClickZ Live Toronto On the heels of a fantastic event in New York City, ClickZ Live is taking the fun and learning to Toronto, June 23-25. With over 15 years' experience delivering industry-leading events, ClickZ Live offers an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!

ClickZ Live San Francisco Want to learn more? Join us at ClickZ Live San Francisco, Aug 10-12!
Educating marketers for over 15 years, ClickZ Live brings together industry thought leaders from the largest brands and agencies to deliver the most advanced, educational digital marketing agenda. Register today and save $500!

ABOUT THE AUTHOR

Bryan Eisenberg

Bryan Eisenberg is co-founder and chief marketing officer (CMO) of IdealSpot. He is co-author of the Wall Street Journal, Amazon, BusinessWeek, and New York Times best-selling books Call to Action, Waiting For Your Cat to Bark?, and Always Be Testing, and Buyer Legends. Bryan is a keynote speaker and has keynoted conferences globally such as Gultaggen, Shop.org, Direct Marketing Association, MarketingSherpa, Econsultancy, Webcom, the Canadian Marketing Association, and others for the past 10 years. Bryan was named a winner of the Marketing Edge's Rising Stars Awards, recognized by eConsultancy members as one of the top 10 User Experience Gurus, selected as one of the inaugural iMedia Top 25 Marketers, and has been recognized as most influential in PPC, Social Selling, OmniChannel Retail. Bryan serves as an advisory board member of several venture capital backed companies such as Sightly, UserTesting, Monetate, ChatID, Nomi, and BazaarVoice. He works with his co-author and brother Jeffrey Eisenberg. You can find them at BryanEisenberg.com.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

Featured White Papers

Gartner Magic Quadrant for Digital Commerce

Gartner Magic Quadrant for Digital Commerce
This Magic Quadrant examines leading digital commerce platforms that enable organizations to build digital commerce sites. These commerce platforms facilitate purchasing transactions over the Web, and support the creation and continuing development of an online relationship with a consumer.

Paid Search in the Mobile Era

Paid Search in the Mobile Era
Google reports that paid search ads are currently driving 40+ million calls per month. Cost per click is increasing, paid search budgets are growing, and mobile continues to dominate. It's time to revamp old search strategies, reimagine stale best practices, and add new layers data to your analytics.

WEBINARS

Resources

Jobs

    • GREAT Campaign Project Coordinator
      GREAT Campaign Project Coordinator (British Consulate-General, New York) - New YorkThe GREAT Britain Campaign is seeking an energetic and creative...
    • Paid Search Senior Account Manager
      Paid Search Senior Account Manager (Hanapin Marketing) - BloomingtonHanapin Marketing is hiring a strategic Paid Search Senior Account Manager...
    • Paid Search Account Manager
      Paid Search Account Manager (Hanapin Marketing) - BloomingtonHanapin Marketing is hiring an experienced Paid Search Account Manager to...