Google Website Optimizer, Part 2

  |  April 27, 2007   |  Comments

A chat with Google's Website Optimizer product manager. Last of a series.

Recently, Google announced the open beta for Google Website Optimizer, a free testing tool available to any AdWords marketer. I had the opportunity to discuss it with Tom Leung, business product manager for Google. Check out part one here.

Bryan Eisenberg: Now that people have a free tool, one of the questions they have is, "What resources will I need to actually implement tests?"

Tom Leung: Let me put it this way: if you have the ability to change an image on a particular page or change a headline, or you have the ability to add our scripts to your page, you can test. However, there are some people who may want extra help and that's why we work with certifying different partners like [your company] to help provide a consultation on what to test or how to evaluate the results. The tool's designed for do-it-yourself marketers. But for those who want extra help, that's available either through our free technical support or more advanced professional services, like our authorized consultants.

BE: One of the questions that keep popping up is if I start tagging the pages and I start making a couple of variations, how long do I have to wait until I get results that are meaningful?

TL: As with all complicated questions, it depends. It depends on a number of factors, and I'll call out some of them. One is the traffic that your page gets; another is how ambitious you were with your experimentations. So if you created dozens of different variations and then created, ultimately, hundreds of different potential versions of the page, it's going to take a lot longer than if you create two or three variations for your headline and just one or two for your hero image.

The other thing it depends on is your conversion rate. So if you define conversion as maybe entering step one of a purchasing process, you'll probably get a reasonably high conversion rate. But if you define conversion as making it through [to] step five of a five-step sales funnel, then the conversion rate would be lower. So the short of it is that our expectation is most tests will take a few weeks. We've had some sites, some of our larger advertisers, run tests and get result within as little as a few hours, although we don't recommend they end the test at that point. You want to let it run at least a week so you can make sure you don't have any weird day-of-week effects and things like that.

BE: Yeah, I can't tell you how many times we've worked with clients in the past where they'll get what looks like an immediate boost, and all of a sudden it just fizzles out and actually becomes lower. You definitely need to give enough time to run its course.

TL: We don't really encourage that kind of hit-and-run testing. Our vision is you'll always be testing and continuously trying new things. It's kind of this marathon where you're just constantly getting more efficiency out of your page.

BE: What kind of things do you see happening over the course of the next six months to a year?

TL: I'd probably say the feedback we're getting from our beta customers is that they're extremely happy with the tool, and many have said that it's just perfect for them. I don't really know much about the details of where other tools are heading, but we see the category as growing significantly and we don't really feel like there's ever going to be just one tool that's perfect for everybody. But we definitely feel like this could be used by potentially millions of Web sites out there.

In terms of where we see the tool going, we're making a lot of effort to improve and make even more sophisticated reports, to make the setup process even faster than it is today and addressing feedback we get from customers. We're very much interested in hearing what people want, and we're working hard to deliver to them. So this is just a beginning, you know. This is version 1.0. There's a lot of great stuff on the way. I don't have anything to announce in regard to future features, but I can tell you we're burning the midnight oil over here.

BE: I'd have to agree. From our perspective, the tool is just as effective as just about everyone else's. There's just a handful of companies that can benefit from a more complex algorithm or whatever it is, the amount of people who could benefit from anything extra is limited. This does what it's intended to do, which is test variations to find out which one converts best.

TL: That's right, and we're always open to ideas on how to make it better. Hopefully, we'll have another opportunity to talk with you down the road when we can announce other cool things you'll see with this tool.

BE: A lot of what we see in testing in general seems to happen more in e-commerce. Do you see more business-to-business Web sites using the Google optimizer to improve experience?

TL: You're absolutely right. The early adopters, even for our early beta, have been the e-commerce guys. I think for them there's a very clear impact on the bottom line. You know, going from a 3 to 4 percent conversion rate makes a big impact on their financial statements in the growth of their business. However, I'd argue every page on the Web should be optimized. Every page on the Web is trying to accomplish something. Why not try and experiment with alternative ways to display that page to accomplish whatever that something is? It doesn't necessarily have to be purchasing a widget from an e-commerce [site], it could be filling out a lead-generation form, or it could be watching a demo of a new brand building campaign, or it could be just staying on the Web site for a certain period of time. There are a lot of ways you can optimize a page. The straightforward e-commerce conversion is a great way for us to start, but we're already seeing a lot of beta users who aren't really selling anything but are using their Web site as a means to communicate with customers or to build awareness. Those are conversion goals, too.

BE: One last question. If you had the opportunity to talk to every single Webmaster out there who never tested before, what would you advise them to do with Google Website Optimizer in terms of first steps?

TL: I'd say pick a page that gets a significant amount of traffic and try to test a few different headlines and a few different main images, and maybe have three headlines and three images and make a conversion event something that's fairly achievable. Something like triggering the first step of a purchase process or going to the sign-up form, whatever it might be. I guarantee you, you'll be blown away by what you learned and you'll be a tester for life!

BE: From your mouth to the world. I hope lots of people hear it and take advantage of it. You guys are offering a phenomenal product, and I think we're going to see huge uptake. I think people are going to be excited, and we'll hear a lot of great stories about a lot of people making a lot more money than they are now.

TL: That's win-win for everybody.

BE: I want to thank you so much, Tom, for taking time out to tell us about Google Website Optimizer.

Meet Bryan at the ClickZ Specifics: Web Metrics seminar on May 2 at the Hilton New York in New York City.

ClickZ Live New York Want to learn more?
Attend ClickZ Live New York March 30 - April 1. With over 15 years' experience delivering industry-leading events, ClickZ Live brings together over 60 expert speakers to offer an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!


Bryan Eisenberg

Bryan Eisenberg is coauthor of the Wall Street Journal, Amazon, BusinessWeek, and New York Times bestselling books "Call to Action," "Waiting For Your Cat to Bark?," and "Always Be Testing." Bryan is a professional marketing speaker and has keynoted conferences globally such as SES,, Direct Marketing Association, MarketingSherpa, Econsultancy, Webcom, SEM Konferansen Norway, the Canadian Marketing Association, and others. In 2010, Bryan was named a winner of the Direct Marketing Educational Foundation's Rising Stars Awards, which recognizes the most talented professionals 40 years of age or younger in the field of direct/interactive marketing. He is also cofounder and chairman emeritus of the Web Analytics Association. Bryan serves as an advisory board member of SES Conference & Expo, the eMetrics Marketing Optimization Summit, and several venture capital backed companies. He works with his coauthor and brother Jeffrey Eisenberg. You can find them at

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!



Featured White Papers

A Buyer's Guide to Affiliate Management Software

A Buyer's Guide to Affiliate Management Software
Manage your performance marketing with the right solution. Choose a platform that will mutually empower advertisers and media partners!

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.



    • SEO Specialist
      SEO Specialist ( - New YorkBankrate, Inc. operates a network of personal finance related web sites for consumers.  Our sites provide...
    • Search Manager
      Search Manager (LOYAL3) - San FranciscoThe Role: We are looking for a bright, driven and personable performance-based marketer to join LOYAL3’...
    • Product Manager - Contract position
      Product Manager - Contract position (Wiley Publishing) - HobokenThis position can be located in either Hoboken, NJ or Indianapolis, IN. We are...