Google Website Optimizer, Part 2

  |  April 27, 2007   |  Comments

A chat with Google's Website Optimizer product manager. Last of a series.

Recently, Google announced the open beta for Google Website Optimizer, a free testing tool available to any AdWords marketer. I had the opportunity to discuss it with Tom Leung, business product manager for Google. Check out part one here.

Bryan Eisenberg: Now that people have a free tool, one of the questions they have is, "What resources will I need to actually implement tests?"

Tom Leung: Let me put it this way: if you have the ability to change an image on a particular page or change a headline, or you have the ability to add our scripts to your page, you can test. However, there are some people who may want extra help and that's why we work with certifying different partners like [your company] to help provide a consultation on what to test or how to evaluate the results. The tool's designed for do-it-yourself marketers. But for those who want extra help, that's available either through our free technical support or more advanced professional services, like our authorized consultants.

BE: One of the questions that keep popping up is if I start tagging the pages and I start making a couple of variations, how long do I have to wait until I get results that are meaningful?

TL: As with all complicated questions, it depends. It depends on a number of factors, and I'll call out some of them. One is the traffic that your page gets; another is how ambitious you were with your experimentations. So if you created dozens of different variations and then created, ultimately, hundreds of different potential versions of the page, it's going to take a lot longer than if you create two or three variations for your headline and just one or two for your hero image.

The other thing it depends on is your conversion rate. So if you define conversion as maybe entering step one of a purchasing process, you'll probably get a reasonably high conversion rate. But if you define conversion as making it through [to] step five of a five-step sales funnel, then the conversion rate would be lower. So the short of it is that our expectation is most tests will take a few weeks. We've had some sites, some of our larger advertisers, run tests and get result within as little as a few hours, although we don't recommend they end the test at that point. You want to let it run at least a week so you can make sure you don't have any weird day-of-week effects and things like that.

BE: Yeah, I can't tell you how many times we've worked with clients in the past where they'll get what looks like an immediate boost, and all of a sudden it just fizzles out and actually becomes lower. You definitely need to give enough time to run its course.

TL: We don't really encourage that kind of hit-and-run testing. Our vision is you'll always be testing and continuously trying new things. It's kind of this marathon where you're just constantly getting more efficiency out of your page.

BE: What kind of things do you see happening over the course of the next six months to a year?

TL: I'd probably say the feedback we're getting from our beta customers is that they're extremely happy with the tool, and many have said that it's just perfect for them. I don't really know much about the details of where other tools are heading, but we see the category as growing significantly and we don't really feel like there's ever going to be just one tool that's perfect for everybody. But we definitely feel like this could be used by potentially millions of Web sites out there.

In terms of where we see the tool going, we're making a lot of effort to improve and make even more sophisticated reports, to make the setup process even faster than it is today and addressing feedback we get from customers. We're very much interested in hearing what people want, and we're working hard to deliver to them. So this is just a beginning, you know. This is version 1.0. There's a lot of great stuff on the way. I don't have anything to announce in regard to future features, but I can tell you we're burning the midnight oil over here.

BE: I'd have to agree. From our perspective, the tool is just as effective as just about everyone else's. There's just a handful of companies that can benefit from a more complex algorithm or whatever it is, the amount of people who could benefit from anything extra is limited. This does what it's intended to do, which is test variations to find out which one converts best.

TL: That's right, and we're always open to ideas on how to make it better. Hopefully, we'll have another opportunity to talk with you down the road when we can announce other cool things you'll see with this tool.

BE: A lot of what we see in testing in general seems to happen more in e-commerce. Do you see more business-to-business Web sites using the Google optimizer to improve experience?

TL: You're absolutely right. The early adopters, even for our early beta, have been the e-commerce guys. I think for them there's a very clear impact on the bottom line. You know, going from a 3 to 4 percent conversion rate makes a big impact on their financial statements in the growth of their business. However, I'd argue every page on the Web should be optimized. Every page on the Web is trying to accomplish something. Why not try and experiment with alternative ways to display that page to accomplish whatever that something is? It doesn't necessarily have to be purchasing a widget from an e-commerce [site], it could be filling out a lead-generation form, or it could be watching a demo of a new brand building campaign, or it could be just staying on the Web site for a certain period of time. There are a lot of ways you can optimize a page. The straightforward e-commerce conversion is a great way for us to start, but we're already seeing a lot of beta users who aren't really selling anything but are using their Web site as a means to communicate with customers or to build awareness. Those are conversion goals, too.

BE: One last question. If you had the opportunity to talk to every single Webmaster out there who never tested before, what would you advise them to do with Google Website Optimizer in terms of first steps?

TL: I'd say pick a page that gets a significant amount of traffic and try to test a few different headlines and a few different main images, and maybe have three headlines and three images and make a conversion event something that's fairly achievable. Something like triggering the first step of a purchase process or going to the sign-up form, whatever it might be. I guarantee you, you'll be blown away by what you learned and you'll be a tester for life!

BE: From your mouth to the world. I hope lots of people hear it and take advantage of it. You guys are offering a phenomenal product, and I think we're going to see huge uptake. I think people are going to be excited, and we'll hear a lot of great stories about a lot of people making a lot more money than they are now.

TL: That's win-win for everybody.

BE: I want to thank you so much, Tom, for taking time out to tell us about Google Website Optimizer.

Meet Bryan at the ClickZ Specifics: Web Metrics seminar on May 2 at the Hilton New York in New York City.

ClickZ Live New York Want to learn more?
Attend ClickZ Live New York March 30 - April 1. With over 15 years' experience delivering industry-leading events, ClickZ Live brings together over 60 expert speakers to offer an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!


Bryan Eisenberg

Bryan Eisenberg is co-founder and chief marketing officer (CMO) of IdealSpot. He is co-author of the Wall Street Journal, Amazon, BusinessWeek, and New York Times best-selling books Call to Action, Waiting For Your Cat to Bark?, and Always Be Testing, and Buyer Legends. Bryan is a keynote speaker and has keynoted conferences globally such as Gultaggen,, Direct Marketing Association, MarketingSherpa, Econsultancy, Webcom, the Canadian Marketing Association, and others for the past 10 years. Bryan was named a winner of the Marketing Edge's Rising Stars Awards, recognized by eConsultancy members as one of the top 10 User Experience Gurus, selected as one of the inaugural iMedia Top 25 Marketers, and has been recognized as most influential in PPC, Social Selling, OmniChannel Retail. Bryan serves as an advisory board member of several venture capital backed companies such as Sightly, UserTesting, Monetate, ChatID, Nomi, and BazaarVoice. He works with his co-author and brother Jeffrey Eisenberg. You can find them at

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!




Featured White Papers

A Buyer's Guide to Affiliate Management Software

A Buyer's Guide to Affiliate Management Software
Manage your performance marketing with the right solution. Choose a platform that will mutually empower advertisers and media partners!

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.


    • Website Optimizer - SEO, CRO, Analytics
      Website Optimizer - SEO, CRO, Analytics (Marcel Digital) - ChicagoMarcel Digital, an award winning interactive marketing agency established in 2003...
    • Director of Marketing
      Director of Marketing (Patron Technology) - New YorkDirector of Marketing We are seeking a Director of Marketing to manage and build our marketing...
    • Senior Interactive Producer
      Senior Interactive Producer (Ready Set Rocket) - New YorkWhat You'll Do As a member of our team, the Senior Producer reports directly to our...