Recently, Google announced the open beta for Google Website Optimizer, a free testing tool available to any AdWords marketer. I had the opportunity to discuss it with Tom Leung, business product manager for Google. Check out part one here.
Bryan Eisenberg: Now that people have a free tool, one of the questions they have is, “What resources will I need to actually implement tests?”
Tom Leung: Let me put it this way: if you have the ability to change an image on a particular page or change a headline, or you have the ability to add our scripts to your page, you can test. However, there are some people who may want extra help and that’s why we work with certifying different partners like [your company] to help provide a consultation on what to test or how to evaluate the results. The tool’s designed for do-it-yourself marketers. But for those who want extra help, that’s available either through our free technical support or more advanced professional services, like our authorized consultants.
BE: One of the questions that keep popping up is if I start tagging the pages and I start making a couple of variations, how long do I have to wait until I get results that are meaningful?
TL: As with all complicated questions, it depends. It depends on a number of factors, and I’ll call out some of them. One is the traffic that your page gets; another is how ambitious you were with your experimentations. So if you created dozens of different variations and then created, ultimately, hundreds of different potential versions of the page, it’s going to take a lot longer than if you create two or three variations for your headline and just one or two for your hero image.
The other thing it depends on is your conversion rate. So if you define conversion as maybe entering step one of a purchasing process, you’ll probably get a reasonably high conversion rate. But if you define conversion as making it through [to] step five of a five-step sales funnel, then the conversion rate would be lower. So the short of it is that our expectation is most tests will take a few weeks. We’ve had some sites, some of our larger advertisers, run tests and get result within as little as a few hours, although we don’t recommend they end the test at that point. You want to let it run at least a week so you can make sure you don’t have any weird day-of-week effects and things like that.
BE: Yeah, I can’t tell you how many times we’ve worked with clients in the past where they’ll get what looks like an immediate boost, and all of a sudden it just fizzles out and actually becomes lower. You definitely need to give enough time to run its course.
TL: We don’t really encourage that kind of hit-and-run testing. Our vision is you’ll always be testing and continuously trying new things. It’s kind of this marathon where you’re just constantly getting more efficiency out of your page.
BE: What kind of things do you see happening over the course of the next six months to a year?
TL: I’d probably say the feedback we’re getting from our beta customers is that they’re extremely happy with the tool, and many have said that it’s just perfect for them. I don’t really know much about the details of where other tools are heading, but we see the category as growing significantly and we don’t really feel like there’s ever going to be just one tool that’s perfect for everybody. But we definitely feel like this could be used by potentially millions of Web sites out there.
In terms of where we see the tool going, we’re making a lot of effort to improve and make even more sophisticated reports, to make the setup process even faster than it is today and addressing feedback we get from customers. We’re very much interested in hearing what people want, and we’re working hard to deliver to them. So this is just a beginning, you know. This is version 1.0. There’s a lot of great stuff on the way. I don’t have anything to announce in regard to future features, but I can tell you we’re burning the midnight oil over here.
BE: I’d have to agree. From our perspective, the tool is just as effective as just about everyone else’s. There’s just a handful of companies that can benefit from a more complex algorithm or whatever it is, the amount of people who could benefit from anything extra is limited. This does what it’s intended to do, which is test variations to find out which one converts best.
TL: That’s right, and we’re always open to ideas on how to make it better. Hopefully, we’ll have another opportunity to talk with you down the road when we can announce other cool things you’ll see with this tool.
BE: A lot of what we see in testing in general seems to happen more in e-commerce. Do you see more business-to-business Web sites using the Google optimizer to improve experience?
TL: You’re absolutely right. The early adopters, even for our early beta, have been the e-commerce guys. I think for them there’s a very clear impact on the bottom line. You know, going from a 3 to 4 percent conversion rate makes a big impact on their financial statements in the growth of their business. However, I’d argue every page on the Web should be optimized. Every page on the Web is trying to accomplish something. Why not try and experiment with alternative ways to display that page to accomplish whatever that something is? It doesn’t necessarily have to be purchasing a widget from an e-commerce [site], it could be filling out a lead-generation form, or it could be watching a demo of a new brand building campaign, or it could be just staying on the Web site for a certain period of time. There are a lot of ways you can optimize a page. The straightforward e-commerce conversion is a great way for us to start, but we’re already seeing a lot of beta users who aren’t really selling anything but are using their Web site as a means to communicate with customers or to build awareness. Those are conversion goals, too.
BE: One last question. If you had the opportunity to talk to every single Webmaster out there who never tested before, what would you advise them to do with Google Website Optimizer in terms of first steps?
TL: I’d say pick a page that gets a significant amount of traffic and try to test a few different headlines and a few different main images, and maybe have three headlines and three images and make a conversion event something that’s fairly achievable. Something like triggering the first step of a purchase process or going to the sign-up form, whatever it might be. I guarantee you, you’ll be blown away by what you learned and you’ll be a tester for life!
BE: From your mouth to the world. I hope lots of people hear it and take advantage of it. You guys are offering a phenomenal product, and I think we’re going to see huge uptake. I think people are going to be excited, and we’ll hear a lot of great stories about a lot of people making a lot more money than they are now.
TL: That’s win-win for everybody.
BE: I want to thank you so much, Tom, for taking time out to tell us about Google Website Optimizer.
Meet Bryan at the ClickZ Specifics: Web Metrics seminar on May 2 at the Hilton New York in New York City.
Advertising to millennials can be challenging, especially when there’s a lack of understanding towards their needs. Here’s what you need to consider before targeting ... read more
As both a Googler and ClickZ team member, I recently attended and participated in the always-inspirational ClickZ Live New York event. Along ... read more