Home  › Email › Email Marketing

Beware: Online Surveys Are Seductively Easy

  |  January 23, 2002   |  Comments

The technology may be user friendly, but the methodology required to put together a useful online survey isn't as easy as it looks.

Confession No. 1: I lied. In my prognostications for 2002, I said that doing market research via online surveys has been overlooked as a powerful email marketing application.

In fact, this method of surveying is catching on. Online surveys are seductive. If you've ever built and launched one through a Web-based interface such as InsightExpress, SurveyMonkey.com, WebSurveyor, or Zoomerang, you know how easy it is.

Compose your questions; choose bullets, drop-downs, or open-ended comment boxes for the answers; toss your logo in at the top of the page -- and presto!

You're immediately assigned a special URL for your survey. You email the link to your list (it can also be posted to your site). Within minutes, you can log into a password-protected page to see who has completed your survey. Results are displayed in graphical form so you can watch -- in real time -- as colored bar charts swell with responses. It's cool.

Here's the rub: there's a difference between doing market research and getting feedback. The two should not be confused. The former belongs in the category of "Don't try this at home." It requires a mastery of statistics, control variables, proper branching of questions, and other stuff that professional market researchers revel in.

Even the latter, getting feedback, should be approached with care. It may look easy, but that anyone can create a professional survey is simply not true. Online survey tools are undoubtedly one of the more powerful and cost-effective applications of Internet marketing. But like much of email marketing technology, it's easy to misuse them.

Confession No. 2: I'm guilty. I am not a professional market researcher, yet I've deployed several online surveys through my newsletter, WordBiz Report. Subscribers have responded to questions such as "Do you pay for subscriptions to any Web sites or e-newsletters?" (70 percent said they do not; 30 percent said they do.)

More recently I designed and launched a longer survey to find out what kind of editorial content readers valued. I got a 10 percent response rate. (Typical survey response rates are 3 to 5 percent.)

I was thrilled, but a professional researcher might question the validity of these responses. As Michael Wexler, director of research for e-Dialog, puts it: "Are there biases from frequent visitors to your site who may be more likely to respond?" Or, are new subscribers more likely to complete a survey?

Don't get me wrong. Self-serve, Web-based research definitely has a place in Internet marketing. Companies are using it to measure satisfaction with customer service. Human resource departments are using it for internal employee surveys. Professors are using it to get student feedback.

It's just that many of us have become enamored with online surveying as an easy-to-use tool, but we haven't taken the time to borrow a few best practices from classic market research on how to design and write a survey that yields valid and clear results.

Here are some tips gleaned from experience and from talking with several experts.

Do Some Offline Homework

Define the business objective of your survey first. Then consider where else you can get information about your target audience. As Alex Williams, director of marketing for CAMO, a business intelligence company, puts it: "In today's market it's quite dangerous to use one source of information or data."

Put yourself in the shoes of your audience and think broadly, he advises. Find out where they hang out online and what else they read. Then, pick up the phone and call five or six people you plan to include in the survey. Tell them what you're trying to accomplish and ask for their input.


"Because it's easy to create a survey and ask questions, it's easy to think you don't need to plan clear objectives prior to launching," says Meg Walker, director of marketing communications for WebSurveyor.

In other words, with a Web-based interface it's tempting to generate a list of questions in a one-on-one session with your computer. Don't. Convene a mini panel of colleagues. Brainstorm and write down a list of questions that prompt answers relevant to your objective.

Construct a Conversation

Now group your questions into a beginning, a middle, and an end. Start with simple, broad questions with checkbox answers. Then funnel down to specifics. "Include three or four questions that will make them think," says CAMO's Williams. (Boring demographic questions -- what you're dying to find out -- go at the end. Name, company, title, size of marketing budget, etc.)

Most important, says Williams, "think of it as a conversation. You want respondents to be driven to respond to the entire survey." That means you have to listen to each response. Each question should flow naturally from the answer to the question before it. Achieving this requires a bit of art as well as methodology. Try your survey out on several colleagues or on the mini focus group you've spoken to by phone.

When to Get Professional Assistance

"If you're trying to test a concept before launching a new product," says Zoomerang product manager Dana Meade, "you need to send your survey to the right target market and obtain a large enough sample." In other words, know when your objective moves beyond getting feedback. It's worth it to pull in a professional researcher at that point.

Where to Learn More

WebSurveyor has a good learning center with free articles and white papers on how to conduct surveys. SurveyMonkey has a good listing of several dozen online survey services. Consult Ezine-Tips.com for several recent articles on online surveys. Note that pricing and features vary widely (a number offer a free version), as does the availability of professional research assistance.

ClickZ Live Toronto On the heels of a fantastic event in New York City, ClickZ Live is taking the fun and learning to Toronto, June 23-25. With over 15 years' experience delivering industry-leading events, ClickZ Live offers an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!

ClickZ Live San Francisco Want to learn more? Join us at ClickZ Live San Francisco, Aug 10-12!
Educating marketers for over 15 years, ClickZ Live brings together industry thought leaders from the largest brands and agencies to deliver the most advanced, educational digital marketing agenda. Register today and save $500!


Debbie Weil

Debbie Weil is publisher of WordBiz Report, which focuses on the business of words online. It was awarded The Newsletter on Newsletters' Gold Award for Online Subscription Newsletter. A former newspaper reporter with an MBA and corporate marketing experience, Debbie is an expert on B2B online content and marketing at both the strategic and creative levels. She was Web content marketing manager for Network Solutions (now part of Verisign) before launching WordBiz.com.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Email newsletters delivered right to your inbox. Subscribe today!



Featured White Papers

Gartner Magic Quadrant for Digital Commerce

Gartner Magic Quadrant for Digital Commerce
This Magic Quadrant examines leading digital commerce platforms that enable organizations to build digital commerce sites. These commerce platforms facilitate purchasing transactions over the Web, and support the creation and continuing development of an online relationship with a consumer.

Paid Search in the Mobile Era

Paid Search in the Mobile Era
Google reports that paid search ads are currently driving 40+ million calls per month. Cost per click is increasing, paid search budgets are growing, and mobile continues to dominate. It's time to revamp old search strategies, reimagine stale best practices, and add new layers data to your analytics.