Beware: Online Surveys Are Seductively Easy

Confession No. 1: I lied. In my prognostications for 2002, I said that doing market research via online surveys has been overlooked as a powerful email marketing application.

In fact, this method of surveying is catching on. Online surveys are seductive. If you’ve ever built and launched one through a Web-based interface such as InsightExpress,, WebSurveyor, or Zoomerang, you know how easy it is.

Compose your questions; choose bullets, drop-downs, or open-ended comment boxes for the answers; toss your logo in at the top of the page — and presto!

You’re immediately assigned a special URL for your survey. You email the link to your list (it can also be posted to your site). Within minutes, you can log into a password-protected page to see who has completed your survey. Results are displayed in graphical form so you can watch — in real time — as colored bar charts swell with responses. It’s cool.

Here’s the rub: there’s a difference between doing market research and getting feedback. The two should not be confused. The former belongs in the category of “Don’t try this at home.” It requires a mastery of statistics, control variables, proper branching of questions, and other stuff that professional market researchers revel in.

Even the latter, getting feedback, should be approached with care. It may look easy, but that anyone can create a professional survey is simply not true. Online survey tools are undoubtedly one of the more powerful and cost-effective applications of Internet marketing. But like much of email marketing technology, it’s easy to misuse them.

Confession No. 2: I’m guilty. I am not a professional market researcher, yet I’ve deployed several online surveys through my newsletter, WordBiz Report. Subscribers have responded to questions such as “Do you pay for subscriptions to any Web sites or e-newsletters?” (70 percent said they do not; 30 percent said they do.)

More recently I designed and launched a longer survey to find out what kind of editorial content readers valued. I got a 10 percent response rate. (Typical survey response rates are 3 to 5 percent.)

I was thrilled, but a professional researcher might question the validity of these responses. As Michael Wexler, director of research for e-Dialog, puts it: “Are there biases from frequent visitors to your site who may be more likely to respond?” Or, are new subscribers more likely to complete a survey?

Don’t get me wrong. Self-serve, Web-based research definitely has a place in Internet marketing. Companies are using it to measure satisfaction with customer service. Human resource departments are using it for internal employee surveys. Professors are using it to get student feedback.

It’s just that many of us have become enamored with online surveying as an easy-to-use tool, but we haven’t taken the time to borrow a few best practices from classic market research on how to design and write a survey that yields valid and clear results.

Here are some tips gleaned from experience and from talking with several experts.

Do Some Offline Homework

Define the business objective of your survey first. Then consider where else you can get information about your target audience. As Alex Williams, director of marketing for CAMO, a business intelligence company, puts it: “In today’s market it’s quite dangerous to use one source of information or data.”

Put yourself in the shoes of your audience and think broadly, he advises. Find out where they hang out online and what else they read. Then, pick up the phone and call five or six people you plan to include in the survey. Tell them what you’re trying to accomplish and ask for their input.


“Because it’s easy to create a survey and ask questions, it’s easy to think you don’t need to plan clear objectives prior to launching,” says Meg Walker, director of marketing communications for WebSurveyor.

In other words, with a Web-based interface it’s tempting to generate a list of questions in a one-on-one session with your computer. Don’t. Convene a mini panel of colleagues. Brainstorm and write down a list of questions that prompt answers relevant to your objective.

Construct a Conversation

Now group your questions into a beginning, a middle, and an end. Start with simple, broad questions with checkbox answers. Then funnel down to specifics. “Include three or four questions that will make them think,” says CAMO’s Williams. (Boring demographic questions — what you’re dying to find out — go at the end. Name, company, title, size of marketing budget, etc.)

Most important, says Williams, “think of it as a conversation. You want respondents to be driven to respond to the entire survey.” That means you have to listen to each response. Each question should flow naturally from the answer to the question before it. Achieving this requires a bit of art as well as methodology. Try your survey out on several colleagues or on the mini focus group you’ve spoken to by phone.

When to Get Professional Assistance

“If you’re trying to test a concept before launching a new product,” says Zoomerang product manager Dana Meade, “you need to send your survey to the right target market and obtain a large enough sample.” In other words, know when your objective moves beyond getting feedback. It’s worth it to pull in a professional researcher at that point.

Where to Learn More

WebSurveyor has a good learning center with free articles and white papers on how to conduct surveys. SurveyMonkey has a good listing of several dozen online survey services. Consult for several recent articles on online surveys. Note that pricing and features vary widely (a number offer a free version), as does the availability of professional research assistance.

Related reading

Vector illustration with a magnifying glass focusing on a pie chart, a graph line trending upwards, and other metrics symbols.
Signpost with two signs pointing in different directions, one labelled B2B and one labelled B2C.