Home  › Email › Email Marketing

Beware: Online Surveys Are Seductively Easy

  |  January 23, 2002   |  Comments

The technology may be user friendly, but the methodology required to put together a useful online survey isn't as easy as it looks.

Confession No. 1: I lied. In my prognostications for 2002, I said that doing market research via online surveys has been overlooked as a powerful email marketing application.

In fact, this method of surveying is catching on. Online surveys are seductive. If you've ever built and launched one through a Web-based interface such as InsightExpress, SurveyMonkey.com, WebSurveyor, or Zoomerang, you know how easy it is.

Compose your questions; choose bullets, drop-downs, or open-ended comment boxes for the answers; toss your logo in at the top of the page -- and presto!

You're immediately assigned a special URL for your survey. You email the link to your list (it can also be posted to your site). Within minutes, you can log into a password-protected page to see who has completed your survey. Results are displayed in graphical form so you can watch -- in real time -- as colored bar charts swell with responses. It's cool.

Here's the rub: there's a difference between doing market research and getting feedback. The two should not be confused. The former belongs in the category of "Don't try this at home." It requires a mastery of statistics, control variables, proper branching of questions, and other stuff that professional market researchers revel in.

Even the latter, getting feedback, should be approached with care. It may look easy, but that anyone can create a professional survey is simply not true. Online survey tools are undoubtedly one of the more powerful and cost-effective applications of Internet marketing. But like much of email marketing technology, it's easy to misuse them.

Confession No. 2: I'm guilty. I am not a professional market researcher, yet I've deployed several online surveys through my newsletter, WordBiz Report. Subscribers have responded to questions such as "Do you pay for subscriptions to any Web sites or e-newsletters?" (70 percent said they do not; 30 percent said they do.)

More recently I designed and launched a longer survey to find out what kind of editorial content readers valued. I got a 10 percent response rate. (Typical survey response rates are 3 to 5 percent.)

I was thrilled, but a professional researcher might question the validity of these responses. As Michael Wexler, director of research for e-Dialog, puts it: "Are there biases from frequent visitors to your site who may be more likely to respond?" Or, are new subscribers more likely to complete a survey?

Don't get me wrong. Self-serve, Web-based research definitely has a place in Internet marketing. Companies are using it to measure satisfaction with customer service. Human resource departments are using it for internal employee surveys. Professors are using it to get student feedback.

It's just that many of us have become enamored with online surveying as an easy-to-use tool, but we haven't taken the time to borrow a few best practices from classic market research on how to design and write a survey that yields valid and clear results.

Here are some tips gleaned from experience and from talking with several experts.

Do Some Offline Homework

Define the business objective of your survey first. Then consider where else you can get information about your target audience. As Alex Williams, director of marketing for CAMO, a business intelligence company, puts it: "In today's market it's quite dangerous to use one source of information or data."

Put yourself in the shoes of your audience and think broadly, he advises. Find out where they hang out online and what else they read. Then, pick up the phone and call five or six people you plan to include in the survey. Tell them what you're trying to accomplish and ask for their input.


"Because it's easy to create a survey and ask questions, it's easy to think you don't need to plan clear objectives prior to launching," says Meg Walker, director of marketing communications for WebSurveyor.

In other words, with a Web-based interface it's tempting to generate a list of questions in a one-on-one session with your computer. Don't. Convene a mini panel of colleagues. Brainstorm and write down a list of questions that prompt answers relevant to your objective.

Construct a Conversation

Now group your questions into a beginning, a middle, and an end. Start with simple, broad questions with checkbox answers. Then funnel down to specifics. "Include three or four questions that will make them think," says CAMO's Williams. (Boring demographic questions -- what you're dying to find out -- go at the end. Name, company, title, size of marketing budget, etc.)

Most important, says Williams, "think of it as a conversation. You want respondents to be driven to respond to the entire survey." That means you have to listen to each response. Each question should flow naturally from the answer to the question before it. Achieving this requires a bit of art as well as methodology. Try your survey out on several colleagues or on the mini focus group you've spoken to by phone.

When to Get Professional Assistance

"If you're trying to test a concept before launching a new product," says Zoomerang product manager Dana Meade, "you need to send your survey to the right target market and obtain a large enough sample." In other words, know when your objective moves beyond getting feedback. It's worth it to pull in a professional researcher at that point.

Where to Learn More

WebSurveyor has a good learning center with free articles and white papers on how to conduct surveys. SurveyMonkey has a good listing of several dozen online survey services. Consult Ezine-Tips.com for several recent articles on online surveys. Note that pricing and features vary widely (a number offer a free version), as does the availability of professional research assistance.

ClickZ Live Chicago Join the Industry's Leading eCommerce & Direct Marketing Experts in Chicago
ClickZ Live Chicago (Nov 3-6) will deliver over 50 sessions across 4 days and 10 individual tracks, including Data-Driven Marketing, Social, Mobile, Display, Search and Email. Check out the full agenda and register by Friday, Oct 3 to take advantage of Early Bird Rates!


Debbie Weil

Debbie Weil is publisher of WordBiz Report, which focuses on the business of words online. It was awarded The Newsletter on Newsletters' Gold Award for Online Subscription Newsletter. A former newspaper reporter with an MBA and corporate marketing experience, Debbie is an expert on B2B online content and marketing at both the strategic and creative levels. She was Web content marketing manager for Network Solutions (now part of Verisign) before launching WordBiz.com.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Email newsletters delivered right to your inbox. Subscribe today!



Featured White Papers

IBM: Social Analytics - The Science Behind Social Media Marketing

IBM Social Analytics: The Science Behind Social Media Marketing
80% of internet users say they prefer to connect with brands via Facebook. 65% of social media users say they use it to learn more about brands, products and services. Learn about how to find more about customers' attitudes, preferences and buying habits from what they say on social media channels.

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising
If you're considering implementing a marketing attribution model to measure and optimize your programs, this paper is a great introduction. It also includes real-life tips from marketers who have successfully implemented attribution in their organizations.



    • Web Writer
      Web Writer (Money Map Press) - BaltimoreDo you have a passion for the markets and investing, and writing? Do you want to spend your days providing...
    • Web Production Specialist
      Web Production Specialist (Money Map Press) - BaltimoreMoney Map Press is looking for a self-starter to perform and oversee the production of daily...
    • Internet Marketing Campaign Manager
      Internet Marketing Campaign Manager (Straight North, LLC) - Downers GroveWe are looking for a talented Internet Marketing Campaign Manager...