Users, Usability, Usefulness

My partner columnist Melaney Smith wrote a great column on usability studies last year. She used a story about purchasing a new TV set for her grandmother to relate the importance of usability testing. Her conclusion was that in addition to helping design a consumer-friendly Web site, usability studies can play an integral role in understanding your customer base.

Last week, I was speaking with another ClickZ columnist, Jeanne Jennings, when the topic of usability studies arose again. Jeanne is currently working on a couple of projects for clients who publish directories. Admittedly, the directory business isn’t exotic, but anyone who knows anything about the business knows directories can be highly profitable. Salespeople rely on directories as a source of leads. They therefore are willing to pay hundreds, and sometimes thousands, of dollars for the valuable information contained in the directory.

A few years ago, one of Jeanne’s clients decided to jump into the 21st century and publish the directory on CD-ROM. Shifting the information to an electronic format was supposed to add value and functionality for customers and increase the publisher’s margins. Mass-producing a CD is cheaper than mass-producing a print version. Sales were good, but not overwhelming.

This year, the client decided to make the jump to full interactivity and publish the directory online. The plan calls for customers to pay an annual fee to access the directory data on a Web site. Benefits include more frequent updates and online access — great for sale folks who travel on a regular basis. Based on the success of the CD project, the client was eager to build the Web directory and start collecting annual subscription revenues.

Jeanne’s primary objectives for the Web-based directory were:

• Ensure the new online user experience maintained the benefits of the offline counterpart. For years, users were happy with the offline directory versions. She wanted to ensure users had a familiar online experience.
• Add new functionality made possible by the Internet to enhance user experience and increase the product’s value.

As a starting point, Jeanne asked to see results of any historical usability studies the client had completed. She learned no usability testing on either the print or CD-ROM directories had ever been conducted. As Melaney Smith points out in her usability column, it’s not uncommon for companies to skip usability studies. They can seem expensive, especially in small, entrepreneurial organizations where senior executives assume they already know how consumers are using their products or services.

Although the client resisted a usability study, they did consent to a research project. They interviewed customers about their experiences with the offline directories. The project’s goals were to determine how customers used the existing print and CD directories, identify things they liked and disliked, and get feedback on additional functionality/content planned for the online version. The results surprised everyone.

CD Usability

Although the CD project was a success for the client, the research revealed users were less than enthralled with the product. A sample of user comments:

• “It has great potential, but the CD was poorly executed. The interface is really clunky.”
• “I like that I can search on a variety of criteria — budget size, job titles, etc. — but the search results don’t reflect what I’ve searched. For example, I can search for a specific job title, but the search returns just a list of the entries which have that title in them. I have to click on each individual entry and scroll through it to find the job title and contact information I’m looking for. It would be so much easier if the results gave me the data I needed (contact information for the job title) rather than just a link to the listing.”
• “It’s a halfway solution. While the searchability is nice, we still need to print it off and have someone rekey the data into our ACT! database.”

Prior to Jeanne’s involvement, the client wanted the online experience to mimic the CD experience. As the research and customer comments revealed, that would have been a huge mistake. As a result, the client now has key insight into how to develop an online version that will better meet customers’ needs.

Data Usage

The client assumed most customers were using the directories to print labels for direct mail campaigns. They were concerned this would cannibalize revenue from their list rentals and wanted to employ measures to either limit the number of labels that could be produced from the online product or create a way to charge for label functionality.

Research showed those concerns to be unfounded. Most customers Jeanne spoke with used the directories to build telephone contact rather than direct mail lists. Only one customer used the directory to create mailing labels. Most didn’t view direct mail as an effective marketing vehicle. If their marketing departments did direct mail, they rented the lists rather than fuss with extracting data from the directories.

Prior to Jeanne’s involvement, the client was trying to figure out how to design an interface and price structure for mailing labels that would not cannibalize their list rental revenue. The interface would have been expensive to build and relatively unused. The client chose instead to shift design and pricing structure efforts into building an interface that allows users to download defined information into a variety of contact or CRM software (e.g., ACT!, Siebel).

Pricing Structure and Strategy

The company envisioned offering online customers regular updates and charging a premium for the value-add service. Logic was customers would pay more to have the most up-to-date information at their fingertips. Again, the results were surprising.

Many customers said annual updates were frequent enough. If contact information is outdated, it’s easy enough to find the right individual through a receptionist.

One customer Jeanne met with had the 1989-90 directory and still used it on a regular basis (although another individual in the organization did have a more recent CD version). For this individual, static data (physical location, main phone number, etc.) have more value than the information that changes. He doesn’t feel the need to update his directory each year. He has highlights and notations in his print copy, which would be lost if he got a newer edition. So more frequent updates, while desired, did not hold as much benefit for the customers as initially believed.

The client is currently reconsidering price structure and strategy based on this new information.

With new knowledge gleaned from this and other research projects, Jeanne helps clients build Web sites customers will find familiar and even more useful than print or CD versions of directories. Her clients now recognize the benefit of usability research to help them produce products customers find useful and valuable.