One-Day Usability Testing
Michael Pastore | October 5, 2000
Usability testing can start at $30,000 and takes at least a few weeks, if not longer. Not an option for companies trying to rush a product to launch on a limited budget. Luckily, 80 percent of what you would learn from these studies can be gained in less than a day. Rule of thumb: You can get solid feedback on a user interface after observing 5 to 10 potential customers trying to use it. You can test more, but additional testing seldom raises significant issues not identified by the first 10 customers.
If outsourced, usability testing usually starts at approximately $30,000 and takes at least a few weeks, if not longer. Not an option for most companies trying to rush a product to launch on a limited budget. Luckily, 80 percent of what you would learn from these studies can be gained in less than a day.
The rule of thumb, which I have experienced to be true, is that you can usually get solid feedback on a user interface after observing 5 to 10 potential customers trying to use it. If you have time to test with more people, it will help you feel more comfortable with the results of your testing. However, additional testing seldom raises significant issues that the first 10 people did not identify.
So, where do you find the guinea pigs? There are four sources of people for testing that can usually be tapped on short notice and who are willing to spend an hour playing with your product in exchange for pizza.
- New hires: The last 10 people you hired probably know about as much about your company as a random person picked off the street. Ask your HR department if you can add usability testing as part of the orientation process under the guise that you are helping new hires learn about the products. If you are looking for feedback on existing products, you might even be able to get some feedback from candidates during the interview process.
- Neighborhood companies: If you are in an area with a lot of Internet or technology companies, it should be easy to find a few peers among your neighbors who would be more than willing to do a testing trade. They spend an hour with your product in exchange for an hour in the future when they need to get some testing done. This is also a great way to build awareness about your products and your company and to attract employees (not that I would recommend stealing employees from your neighbors).
- Existing customers: Oddly enough, a lot of customers enjoy being involved in the development process. In exchange for a T-shirt or pizza, they will gladly visit your office and spend an hour with you. Why don't you give a real-world example here to add credibility?
- Friends and family: Although she may not be in your demographic, your grandmother may be the best resource for usability testing. If you can build an interface that makes sense to someone with minimal Internet experience, you will probably appeal to 99 percent of the U.S. population. Note that this can backfire if you have to spend the first hour explaining how to use a mouse.
Running the Tests
Given that you have already spent time and put in the effort to get the subjects and schedule a conference room, you might as well invest an extra hour or two to make sure that you are prepared. Avoid doing improvisational testing (that is, sitting down with a subject and winging it as you go). Here are a few tips on how to prepare for your tests.
- Use scenarios: As part of the product-development process, you should have already identified how you expect customers to use your new product. Ask your subjects to attempt to do a few of these tasks. Have them try to buy a product from your catalog, register for your new service, or change their payment method within their account.
- Have customers verbalize their thoughts: You can learn a lot from simply watching someone use your product. However, the results of your testing are significantly enhanced if you ask the subjects to verbalize their thoughts as they navigate. This is a good way to avoid making assumptions about why a subject did not go down the path that you intended. You might also catch some random comments that change your thinking about a feature or design.
- Keep yourself honest: Try to always have more than one member of your product team involved in the testing. This decreases the likelihood that you will misread the actions or comments of the subject. This is especially important if you are invested in having the results turn out a certain way - you might be under a tight deadline and not want to see that your product needs major changes or have your ego tied up in a feature or page layout. Another approach would be to videotape the test and invite your entire product team to watch and comment.
- Test the total product: If you spent the time to catch your subjects, milk it for all it is worth. Try to get feedback on more than just the features, placement of buttons, and page flow. Ask about the impression they got from the marketing pages, their expectations for email communication after they use the product, their reaction to different pricing strategies, etc.