Originally published: Jan 01, 1998
Developers may hesitate to start usability testing because they worry that their product poses special problems in finding, scheduling, or compensating the right users. This shouldnt stop them. We successfully find and test hundreds of users a year and about 10% of these require special tactics for scheduling. Some of these tactics include:
Here are some other things weve learned.
Even with detailed information on 800-plus users, we continually encounter new profiles we need to fill. Although our basic rules are the same for all kinds of users, some requirements need special treatment.
After 10 years of finding users, were pretty confident weve tried every method there is. One thing we know is that there are some sure-fire ways to find users and some that we never want to try again.
We've had lots of luck with the following sources:
We've also found that some sources aren't worth the time or effort. You might want to avoid the following recruitment strategies:
In early tests, we sometimes test surrogate users when the real users are difficult to find. This can help eliminate gross imperfections in a product that might irritate the real users and waste their time. Testing with surrogates is okay as long as you realize that they may bring biases that your actual audience would not have.
For example, when we needed to test ER physicians, we first used medical students. When we tested subsequently with real physicians, we found that the students had helped us refine the design, but their lack of knowledge about real life did send us down the wrong track in one or two places.
We sometimes use existing customers as surrogates. Some of the biases the bring include knowing a lot about the product or company, and having a predefined way of working. In addition, when we test existing customers, we always involve sales and marketing in the testing and contact. We also ensure that customers understand the exploratory nature of the testing, and that features they like or suggest may not turn up in the final product.
Employees of your company are at best adequate surrogates for testing your product. They can catch obvious design gaffes before real testing begins, but because they work for the company, many factors may inhibit their honesty and frankness. Also, they probably know a lot more about your target audience that your actual users do.
Weve found it invaluable to learn something about the users we want before contacting them. Knowing a little of their jargon and their time and work constraints makes it easier for us to offer something they can agree to. For example, once we learned that bond-trading desks usually close at 3:00, we scheduled the testing for 4:00 and had much more success finding willing participants.
Although specialized users may be hard to locate, weve found them easy to schedule because of their interest in the product.
We do pay users usually $50 an hour but they dont generally do it for the money. This goes double for hard-to-find users, who are generally highly paid in their work. However, sparing the user inconvenience, such as paying for parking and gas money, if it seems warranted, always helps.
On the other hand, we dont usually go out of our way to accommodate users schedules. Most take our timetable seriously and meet it or recommend someone who can.
Most qualified users want to test. However, to commit their time, busy people want (and like) to hear exactly why we want them for this testing. They also want to know what theyll get from testing this product. Among the things we tell them:
If these dont work we dont schedule the user. Users who test for other reasons, such as money, cant be counted on for the kind of participation we need for improving a product. •
Read related articles: