Originally published: Sep 01, 1998
Usability testing isnt just for software and web sites. Testing documentation can ensure that it includes and accurately conveys all the information users expect and need.
Testing gives you accurate information on how well your documentation and Help work. It can even uncover problems that are better solved by changing the interface.
As with most parts of the development, its easier and less expensive to find and correct doc problems early in the process, so we try to test as soon as possible.
Choosing a method depends on what you want to learn, how much time you have, and where your greatest risks are.
This technique lets you discover what information users actually need, before youve written anything.
We bring in test participants as soon as we have a version of the product that will let users do some of the tasks theyd do in the real world. This may be well before weve written the documentation, but this isnt a problem because a member of the development team simulates the documentation.
The incredibly intelligent part comes in when users get stuck or seem confused. At this point, the person simulating the docs asks, "What is your question right now?" When the user replies, the Help person provides a verbal clue, but gives as small a piece of information as possible. If this isnt enough, the user can ask another question.
By giving information only a bit at a time, we can find out exactly which piece is most important to the user. Knowing this helps teams provide appropriate documentation not too much, not too little or, better yet, to fix the interface itself.
You can determine whether users can get to the right topic in your index by having them work with the draft index. You can even test the index before youve written the documentation.
To test the index, we show users screen shots from the product and ask them to write down the first three terms theyd look up to find more information. This test lets you look inside users heads and learn some of the terms they use and expect to see. We write down all the terms they use; they could be words weve omitted or not referenced. The more of these we include, the better the index will work.
By testing how well users comprehend conceptual material, youll know how successfully youre communicating the information.
We ask users to read a section or examine a picture in the docs, then ask them to describe the three main points this element is trying to convey. (You should already know what you expect them to be!) If the users answers match yours, youre probably getting your information across; if not, you may have some work to do.
This is a good way to test how accurately youve written the procedural details of your docs.
We give users the documentation or Help, asking them to read a specific page or topic and then work with the software to complete the procedure this section describes.
We note where they have problems, where they get confused, or where steps are missing. Then we make the necessary changes in the next draft.
Lookup tests demonstrate if the docs or Help include the necessary information and if users can get to the right topic and use it. They also let you find the users biggest unanswered question.
To do a lookup test, we give users questions to answer or tasks to complete and see if the draft documentation helps them succeed. For example, when testing the documentation for a video product, we told users, "The picture youre getting isnt as clear as youd like. Use the documentation and online help to find out whats wrong and how to fix it."
After conducting lookup tests, some teams have changed from printed to online help (or vice versa), added index entries, or reorganized the information.
It can be enlightening and fascinating to discover what experienced users dont know about a product. To do this, weve recently started experimenting with the Rose in the Thorns test.
We ask experienced users to look through the documentation or Help until they find information about features they didnt know before.
Then we ask them if this new information is valuable to them. If there are important features users dont know about, we work with the interface designers to communicate them better.
Three things help ensure successful tests of docs and help:
Read related articles: