Nice post! Krug’s Rocket Surgery book where I learned about developing good tasks.

Basically, he says to test the participants interaction with the site not their ability to read. “buy a bookcase” tests the participants ability to read and, in this case search. That’s not bad! It gets at the usability / mechanical functionality of a site. But, you don’t need to go through the trouble to recruit users to collect this type of data. Anyone hanging around the hallway will do.

However, if you want to dig down to the other parts of the experience (findable, desirable, Morville’s honeycomb types), then a more authentic, believable, contextual task is important.

By changing the directive command – “Find a bookcase” – to a more contextual based question – “How would you do that?” – you were able to get a different and possibly more insightful / actionable result. As a matter of fact, from now on, I’m using this format for all my user tests. Done and Done, Mr. Spool.

@Mike – (A/B) test your (user) test? I smell recursion and an out of memory error.

cheers!
newman
~~~~
Newman S. Lanier
Builder and Co-host, http://aBetterUserExperience.com

I’d get the Handbook of Usability Testing by Jeffrey Rubin.

Great timing on this, as I am dealing with the same issue with a test I’m currently running. Do you have any guidelines for how to approach question wording? Other than running the same test multiple times with different wording, how do you know when task failure is the fault of your IA or the way the question is worded?

Do I smell a virtual seminar?

Thanks,

@MikePauley