Guess What?!? Task Design is Critically Important! – A hard-learned lesson

Jared Spool

April 27th, 2012

Years ago, we were watching people try to find products on IKEA.com. (Not for IKEA, but for our other nefarious purposes.) We took several stabs at our study because, well, the first ones seemed fishy to us.

Our initial stab had simple tasks. One of them was “Find a bookcase for your living room that can hold 200 books.” Seems straightforward and nearly everyone completed the task easily. In fact, it seemed too easy.

It turned out that after reading the task instructions (we often write them on paper instead of delivering them orally, so the participant can refer back), every participant did exactly the same thing. They clicked in the search box on the IKEA home page and typed Bookcases. From here, it was pretty easy to get to the right product.

Yet, it was because everyone did the exact same behavior that made the UX hairs on the back of our neck tingle. That didn’t seem right.

In a second round of studies, we changed the task description to something more context-rich, avoiding any clues as to what to type in to the search engine. We ended up with:

“You’ve just moved into a new place with a living room that you love because of its expansive walls. Now, you just need someplace to put those 200 prized books that you’ve had in boxes forever. How would you do that?”

As we suspected, our participants’ behaviors changed. A couple folks typed bookcase into the search box. Others typed shelves, book shelves, book shelf, and one shelves for books. And others didn’t type anything into the search box, but clicked on the navigation links on the page, including storage systems. Most (but not all) found success using their own strategies.

What was happening was that we had guided the users with the wording of our task. By changing the wording, we saw different usability results. The design was the same, yet the results changed because of what we asked the users to do.

The wording we choose for our tasks is a critical component of the research we do, yet we hardly ever see any discussion of how to do it right or what happens when we get it wrong. If we want to make sure we’re guiding our teams in the right direction, we definitely need to nail the right research at the start.

4 Responses to “Guess What?!? Task Design is Critically Important! – A hard-learned lesson”

  1. Mike Says:

    Jared,

    Great timing on this, as I am dealing with the same issue with a test I’m currently running. Do you have any guidelines for how to approach question wording? Other than running the same test multiple times with different wording, how do you know when task failure is the fault of your IA or the way the question is worded?

    Do I smell a virtual seminar?

    Thanks,

    @MikePauley

  2. Steve G Says:

    http://www.amazon.com/Handbook-Usability-Testing-Conduct-Effective/dp/0470185481/ref=pd_sim_sbs_b_1

    I’d get the Handbook of Usability Testing by Jeffrey Rubin.

  3. Mike Says:

    Thanks for the suggestion Steve.

  4. Newman5 Says:

    Hey Spool, Mike and Steve,

    Nice post! Krug’s Rocket Surgery book where I learned about developing good tasks.

    Basically, he says to test the participants interaction with the site not their ability to read. “buy a bookcase” tests the participants ability to read and, in this case search. That’s not bad! It gets at the usability / mechanical functionality of a site. But, you don’t need to go through the trouble to recruit users to collect this type of data. Anyone hanging around the hallway will do.

    However, if you want to dig down to the other parts of the experience (findable, desirable, Morville’s honeycomb types), then a more authentic, believable, contextual task is important.

    By changing the directive command – “Find a bookcase” – to a more contextual based question – “How would you do that?” – you were able to get a different and possibly more insightful / actionable result. As a matter of fact, from now on, I’m using this format for all my user tests. Done and Done, Mr. Spool.

    @Mike – (A/B) test your (user) test? I smell recursion and an out of memory error. :)

    cheers!
    newman
    ~~~~
    Newman S. Lanier
    Builder and Co-host, http://aBetterUserExperience.com

Add a Comment