UIEtips: Five Techniques for Getting Buy-In for Usability Testing

Jared Spool

June 10th, 2009

Producing a usable design takes time, money, and resources. It also requires an organization’s dedication to focus on usability testing and customer needs throughout the entire design process.

Knowing how to sell usability testing will substantially help it get approved and supported by an organization. Most development teams we work with understand the benefits of usability testing, yet still struggle to communicate the value to stakeholders.

In today’s UIEtips newsletter, we look back on an article that former UIE staff member Christine Perfetti wrote in April 2007. The article, Five Techniques for Getting Buy-In for Usability Testing, discusses some of the best techniques for getting stakeholders onboard for testing. I think you’ll really enjoy it.

As always, I want to hear your thoughts on this topic. Are you challenged with selling usability testing within your organization? Is your team struggling to get support and buy-in? How have you gotten your organization onboard? Leave your thoughts and join the discussion below.

Read today’s UIEtips article.

If you find this article interesting, I highly encourage you to attend the June 17 UIE Virtual Seminar on Upgrading Your UX Team,with Sarah Bloomer. In this seminar, Sarah will touch on how to get buy in for usability testing. Use the promotion code MYARCHIVE when you register and receive life-time access to the recording of this seminar at no additional charge.

9 Responses to “UIEtips: Five Techniques for Getting Buy-In for Usability Testing”

  1. Moe Rubenzahl Says:

    Excellent!!! Coincidentally, we just did our first real usability tests using your methodology this week. Christine’s numbers 1-3 perfectly capture our experience:

    1. Start Testing Right Away

    2. Debunk the Myth that Usability Testing Is a Big Production

    3. Start Testing Early in the Process.

    Usability tests are like cleaning the garage: You know there will be great benefit, but getting started seems so daunting, and once you have done it, you wonder why you didn’t do it sooner!

    We will do this often. A reasonable test is not much more than a test subject (customers, random customer-like people within the company, etc.) in front of a PC and you, standing behind with a clipboard. Encourage the subject to tell what he is thinking and you have the essence of a usability test, UIE-style!

    I wrote about this in my own blog .

    Hell of a lot easier than cleaning the garage!

  2. Alex Says:

    We’ve just managed to convince the company to convert two meeting rooms into a user testing lab. We installed a few microphones and cameras, knocked a two way mirror in between the two rooms and we’re up and running our first session this morning.

    For a few thousand pounds of expense we now have a lab we can use 24/7 (and have a nice, homely meeting room when the lab is out of use). Previously we were booking external facilities and paying around £3,000 a day (~$6000) which we had to charge back to the business unit. Although the company knows user testing and feedback is vital, when it comes to coughing up the cash it can easily be ‘out-of-scope’. Keep on banging the drum and showing the business how-for little expense-you can produce invaluable insights

  3. Zoe Says:

    I only started doing usability testing late last year, and one objection that I got was that it wasn’t scientific testing. It wasn’t controlled, the sample size was too small, etc. My only counter-arguments were:

    1. It’s not supposed to be scientific. It’s qualitative, just seeing how users interact with the site. We’re not trying to find right and wrong answers.
    2. The usability experts say you get diminishing returns after testing more than about 5 people. You’ll see the same problems over and over at that point and not be learning much of anything new for your time.
    3. Even if only three people have problems using Feature X, isn’t that something we want to fix? We’re not going to fix it by breaking it for those who don’t have problems using it. We’ll test our solutions as well to ensure this.

    But, these counters really weren’t enough for him. Usability tests aren’t scientific, and that was that. Luckily, he wasn’t the decision maker, so I got to do my tests anyway. He hasn’t really pushed back on implementing the things we found from our tests, so I think this proves Christine’s first point: just do it — they’ll see it’s valuable and stop complaining about it. 🙂

  4. Moe Rubenzahl Says:

    re Zoe’s point: one objection that I got was that it wasn’t scientific testing. It wasn’t controlled, the sample size was too small, etc.

    In a way you’re lucky — a worse issue is when people start drawing conclusions about user preferences based on 5 testers! At least your guy recognized that the results would not be statistically usable.

    This is such a good point that it should probably be included in the original article! It’s a common problem of focus groups: Many people, especially managers, attempt to derive data from it. We can try to explain that a focus group is good for determining the right questions and not the answers, but in the end, you have to be prepared to remind people again and again that it’s qualitative research.

    I expect we will use usability testing to find the sticking points; we’ll use A/B testing on the live site to get stats and resolve questions the usability tests expose.

  5. Christine Perfetti Says:

    Zoe: You make an excellent point. We hear all the time from usability professionals who are having trouble getting buy-in for testing because the results aren’t statistically significant.

    My response is that usability testing isn’t a science. In science, to prove a hypothesis, scientists need to obtain statistical significance. With usability testing, the goal is to uncover design problems, not prove a hypothesis.

    In my experience, I’ve never been successful trying to convince others to see the benefits of testing with counter-arguments about its value. Instead, the only thing that works is to start testing, with or without buy-in. I’ve seen that once a manager or stakeholder sits in on one usability test, they start to see the value right away.

  6. Janet Says:

    A method i found useful for getting buy-in relates to Christine’s second point, ‘don’t make it a big production’. In my case, i focused on this in the reporting and recommendations. On a big, high-budget project, i made sure that i highlighted the fact that there were small changes that could make a huge difference to the user experience. The project team had been very anxious about user testing because they thought it would require massive changes that would impact their dates and budget. Giving them a prioritised list of changes, and making sure that they could get at some ‘low-hanging fruit’, meant their perception of user testing changed and they became champions of the process within the business.

    I am often challenged (usually by designers) on the value of doing user tests on black and white wire frames. Their feeling is that the colour, font, size, etc. in the creative design will address any issues that might have come up when interacting with wire frames. I have certainly observed testing where subjects repeatedly came up against issues that probably wouldn’t occur with a creative design, and it can be frustrating. But i also try to explain that basic structural issues can be ironed out using testing on wire frames, and that it’s preferable to do this before investing in creative design work. I’m wondering if anyone else has come up against this objection.

  7. ANG Says:

    I am wondering how usability testing and seo go together? I have a client who’s design is scattered with form and shop cart issues, and a design that is based on images and looks 1980s, But they only care about their seo rating… if I apply usability I have to update design, how do I show the value and get buy in for a redesign based on the user?

  8. Jeffrey Says:

    @Zoe: If you are dealing with someone who is having trouble looking at the “science” of testing, read some of Jakob Nielsen’s Alertbox articles. You can find them at http://www.useit.com/alertbox.

    I think you will find his Mar 2000 article on “Why You Only Need to Test with 5 Users” to be very helpful in making your case. The brief summary is many tests with a few users is statistically more valid and leads to better improvement. If I read his graph correctly, you can capture about 65% of usability problems with 3 users and 85% with 5 users. Adding additional users for usability testing gives diminishing returns after this point and projects are best served by correcting the discovered deficiencies and then running another round of testing.

  9. Paul Rouke Says:

    Throughout my 6 years as lead user experience designer at Littlewoods Shop Direct and for the last 12 months working with clients large and small at my user experience agency PRWD I have been very surprised to say the least at the complete lack of budget allocated to user testing. Especially with the continuing rise of money spent on recruitment through the likes of Google Adwords, if businesses took a step back and actually committed to understanding how their users are actually interacting with their site, the usability issues and barriers could be identified, site improvements made and increased conversions achieved.

    ANG in response to your dilemma, we have a client who initially approached us to take over responsibility for their search engine marketing. Taking a look at the their current site we immediately identified usability barriers which we strongly recommended fixing before we took on their Google marketing campaigns. This ensured that we could clearly show the conversion improvements that were achieved by improving the sites usability. To summarise we didn’t want to waste our clients money sending new visitors to a site which simply wasn’t going to convert.

    By having conversion data from before and after the usability issues were resolved our client witnessed for themselves the increase in conversions as they continue to get new online orders and enquiries.

    I hope this helps!

    Finally, for both SME’s and large/blue chip/multi-nationals, I have recently posted an article on the significant business benefits of user testing which may be of use to people reading this.



Add a Comment