Assessing Weights for Users’ Needs, Part 1

Jared Spool

July 6th, 2008

When I presented my suggestion for how to compare multiple design alternatives, one step involved creating what I called a Weighted Differences Matrix. In the matrix, each row represents a difference between the design alternative, which, in turn, we interpret to be a factor to help us decide which design is better. How do we know what the list of differences should be?

An Example Weighted Differences Matrix -- Where do the differences come from?

The process we’d use is to compare the designs side-by-side and list the differences. A method I’m fond of is to do the comparisons with two sites at a time. In this case, we’d probably start with the current design and alternative #1. We’d put them side-by-side and ask, “What makes these designs different?” Once we’d exhausted our thinking, we’d replace alternative #1 with alternative #2 and repeat the questioning, looking for new differences to add to the list.

To be complete, after comparing all the alternatives to the current design, we might try comparing them to each other. However, we can probably just eyeball the different designs to catch the last few differences that we may have missed on the first pass of comparisons. In my experience, 95% of the differences are discovered on the first pass. For most projects, that’s good enough.

Given five alternatives, we’d allocate an hour to generate this list.

Tomorrow, we’ll talk about how we would assess the weights for each difference.

10 Responses to “Assessing Weights for Users’ Needs, Part 1”

  1. Assessing Weights for Users’ Needs, Part 2 » UIE Brain Sparks Says:

    [...] I talked about how we come up with each row in our Weighted Differences Matrix. For each of these differences, we need to then assess how important it will be to the user’s [...]

  2. Chris Says:

    Great idea!! Bridging the subjective into the objective is huge when trying to have usability discussions with clients around IA/UXD decision making/rationale. Now, if I can only find the time to develop an exhaustive list of differences for the matrix and then jam it into our design process… Thanks for the insight!

    Curious to know how many times you’ve presented this to clients? What has the response been like?

  3. Jared Spool Says:

    Hi Chris,

    I’m not sure what you mean by “presented this to clients”?

    We’ve used weighted matricies of various sorts — it’s a standard biz decision support tool — at clients hundreds of times. We’ve used this approach to describe differences between alternatives a couple of dozen times over the 20 years we’ve been in business.

    We’ve also taught this technique to teams, so they could do it themselves.

    In all the cases, the response has been very positive. It’s time intensive, so you have to set that expectation up front. But, the level of discourse and the value from the process makes itself visible very quickly.

    Thanks for your question.

  4. Stefan Wobben Says:

    How do you set the weight score? do you base this of observations, expert knowledge or selfreports of users?
    Have you ever thought of using the ‘opportunity algorithm’ (Ulwick, A.W. (Jan 2002). “Turn Customer Input into Innovation”. Harvard Business Review.) to calculate your scores?

  5. nortypig » Blog Archive » Comparing Design Alternatives Says:

    [...] In each row of the matrix a number represents the difference between the design alternative. In Assessing Weights for Users’ Needs Part 1 and Part 2 he provides more information on how to develop a weighted differences [...]

  6. Pleasure and Pain » Links from 7/24/2008 to 7/31/2008 Says:

    [...] Assessing Weights for Users’ Needs, Part 1 » UIE Brain SparksWeighted Differences Matrix quantifies the differences between multiple design alternatives [...]

  7. Pleasure and Pain » Links from 8/1/2008 to 8/3/2008 Says:

    [...] Assessing Weights for Users’ Needs, Part 1 » UIE Brain SparksWeighted Differences Matrix quantifies the differences between multiple design alternatives [...]

  8. Pleasure and Pain » Links from 8/4/2008 to 8/5/2008 Says:

    [...] Assessing Weights for Users’ Needs, Part 1 » UIE Brain SparksWeighted Differences Matrix quantifies the differences between multiple design alternatives [...]

  9. Dan's E-Marketing Guide >> Online Marketing Strategy >> e-marketing strategy Says:

    [...] Example Activities User centered design Keeping things simple Development and Release plans Design and development reviews Planning content [...]

  10. Webdesign Says:

    Great comparisons! But I also like to know how to set the weight score?

    Good article,
    Mark

Add a Comment