User Interface Engineering

We’re big fans of the KJ Technique, a method that helps teams rank the important issues for a focus question, such as “What are the most important usability problems we need to fix in this version of the design?” or “Which user populations are most important to our business?”

In the method, teams brainstorm on potential answers to the focus question, group the answers, name each group, then vote on the group names that best answer the question. The method, in less than 45-minutes, allows teams to come to a democratic consensus on an answer, avoiding endless discussion for elements that turn out to be unimportant.

Our friend Cheryl recently decided to try the technique with her team and ran into a little confusion. Here’s what she wrote us:

After taking your “Making Sense of Usability Test and Field Study Data” webinar, I decided to try out the KJ Analysis with a team. I went t the UIE website and read the paper you published in May 2004. I ran a pilot with a group and ran into some problems.

I stumbled when it came to [the "Naming each group"] step and [the "Voting on the most important groups"] step in the technique. Folks had used different names for each of the groups of post-its. When I asked them to rank the groups, folks recorded their stars on their on group post-it — so a group may have had 9 stars, but it appeared as 3 stars on each of the three group post-its.

How do you resolve different names for groups? I’m a bit desperate — I need to figure things out by then end of the week. The “real” KJ process will occur early next week.

The problem Cheryl had is common for people trying this for the first time. The reason for these two steps is to work out the problems she encountered, so let me share some guidelines:

• When people are naming each group, they should be reading through the stickies and looking for a “theme”. The name will represent what they think the theme will be.


Naming Each Group: Group names are on the smaller orange and green stickies.

• Sometimes, groups form with multiple themes. If this is the case, you want to split the groups up. (You don’t want to discuss this — just instruct anyone who discovers this to go ahead and do it.)
• Similarly, you may discover, in the course of identifying the themes and naming the groups that more than one existing group share the same theme. In that case, you can merge them. (Again, no discussion necessary — just do it.)
• Two people, reading the same group, may see different themes and give them different names. Perfectly natural and it works out just fine. In fact, we instruct the team that they should attempt to give their own name to every group. The only reason they can skip naming a group is if someone else has “already used exactly the same words” they would’ve used to describe it. This will often encourage 2-3 names on many groups.
• When it comes time to vote for their personal “top groups” — the ones they think are most important to addressing the focus question (the focus question is the question being answered by the exercise) — it’s ok if team members put votes on different names within the same group. They should put the votes on the group name that best represents their thinking of the focus question.


Voting on Groups: Each team member chooses the group names that best address the focus question.

• When you collect up the groups to rank overall, you collect up all the group names with votes on them. Don’t worry about keeping group names from the same group together at this point. Just rank them as if they were from different groups.
• During the “Combining Like Categories” step, you’ll ask the question, “Are these two groups the same?” While they may have come from the same initial groups, they may actually mean different things. Only if you can easily substitute one for the other, do you combine their votes. So, if the group has complete consensus they mean the same thing, the fact they started with separate names will come out in the wash. If someone believes they mean something different, you’ll have an important discussion as to why they are different and then let them stand on their own in the final ranking.


Ranking Top Groups: Team combines the votes of groups names that are true synonyms.

The result of this is that group names that are true synonyms get resolved and their votes are combined. Group names that turn out to be different perspectives on the same results end up standing on their own. You’ve allowed these different perspectives to emerge with just the right amount of discussion based on their importance to the task (it doesn’t matter if they don’t get anyone’s vote) and the team is still spending the most “discussion time” talking about the most important outcomes of the exercise — not discussing the un-important edge conditions (which is where many teams get trapped).

This entry was posted on Friday, August 24th, 2007 at 9:17 am and is filed under Team Management, Usability Toolbox . You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.