What is a good success rate for a usability test task?
We just conducted user testing on a site map. So we have success rate percentages for each task. They range from 9% – 51% success (in up to 3 tries). Obviously there are problems. (And no, we didn’t create the site map, which makes me feel good.) But what would be considered a “good” success rate? I want to say over 70% for this test. It is only site map, no content, which will limit the success anyway. Maybe I’m aiming too high?

Thinking in terms of % of completion may not be the right approach. (In fact, I’m hard pressed to come up with a time when it is the right approach.)

You haven’t said anything about who the users are or what the site map information contains. But let’s pretend the users are doctors and nurses and the site map contains the necessary information for them to administer drugs safely. If one of those doctors or nurses doesn’t find the information they need, they could improperly administer a treatment which could kill their patient. What would be an acceptable failure rate under these conditions? I’d say 0% — the system needs to ensure success of every user.

Why is your system any less important? Why would you be willing to tolerate any failures?

The real question isn’t “what is an acceptable level of failures?” The question I think you want is “What’s preventing people from succeeding?”

Instead of looking at how many people succeed versus how many fail, what if you were to analyze the failures themselves. Can you rank and categorize all the things that prevent your users from succeeding? Can you assign a classification that helps you understand whether the problems are life and death (as in the example of doctors and nurses I used above), problems that will lose customers, problems that will cost support money, and problems that are annoying without painful side effects?

This will also help you look at the participants you’re recruiting for your study. How similar are they to real users? How realistic are the tasks you’re asking them to complete? How well does the system, if they make a mistake at the site map, help them still succeed by having guidance for common errors on the content pages themselves? (Such as “If you’re looking for x, click here.” type lateral navigation.)

In the end, you really want to understand the problems real users will encounter. That’s the purpose for the studies. Then you want to explore solutions that resolve those problems. In an ideal world, it’s not that you get 100% task completion, it’s that you have addressed and solved all the problems.

The closer you can get your studies to map true in-the-wild user behavior, the more you’ll understand about the problems you’re uncovering and the solutions that will help. Focus on the problems and their resolution and you’ll get the design to where you’d like it to be.

How we present our information is critical, yet it’s something we haven’t discussed as frequently as we should. Often we’re trying to fit our information into formats that don’t work effectively. All too often we let the underlying technology, such as our content management system, dictate how we present our information to the user. Yet we have control over these things and we should use it.

In this week’s UIEtips, we’ve turned to our favorite information architect, Donna Spencer. She’s written a great article on the different schemes for presenting information, such as when it’s instructional, geographical, or time-based. You’re sure to find great inspiration in her classifications—an idea so perfect I’m envious I didn’t come up with it first.

Read the article Classification Schemes—and When to Use Them

If you find Donna’s article as interesting as I did, then you’ll want to join me for her UIE Virtual Seminar on 9/16. She’ll dive deep into this topic, showing exactly how to create the most effective pages based on the underlying nature of the information. This seminar will likely be one of the best of the year, so don’t miss it. Get more information on Donna’s webinar, Organization Schemes.

Have you come up with your classifications for your pages? What’s worked? What challenges have you encountered? We’d love to hear your thoughts below.

Listening to her, I got this idea about links like “Products”, which we see on a lot of corporate sites. Vanessa was talking about these words from an SEO perspective, explaining that, when we use them as the headings and main navigation on the site, the search engines don’t know what to do.

Vanessa points out that nobody goes to Google and searches for “Products”. Instead they search for what they are looking for. But the heading and navigation links are critical to helping the search engine do what it needs to do.

Home pages, like at 3m.com, use generic words like “Products & Services” and “Our Company” for their links and headings.

So, her recommendation was to stop using words like Products, Solutions, and Clients and start using words that actually describe what you offer. This way the search engine would list you higher for those terms.

Interestingly, I’ve found the same when watching people use the web sites. Rarely, does someone say, “Hmmm. I wonder what products they have.” Even when they do, the menu (usually a simple drop down, but these days, a mega menu is common) lists the trademark names of the products, which, almost always, don’t actually say what the product does.

Sites like Progress.com use a mega menu for their products.

If you aren’t already familiar with the product offerings, how do you know where to click next? Sybase also offers a solutions tab. Again, these are buzzword-filled terms that are vague in what they actually mean. What is the difference between predictive analytics, high-performance business intelligence, and quantitative analytics? What do these terms actually mean? (Would we ever be in the market for low-performance business stupidity?)

Sybase.com‘s solution tab is riddled with buzzwords. How likely is it that anybody but dedicated customers and employees will know what this means?

So, once again, the parallels between what search engines needs for SEO and what users need from the design are striking. SEO strives to make it easy for the search engines to understand your content, so they can offer it to their searchers when that’s what the searcher is seeking. And what users need is an easy way to understand your content, so they can choose the right path through the site.

Coming up with terms for one will easily buy you terms for another. And it all comes down to providing great scent for both your users and for the search engine crawlers.

Update: Keith suggested Verizon Wireless’s site as a good example and I agree.

Verizon Wireless uses terms like Phones & Accessories instead of Products.

If you search on Phone & Accessories, their site appears in the top 10 organic listings. And, from a user perspective, the labels mean something more than Products.

Another Update: @OppDes tweeted that Gerry McGovern’s new article on Top Tasks versus Tiny Tasks gets to the core of this. I agree with that too.

Ok, really it’s just the story of the links. (We’ll talk about the content later, I promise.)

About 10 years ago, we started looking at how users decided to move from one page to the next. Curiously, we found a consistent pattern, independent of the users’ previous experience or the design of the page. Upon reaching a page, the users scanned the page for the phrases or words that were important to them. If they found them, they’d try to click on them (or the link that seemed to go with them).

We call those magic phrases “Trigger Words”, because they trigger the user into action. They are key to understanding the secret to getting your users to the content they’re seeking. Hardly a day goes by where we don’t talk about their importance to one client or another.

That’s why we decided it’s time to republish the article we wrote about them, The Right Trigger Words. While we wrote this back in 2004 and the examples have aged a little (Analog’s home page, for example, now uses flyout menus instead of listing all the trigger words right on the page), the article is still the best resource we have to explain what we mean. I’m betting you’ll enjoy it.

Bringing this article up is timely, because it fits perfectly with Shari Thurow’s upcoming UIE Virtual Seminar, When Search Meets Web Usability. Join us on the 12/16 webinar, and you’ll see how Shari uses trigger words to make sure you’re getting the most out of your search engine optimization efforts.

Are trigger words important to your design strategy? What techniques have you used to identify and integrate them into your site? Share your experiences below.

Subscribe to our podcast via

Last month we held a UIE Virtual Seminar where I presented my talk, Search, Scent, and the Happiness of Pursuit. As always, we had a number of excellent questions from the live audience that we couldn’t attend to during the seminar, so, with a little help from Adam Churchill, I recorded this podcast and cover a number of those remaining questions.

If you didn’t attend the live seminar, and are interested in helping users find what they seek, then you’ll still enjoy this podcast. If you find yourself wanting more afterwards, don’t forget you can still purchase a recording of the session for another 90 minutes of Search, Scent, and the Happiness of Pursuit.

During the podcast, I dig into these questions:

• Should the failed search log be getting smaller if the design is good?
• Is there a correlation between search volume and customer satisfaction?
• How important is the number of search results shown? Do users want to see lots of results and filter down, or see fewer results and browse?
• Would you expect these results to change if you were studying Intranet sites?
• Is Advanced Search relevant or necessary?

Tune in to get some tips on how to get your users to what they seek. Still have questions? Start the discussion in our comments, below.

In today’s UIEtips, we continue with part 2. In this article, Daniel explores specific aspects of faceted search interfaces that raise front-end usability concerns such as the search box and dealing with multiple selection. He then looks at a more holistic approach through design patterns.

If you’re in the process of implementing faceted search within your web site, or thinking about doing so, you’ll want to watch the upcoming UIE Virtual Seminar with Daniel Tunkelang and Pete Bell on faceted search. You’ll get some real insights into the challenges and tricks when implementing faceted search. And, as a bonus, you’ll get a free copy of Daniel’s book Faceted Search when you register. Read all about the virtual seminar and see a preview. 

Have you planned out a faceted search interface to your data? What were some of the hurdles you ran into? How did you work through them? Share your experiences below.

Users arrive at your web site with the simple goal to find something that’s important to them. If they find it, whether they search or not, they’ll be happy. When they don’t find it, frustration follows.

Teams often turn to a sophisticated built-in Search capability to help their users find what they seek. However our research has shown that technological magic isn’t going to make the users successful. Instead, it’s a simple understanding of what the users are seeking and how they look at it. We’ve put together the next UIE Virtual Seminar to address this Search issue.

Be prepared to see how Search fits into your site in an entirely new way. Not only will you come away with solid insights from the most up-to-date research, you’ll be chomping at the bit to start making improvements right away. And you’ll be on your way to the world of User Happiness.

UIE Virtual Seminar
Search, Scent, and the Happiness of Pursuit
with Jared M. Spool
Thursday July 9, 2009, 1:30pm ET
90-minute online presentation

Read more about the Search, Scent, and the Happiness of Pursuit, or see the great preview Jared put together, to help you understand what to expect out of this seminar.

In advance of the presentation, we’d love to hear from you. What does your team struggle with when designing for Search? What type of feedback do you get from your users on how well they accomplish their goals on your site? What does a successful visit mean? We’d love to hear your thoughts, questions, and concerns. Please share your thoughts below.

In this week’s UIEtips, I conclude our feature discussion on producing great search results pages. In the article, Producing Great Search Results: Harder than It Looks, Part 2, I share behavior patterns we’ve uncovered as we researched how people interact with the results from a search query, including how they deal with link relevancy and the chunking of results. When we initially published these articles, there was some interesting buzz across the blogosphere. Our findings didn’t match conventional thinking. Almost a year has passed since the original prinitng, and I wouldn’t be surprised if these articles still creates some buzz. Below you can read comments from when we originally printed part 2.

If Search is high on your priority list, then you’ll want to attend the next UIE Virtual Seminar that I’m presenting: Search, Scent, and the Happiness of Pursuit. In this seminar, I’ll smash some important myths to smithereens. You’ll see how the home page isn’t where people are searching from (and why that changes your entire Search strategy). And you’ll see how Search is more related to the links on your site than you ever imagined. Learn more about this UIE Virtual Seminar. 

When you’re watching your users interact with your site’s search result pages, what behaviors have you noticed? We’d love to hear your insights below.

UIE Virtual Seminar
New Ways to Think about Taxonomy:
The Role of Taxonomies in Your Organization
May 7, 2009, 1:30pm ET
90-minute online presentation

We’ve asked Seth Earley and Stephanie Lemieux, experts in creating and maintaining sophisticated taxonomies, to broaden your thinking about how a taxonomy can make your life easier, whether you’re designing a public-facing web site or a large-scale intranet. In this 90-minute online session, you’ll see detailed examples of taxonomy applications and how to leverage key design principles across your organization. Stephanie and Seth will provide a better understanding of your own taxonomy and the navigation of your information. You’ll get your content management system under control and improve your search results.

To help you understand what to expect out of this seminar, Seth & Stephanie put together a great preview for you:

When people disagree, it’s often because they think I’m suggesting that we stop putting breadcrumbs in our designs. I’m not suggesting this at all.

I’ve defined a design cop-out as something that happens when the designers focus on treating a symptom instead of addressing the root problem. A cop-out is a red flag that should be raised in the design process, to ask the question, “is there a better way to solve the problem?”

At doteduguru.com, blogger Michael Fienen wrote a thoughtful rebuttal to my article with many of the questions I often get when I start talking about my thoughts on Breadcrumbs. Responding to Michael’s points makes for a nice way to talk about these issues, so I thought I’d take some time to do that.

Surfacing the Content

In the original article, I said:

The idea behind how breadcrumbs should be used is simple: the user ignores them until they get to a page that isn’t quite what they wanted. They discover the trail of links and click on the one most likely to contain the correct path to what they were originally seeking.

To which Michael responded:

I think [this idea] is patently incorrect. A user doesn’t necessarily click on a bread crumb because they think it will take them somewhere better or put them on a correct path, nor is there any reason to believe they are used only by lost visitors in the first place. They click them so that they can surface up in a web site and potentially begin navigating anew.

Micahael’s not the first to suggest this. Many information architects I’ve talked to hold this, as we see it, common misconception: breadcrumbs are not only a loss-recovery mechanism—they also serve as a tool for “surfacing the content” of the site.

What’s interesting is when we’ve studied users, both in the lab and in the wild, we almost never saw them interested in “surfacing the content” or learning more about the site. Sure, they want to find the content they desire. If the target content is on more than one page, then they need to get to the subsequent pages. But beyond the user’s explicit target content, we never see them show any interest in the other available content on the site.

Since our early studies on the web, more than 12 years ago, we noticed that users are always on specific missions when they come to sites. With only one exception, users never visit a site “just to see what it has.” (The one exception? Web designers.) They always have a mission:

• Buy a new winter coat and accessories
• Find out what my portfolio is worth
• See if my favorite blogger has posted anything new
• Figure out a nice gift for my niece even though I have no idea what 15-year-olds want these days

Even the last one, where the user can’t describe the outcome, is not about the site. It’s about their niece’s gift. That user (like every other user) would want to surface all the content related to their goal, but will show no interest in content that’s unrelated. Only designers are interested in seeing what’s on a site.

In our studies, almost 94% of quests on web sites have a single objective. When the user reaches the target page, they’ve accomplished their goal. (Or, at least the “finding” portion of the goal. There still may be transactional component, such as purchasing.)

So, in 94% of the tasks, if the user turns to the breadcrumbs, it’s likely because they couldn’t find their target page and are lost. That leaves at most 6% where the user completes their initial objective and needs to start on a subsequent objective: “Ok, I’ve bought the down jacket. Now I’d like a matching hat, scarf, and gloves to complete the outfit.”

Michael’s argument is even if multi-objective quests happen infrequently, the breadcrumbs still serve a useful purpose, revealing the rest of the content to the user:

Assuming you have taken the slightest modicum of care with building bread crumbs, users will recognize them as a reflection of the hierarchy of your site’s information architecture, making them a tool that users have no reason to ignore if they are viewed as an aid to going where they want to go.

But that’s the point: users don’t care about the hierarchy of the site. The thousands of users we’ve observed for the last 12+ years clearly tell us that users don’t care how the site is constructed. Users only care how to get from they page they are current at to the page containing the content they seek. Even with repeated use, they’d prefer that each site visit just have clear scent. Memorizing the nooks and crannies of an information architecture is not their desired outcome.

Secondary Navigation

Michael agrees with this statement from my original article:

We’re recommending that when teams see users needing breadcrumbs, they look for other holistic design solutions. They’ll need to watch users and see the circumstances leading up to how the need arises. In almost all cases, they’ll find a better way to solve the problem than traditional breadcrumbs.

He goes on to say:

The key to successful bread crumbs is that they should be a secondary navigational tool. But, I would argue that people don’t use them because they need them, they use them because they see them as a means to get to where they want to go. As far as the user is concerned, that might be a quick link, an A to Z index, a menu, or a bread crumb (and all of these, minus menus, are generally secondary tools). The thing is most users neither know these terms nor care about them. All they care about is “I click here and go where I want.”

Michael is correct that users don’t distinguish between what he calls secondary navigation and the other types. The idea he proposes, “I click here and go where I want,” is a basic notion behind thescent of information theory: if the target content gives off good scent, users will click on it.

Let’s return to our down-jacket purchaser, now looking for matching accessories. If that user’s trigger words (such as “scarf” or “hat”) appear in the quick link, A-to-Z index, or breadcrumbs (Michael’s secondary navigation tools), then all is well.

Yet, on many sites, it’s dumb luck if the site designers have included the trigger words in those tools. In most cases, the designer hasn’t researched the specific trigger words users will want. Instead, they produce a set of generic terms (“accessories” or “outerwear”, for example) that may or may not resonate with the user.

So, I’d go further to say that all the secondary tools that Michael mentions are also cop-outs: fixing symptoms (in this case, providing a standardized navigation element) instead of the users specific problem (getting match accessories). (I wrote how sitemaps, which are parent to A-to-Z indexes, are also cop-outs in another article.) If I asked any designer worth their weight in salt to design a way for someone who just picked the down jacket to find the desired matching products, I’m betting, of all the design alternatives, Michael’s list would be the last choices.

Michael continues,

I agree with Jared that given perfect IA, smart menus, and intelligent visitors, bread crumbs are a waste of time. In reality, few people run sites that function in such a static bubble that one person has control over every facet of how information is disseminated. [...] It’s like saying “In a perfect country, we wouldn’t need laws to punish robbers, because no one would steal from each other.” The reality is, people do steal. That doesn’t mean we shouldn’t strive to stop them, and shouldn’t minimize the problem, but you still must address the issue. So what do we do? We create a ton of secondary navigational elements, build them nicely into our layout, and let the user decide how they want to combine them to go where they need.

In the stealing analogy, it would make sense to look at the economic conditions driving people to stealing. Solve those and the robberies diminish. Focus only on punishment and you end up spending your resources building more prisons indefinitely.

I’d say the same is true for breadcrumbs. Users don’t want choices in their navigational tools. They want clear scent to the content. It’s the designer’s responsibility to provide that. Anything else is just a cop-out.

Breadcrumbs are Simple to Implement

One of the most common objections to my argument is “breadcrumbs are so simple to implement that there is no harm to just doing it.”

Unfortunately, this isn’t true. On a site of any decent size (greater than 500 pages), breadcrumbs become very difficult to implement well.

Often, in an attempt to make life easier, the designers use the category hierarchy as the breadcrumb links. On the surface, this sounds like a good idea. After all, if the categories are well thought out, then they should work in breadcrumbs as well as anywhere else.

Alas, that isn’t the case. Breadcrumbs stand by themselves as solo links. The categories are usually created to be shown as a collection. A category may have a clear meaning when shown alongside its siblings, but is often baffling when shown alone.

Take this example from Michael’s post – the breadcrumbs from NewEgg.com:

It’s not clear what the siblings are. I’m betting most folks would be surprised to find “Networking”, “PCs & Laptops”, and “Apple” to be listed as siblings to “Computer Hardware”, for example. Arriving at links that would describe the entire category well are difficult and usually require more than one or two words. That’s where it becomes difficult to implement breadcrumbs.

Throwing the Baby Out

As I’ve said before, I’m not suggesting that designers stop implementing them. I’m just trying to prevent the knee-jerk reaction of always including them under some misguided notion that they always improve the site.

In the best case scenario, they take no effort (as in automatically compiled by the CMS) and are ignored by users—thus are no harm done. But, that’s rare and unlikely for most situations.

Good design understands why every pixel is in the design. The designer knows how every element is directly serving the user in each instance. Automatic design (“every page needs breadcrumbs at the top, whether we have evidence it helps or not”) rarely accomplishes this.

But, here’s the rub: In the end, it doesn’t matter what I say. It only matters what happens with your users on your site. If Michael’s observations of his users shows that breadcrumbs are the most useful way for them to achieve their objectives, then I think his site should have breadcrumbs—cop-out or not. (And I’d like to learn more about his situation, because I’m always interested in proving my theories wrong.)

Does your site need breadcrumbs? The only way to know is to watch users. It’s simple, really. When we see someone click on one, we stop them and ask what they’re hoping to accomplish. That gives us a use case to work with. If the use cases point to a breadcrumb element being the best solution, then we go ahead and make that work.

Some find my labeling specific elements (like breadcrumbs) as cop-outs is harsh. But, that’s the point. Had I said, “breadcrumbs might not help as much as you think”, you probably wouldn’t have given this topic as much thought.

My purpose is to get you to think twice about using them. If I’ve made you seriously question your usage of them, then I’ll sleep well.

So, it shouldn’t be news or a surprise to anyone in the world of web site usability that having clear links that describe what the user will find after clicking is a priority in the design process.

Fortunately, making things usable isn’t a priority for the folks at the Journal of Usability Studies, who just released their latest issue. It’s great that they now have online versions of the articles, instead of having to read the PDFs.

However, we think they could do better on the table of contents for the article:

They could say a little more about what the article says in each section, don’t you think? I wonder what would happen if they did a little testing?

Brian Christiansen and I recorded a special episode comprised entirely of questions from our customers. On August 14, we held the UIE Virtual Seminar – Galleries: The Hardest Working Pages on Your Site. During the seminar, we received far more questions than time would allow answering. As is tradition, we put together this follow-up podcast to answer even more of your excellent questions.

In this episode, we discussed:

• A review of Gallery pages and their connection to Scent
• How Gallery Pages are special, and a comparison to a Department page
• The role of Galleries in Intranet environments
• We dive into the use of “learn more” links with a little more depth
• Our observations about how users really “browse” the web
• Advice on coping with large numbers of links on Gallery pages, and what links belong there
• A preview of September’s Virtual Seminar on Critiquing

If you missed our live seminar, a recording of Galleries: The Hardest Working Pages on Your Site is available for viewing.

Still have questions or comments about gallery pages? Ask them in the comments below!

(Producer’s note: My apologies to the Discount Tire folks for calling you “Direct Tire” once in the episode. I got it right the second time, though!)

Brian Christiansen and I recorded a special episode comprised entirely of questions from our customers. On July 17, we held the UIE Virtual Seminar: The Scent of a Web Page—The Five Types of Navigation Pages. During the seminar, we received far more questions than time would allow answering. As is tradition, we put together this follow-up podcast to answer even more of your excellent questions.

In this episode, we discussed:

• how we determined “failure” and “success” when we studied users
• how our research applies to college sites
• the undesirable trait of pogosticking up and down between levels of pages and why that’s a sign of navigation failure
• examples of link-rich homepages that users love
• why, contrary to popular opinion, users still don’t like to search

In the podcast, we referred to an article we wrote a little while back, called Lifestyles of Link-Rich Pages, which provides more information on long-links and our home page research.

If you missed our live seminar, a recording of the session is available for viewing. See The Scent of a Web Page for details.

Still have questions about the five types of navigation pages? Ask them in the comments below!

In this 90-minute online presentation, I’ll share the results of years of research examining how the best sites navigate users to their content. I’ll introduce you to the concept of the Scent of Information, the biggest secret to successfully getting users to the content they’re looking for on your site.

You’ll learn:

• How the successful sites provide a strong scent and what happens when they don’t
• How users follow a scent trail and the different ways your design could be blocking scent
• How the quality of links, page length, page density, and graphics affect whether users find their content

There’s still room to sign up for the live event. You can gather your team around a computer and watch it together! (If you can’t see it live today, we’ll make a recorded version available in a few days.)

More details about today’s UIE Virtual Seminar.