Web App Usability Techniques Part 1 Brian Usability Tools Podcast, Episode 14: Web App Testing, Part One. Christiansen: Recorded at the studios of User Interface Engineering, January 25, 2008. [Musical interlude] Brian: Welcome. I'm Brian Christiansen, producer of UIE podcasts. Each week, we're sharing tools for improving your site's user experience, based on our research at User Interface Engineering. If you're interested in the topics we discuss in the podcast, sign up for our popular, free newsletter, UIE Tips. I'll have more details on how at the end of the podcast. And now, this week's episode. [Musical interlude] Brian: Welcome back to another edition of the Usability Tools Podcast. I'm Brian Christiansen, and with me today is our founding principal, Jared Spool. Hello, Jared. Jared Spool: Hello, Brian. Brian: At UIE, we're only a few months away from our fabulous WebApp Summit, and that has us in a web apps state of mind. Which leads me directly to our first question: why are web apps different? Why do we have a separate, whole conference about them? What sets them apart? Jared: Well, they're different because you have to be in sort of a different state of mind when you're designing them than say a website. A website, when you're designing it, that's primarily informational--let's say a news website or a corporate site that talks about products and services--is about finding stuff. It's about finding the information. And web apps are about transactions. They're about doing something. So web-based applications are different in that regard. You're completing a transaction. You're trying to order something. You're trying to book a trip. You're trying to update your information. You're trying to check up on the inventory out of a database. So you have this sort of transactional notion of the user puts in information and get something back. And so that makes them different. The other thing that makes them different is that they have these sequential flows to them, and you have to think in terms the sequence of things, and that makes for interactions that are interesting. So they're very interactive, and more than just click on the link, get a page, click on the link, and get a page, which is what a non-web app website is about. So that's what makes them different. Brian: So, since we believe that testing is the key to providing an outstanding user experience, is testing different for web apps? At what stages should a web app be tested? Jared: So, when we're talking about usability testing and usability research, the types of things we want to do basically falls into three categories. There is the stuff you need to do when you're just figuring out what the requirements are. So it basically answers the question: what will this application need to have in it? You look at your competitors. You look at comparable systems that do similar things to what you're doing that may not be competitors. The second stage would be when you're trying to figure out what the thing should do. So now it's what design elements are going to work for this problem? And that's a different type of testing. And then, finally, when you're close to done, you need to see if the design is actually doing what you thought it did. So you're going to actually test it in its entirety, or in major sections of it, and make sure that the design elements are doing what you thought they did, and comparing it what your expectations were. Those are the sort of three stages that you're thinking about usability testing when you're doing web-based applications. Brian: Great. Since "web app" is nearly as generic a term as "software," can we talk a little more specifically? What are the different classes of web apps? Do we test them differently? And if so, how? Jared: So, basically, the types of web-based application designs you come to fall into mostly two categories. You have something called a hub -and-spoke design. This comes from Hagan Rivers, who has done a lot of work on this, and she wrote a report with us on this topic. And basically, the hub-and-spoke design idea is that you've got sort of this centralized set of functions that are all in a hub. So, for an example, the "My Account" page on Amazon is a hub-and- spoke thing, in that, on that page, you can check the status of an order, you can change your shipping address, you can buy a gift certificate. There are all sorts of little functions off this one page. There's like 50 different things you can do. And so that's the hub. And then you click on one of those and you go down one of the spokes, and then, as soon as you're done, you come back to the hub. So that's the hub-and-spoke thing. The other type of design of a web app is what's called an interview. And the interview is this long sequence of things. So you can think of, for instance, an online tax application: basically keeps asking you question after question after question after question. It doesn't have the hub-and-spoke design because you don't keep going back to the same page, per se. Brian: Right. It's kind of analogous to a wizard in software. Jared: Exactly, exactly. Now, to be fair, you merge the two often. So, for example, buying a gift certificate on Amazon, the act of buying the gift certificate, that's an interview process. It's got several screens to it: "Who are you sending it to? How are you going to pay for it? What's the amount?" Right? So that's the sequence. That's an interview. But then it comes back to the "My Account" hub when it's done. So you combine them. You mix and match as you go. And sometimes you have little hubs. You can actually draw them out on your whiteboard, or chart them out, as little wheels, and then sequences of spokes that go out, and that's it. So, in terms of testing, you test the hub-and-spoke differently from the interview. They're different things. Brian: Right. So, with those two general types of application designs--the hub-and-spoke and the interview--I would assume, since their flows are different, that you test them differently. Jared: Yeah. Well, you're looking for different information, right? So the hub-and-spoke, you need to sort of test the individual functions. So you need to know if a given function works the way it's supposed to. But then you have to test the sort of multifunction operations, because, for instance, if you're building an application like Google Docs, where you're doing word processing, the various functions on the menus, making something bold, those are basically little lines off the hub, so you need to test them working in conjunction with each other. If you do italics then bold, is it the same as if you do bold then italics, right? Should produce the same result, but sometimes order matters and you don't get what you expect you're going to get. So you need to think of the discrete elements, and then you think of the whole thing. When you're talking about interviews, it's the same sort of thing. You want to focus on the individual page. Because sometimes the pages can be very complex. You have a whole bunch of fields; you want to make sure each field works. And then you want to test the entire flow. Do you have the screens in the right order? Do you ask something earlier? There's one [laughs] website that I talk a lot about. It's a registration. It's a major vendor, their online registration thing. And it turns out--it's a four-page process--on three of the pages, they keep asking you what language you speak. Brian: [laughs] Jared: It's very clear that someone tested the individual pages but never looked at the whole sequence and that people kept adding things to it. So now, it keeps asking you if you speak English over and over again. Of course, it keeps asking you that in English, so if you don't speak English, you have to keep answering the question over and over again in English. It's sort of weird. Brian: So we've been talking about the how of testing. What about the what of testing? Are there specific issues that web app developers should be looking for? Jared: Yeah. Well, there's a whole bunch of things that you want to be looking for when you're testing web-based applications. For one thing, you want to make sure that the user can find the application in the first place, right? So, in the Amazon example, the "My Account" link, you have to be able to find that some place, right? And they keep moving it around. Every time I... [laughs] Brian: I have the same thing. I go in there to print a receipt or something, and I'm like, "Where?" Jared: Exactly. Exactly. So that's one challenge is to be able to do this. And this is particularly an issue where you have multiple applications to choose from. For example, on an airline site, you can reserve a reservation, you can check in for a flight, you can check on your frequent flier miles. So you've got these different applications off the home page, and can the user find the one that's specific to them? On Delta, for example, they use tabs which are graphically sort of non-distinct. So it's actually hard to find some of these things, occasionally. So that's one thing. And then you're also, in that process, making sure the user understands what the application is called. If I want to go look up a wish list for my friend, or my daughter, for example--I want to go look at my daughter's wish list--I have to know where to find that function on the Amazon site. And it's not trivial to find. It can be very difficult. I don't know what they've called it. I know that they generically call things "wish lists," but what I didn't know was that you had to look under something called "Gift Organizer," right? Brian: Oh. The whole language thing and trigger word related aspect. Jared: Absolutely. Yeah, it's very much like any other part of the website. So the finding part is sort of strategic website stuff. It goes back to the scent of information. It goes back to trigger words. That's basically it. Now then, another thing you want to test is the user's ability to prepare to use the application. Do they know what they're going to need up-front? So, for example, someone buying a gift certificate, are they aware they're going to need their credit card? Because they get into the middle of the sequence and they don't have their credit card on them, they're sort of dead in the water. Brian: Yeah, they could bail. Jared: Exactly. And with something like Amazon, where sometimes you need a credit card and sometimes it remembers a credit card from previous things, it's not clear. It turns out, with gift certificates, they don't let you use your stored credit cards, for security reasons, because they don't want somebody buying themselves a gift certificate while you've snuck away from your machine without you knowing it, right? Brian: Right. Jared: But making it clear up-front that they're going to need that information, that they may have to go grab it so it's there, that's the type of thing. Another thing in preparation is letting people know how long the application is going to take. If someone goes in thinking something is only going to take a minute, and it takes five minutes or 10 minutes, they could be in a situation... For instance, being able to check in for a flight from home. You realize you hadn't checked in yet, you want to do it quickly, but you have to leave for the airport. You want to know up-front if that's going to be a 2-minute things or a 10-minute thing, because that could affect whether you decide to do it at home or you decide to just bag it and do it at the airport. Brian: Right. Jared: So those are the key things there. The third thing is, basically, matching the flow of the application to what the user needs. Sometimes flows don't quite meet the user's needs. An example of that is a checkout process. We've seen many checkout processes where they don't tell you the shipping price until after you've given them the credit card information, and people are reluctant to hand over their credit cards until they know how much the total package is going to cost. Brian: Yeah. Jared: And so we've seen people stop dead at the credit card screen, not knowing that, and because of that, it is a problem. We watched one company that was losing sales on the credit card screen, and they thought it was because people thought the credit card wasn't secure. And then we watched users and found out, no, it's because they didn't learn what the shipping was on the next screen. And in fact, what they would've learned was that, most of the time, the shipping was free. But the site never told you that until after you put in your credit card information. And people were like, "Well, they're not telling me what it's going to cost to ship. And this is a printer that I'm buying that's probably going to cost a lot." But because it was a $2,500 printer, in fact, it was free. And so they lost the sale. So that turns out to be an issue. The other thing that you want to find out is, when there are forks off the path, does the user choose the right one? Or do they accidentally go down the wrong path and make life more difficult for themselves or not get what they need? So that's part of what you're looking for. And then, when they get to the end, are the results what they expected? Do they get all the things they expect to get? So, for example, in a checkout process, if they're expecting to get some sort of sales receipt, so they can take it to their boss and get reimbursed for the expense, are you giving them that? So these are the types of questions that you want to be looking for. The fourth thing: handling contingencies and exceptions, dealing with special cases--what the folks at 37signals likes to call defensive design. And so, what happens when someone has a really long name? How do you handle that? Do you truncate it? Do you require them to shorten it? Things like that. It's interesting. A lot of sites don't let us enter our company name, User Interface Engineering, because it's just a little too long. They have a 20-character limit, and ours is more than that, so we often have to shorten it to "User Int Eng" or something like that. It can be problematic. So have you considered all those types of things? What happens when the user puts in the wrong information? So, for instance, they make a mistake in their credit card number. How do you handle that? Or they put in a product number you don't recognize. How do you handle that? What happens when they leave things blank that they shouldn't leave blank? The problem is that real life is messy, and you see these things. One of my favorite examples is someone gets a license number, right? You have to authorize use of something, so you get this license number. So you purchase it and get the license number. They copy it from the email that had it, they try to paste it into the application, and because it has hyphens in it, it breaks. OK? Brian: Yeah. Jared: The hyphens are in there to make it easier for the user to see and make sure they get it accurate. But the system doesn't want hyphens. It wants it all scrunched together. Credit card numbers with spaces... Brian: Absolutely. Jared: What happens if a user puts hyphens in the credit card number? Now, it's like two lines of code to take all the spaces out from between the credit card number, yet so many systems will sort of barf on that. The phone company gets upset if you put hyphens in phone numbers. I mean, that's insane. Somebody should tell the phone company that people like to put hyphens in their phone numbers. You think they'd know that. They deal with these numbers all day long. Brian: Right. Jared: So those are the types of contingencies you have to deal with. And real life is messy in that regard. So we have four. There's a fifth one. So the first, finding the application; being able to use the application, preparing to use it; making sure that the flow matches the user's needs; making sure you're handling the contingencies and exceptions. And then the final thing is dealing with the fact that a web-based application-- until we get more sophisticated with Silverlight and AIR and the things coming down the pipe--are all based on the browser, right? They're all in the browser. So, what happens when the user hits the back button? What happens when they've turned off JavaScript? What happens when they're using a screen reader? What happens when they're running it on a mobile device? What happens when they've bookmarked a page in the middle of the application and they click on that bookmarked link? All of these things. Printing's another one. Does the site handle printing properly? Does it print the page right? So all these things have to do with the browser. You're going to have to test with multiple browsers and that type of thing. So those are the five different types of concerns that we have when we're testing web-based apps: finding the application, preparing to use it, making sure the flow matches the needs of the users, dealing with contingency design, and dealing with the browser. Brian: Well, great. That was a lot of great information here. And we have a lot more. So we'll stop here and make a second episode next week that has the conclusion of "how to test web apps." Thank you, Jared. Jared: You're welcome. [Musical interlude] Brian: We hope you've enjoyed this Usability Tools Podcast. If you're interested in more of UIE's research, sign up for our free email newsletter. You can subscribe easily at uie.com. We love hearing from you. Send us your comments at mailbag@uie.com. That's all for this week. Thanks for listening. Good bye. [music]