Web App Usability Techniques Part 2 Brian Usability Tools Podcast Episode 15: Web App Testing Part 2, Christiansen: recorded at the studios of User Interface Engineering, January 25th, 2008. [Music interlude] Brian: Welcome, I am Brian Christiansen, producer of UIE Podcasts. Each week, we are sharing tools for improving your site's user experience based on our research at User Interface Engineering. If you are interested in the topics we discuss in the podcast, sign up for our popular free newsletter UIE Tips. I will have more details on how at the end of the podcast, and now, this week's episode. [Music interlude] Welcome back to another edition of the Usability Tools Podcast, I am Brian Christiansen and with me today is our founding principal, Jared Spool. Hello Jared. Jared Spool: Hello Brian. Brian: At UIE, we are only a few months away from our fabulous Web App Summit that has us in a web app state of mind. This episode is a continuation of the prior episode about web app specific testing techniques. If you haven't listened to that one yet, you want to head over to UIE.com/audio and listen to Part one first and then rush back here for Part 2. All right, so let's dive back in. Jared: [laughs] they may still be listening to Part 1. Brian: You are such a trouble maker. Jared: [laughs] OK. Brian: Last time, we talked about why testing for web apps is different than testing other creations. Now let us talk a little bit more about the how. What is at our disposal in the testing toolbox? Jared: Well, there is basically a basic usability test which is something that doesn't really differ whether you are testing web-based apps or anything else. And then there are variants on that technique that you can use, because usability testing is sort of like a saw or a hammer, there are different types of saws. There are saws that cut glass and saws that cut wood and saws that cut metal and you have got electric saws and hand-powered saws and gas-powered saws for cutting down trees. So there are all different types of saws. Similarly, there is all different types of usability tests. They are all sort of based on the same basic concept, but there are all sorts of variants to put that in. And then the other thing is field testing, going out into the field, something we have talked about a lot in the past, going out into the field and actually testing there. Brian: Well, let's start with the basic usability tests, where would we start with those? Jared: So a basic usability test is very simple. In its simplest form, you sit next to someone and you watch them use your site. Basic usability tests use what we call a directed task. So a directed task is when you ask someone to do something specific, so it would be something akin to: you would like to buy a gift certificate for a relative for $25, go ahead and do that. Because it is directed, you are going to be testing whether the user can find the application, you are going to be testing how they get ready for it, you are going to give them some materials, you might tell them who they are buying it for, you might tell them the credit card number to use if you don't want them to have to fuss with getting out their own credit card. But you are basically going to set up this scenario and just have them work with that. So you are going to provide them the materials to do that and then you are going to see if for the specific scenario that you will set out, that the flow matches the user's needs, you are going to see some contingency and exception stuff, but only in mistakes that they might make. Because whatever format for instance you give them the credit number in, that is likely what they are going to do. So if you put spaces in it, you will see them maybe put spaces in it. Things like that. Sometimes what people do is they make a fake card that actually looks like a card. It is got a logo and everything on it, so you can see if they are reading off the card that they do the same thing. And then you will be able to test for what we call the "happy path," which is the happy path is basically what happens when the user follow the path that is best. Let's say you want to test a registration system, it is just a simplest way through that registration system without any special options or contingencies or anything. It is just the clearest way through it. And so directed tasks are useful for that. So, they can get you through the basic tests, they can get you through the basic process and it works very well. They are simple to do. We have talked a lot about how to do a usability test. If you don't know how to do usability tests Christine Perfetti did a fabulous virtual seminar on usability testing that is available on our site. There is also the Handbook of Usability Testing by Jeff Rubin and there is a bunch of other resources on our site for doing that. Ginny Redish and Joe Dumas have a great book, whose name is escaping me right at the moment, but it is a fabulous book. We will put a link up in the show notes, right? Brian: Absolutely. Jared: So those are good resources for conducting usability tests if you have never done one before, but in its simplest form: it is just sitting next to someone watching them do stuff, asking them to do a particular thing. Brian: So those are basic tools that we would use for just about any web creation, are there any variants one might want to look into that can give us additional insight or data for this particular type of testing? Jared: Yeah. Well, for very large applications -- there are a whole bunch. So for one instance is benchmark testing. So benchmark testing allows you to sort of monitor the change in the design over a time period. So for example, you might look at an application, let's say you were doing an online expense reporting application, so you might test to see for a common expense report, maybe someone going on a five-day trip trying to get their expenses all assessed. You might put together a set of tasks that simulate that and ask people to do it with the existing design and then you change the design and ask them to do it again and see if you see productivity improvements. So you are looking for productivity improvements in that. You might compare your particular process to somebody else's. So for example, signing up for a new account, you might compare the time it takes to signup for an account on your system to the time it takes to signup for an account on an competitor's system. So these are benchmarks that you can do. So keep actually doing the same test over and over and over again with the intent of seeing how it is different as you change the design and how it is different as you come upon competitive designs that you want to see how they perform. So that's one thing. So that is benchmark testing. Another technique which is really different is paper prototyping. Paper prototyping helps you -- whereas benchmark testing basically helps you with that sort of end-game stuff; is this thing meeting the needs of the business, is it meeting the needs of the users at the end -- paper prototype testing helps you with figuring out what the requirements are. So what you do with paper prototype testing is you put a design together and sometimes you don't spend much time thinking about what that design should be at all. It is like the first thing that comes to mind, you create a fully working paper prototype and little pieces of paper, usually uses their fingers, the mouse, they click on things, you can use transparency film for forms or little pieces of writable tape that you can write on -- removable tape, post-it notes, things like that. So you put together this test and then people actually use this thing. They go through. They fill out the application. And from this, you can see, do you have the right design or the screens in the right order, are the flows right, so all those sort of requirement stuff that happens at the beginning is there. So paper prototype tests are great for that. And a great resource for paper prototype stuff is Carolyn Snyder's book Paper Prototyping and she also did a virtual seminar for us a few months back. It is a great virtual seminar. Brian That is a great primer on getting started with that, inexpensive Christiansen: way to testing. Jared: Yeah, really is. And there are a lot of articles on our sites about paper prototyping. We should probably do a podcast on that at some point. That will be a good thing. Brian: Definitely. Jared: Another technique that actually works really well for another variant on basic usability testing -- that really works well -- is co-discovery. Co-discovery is a way to conduct a basic usability test by actually having two users work on the design at the same time. They are working together with the application. So let's say you have an application, basically booking-a-vacation type vacation. You can have them do the research which vacation they want, because they may both want to go to different places. So that part you have to sell. So you give them a task, a directed task. It is like your family has decided or voted that Disney World is the destination. And what we want you to do now is we want you to book a six-day trip to Disney World. Your youngest son has decided that he wants to see the Buzz Lightyear ride, so you have to make sure that you can do that. And your daughter loves Princess Jasmine, so you want to get into the character breakfast. So you put all these things together and then you have these two people work together at the same time to do this as if they were one. What we often do is, we give one person the mouse and the other person the keyboard. And they go through and they now have to book this trip and they have to arrange the breakfast and they have to get close to the ride and they have to do all those things. And the reason you do co-discovery, there is a couple of reasons; first, because you are testing two users at the same time. It is as if you did two separate usability tests, but you collected the data in the same period of time. So it shortens the amount of time it takes while you collect almost, twice, as much information. But the other thing that is really cool and the reason it is called co-discovery is when you do usability tests, people get really quiet and you don't know what is going on in their head, they just staring at the screen and they are wondering and they are scratching their chin, you don't know what they are thinking. Are they confused, are they just reading, what are they doing? So as a facilitator, you have to do things like say: what are you thinking? And it makes life difficult, it makes life slow. So co- discovery, you don't have to that, because what happens is they talk to each other; because they have to collaborate on the task, they talk to each other. So it works really well for this. It doesn't work for regular website testing because finding a novel you want to read on Amazon, the two people could have completely different taste on novels, it is not going to work. But in application stuff, it works really well. Another variant is interview-based tasks. So now, everything we have talked to right now has been directed tests, where you make up the tasks and events. And what we want to do is have them do those tasks, but the problem with directed tasks is they may not be the things people naturally do. Brian: Right. So they may act differently looking up something that they don't have experience with versus something they do have experience. Jared: Exactly, exactly, someone who is a regular camper at a state park would book a camping trip at that park differently than someone who is new to it. And they may have preferences and they may say, "Well, you know, I don't do this element of the camping experience because I have done that three times and I am done with it, so I will just skip that part, " or I know that they don't need this information, so stuff like that. So what you do is first, in the recruitment process for your users, you make sure you find people who are naturally interested in whatever it is, so if you are doing camping, you want to find camping and hiking people. If you are doing - we have done podcasts on interview-based tasks in the past. Brian: Yeah, we have, just look into the archives. Jared: Right. But the basic element is you get people who are naturally doing this and then what you do with that information is you interview them at the beginning of the task, so you spend 10-15 minutes interviewing them. You find out what their interests are you find out how they do it and then based on their language, you say "OK, let's go do that on the site; let's go book this camping adventure." And you use the discussion in the interview to do that. And the beauty of doing this is that you get those contingencies and exception things and you see the preparation stuff that we talked about in the last episode. You see those things far more clearly than if you hand somebody pre-prepared materials. We do this with shopping all the time where we actually go out and find people who need to buy something. They are in the market for a new laptop and so, we actually have them buy it in front of us. And it turns out that the way they buy the laptop is very different than if we just said pretend you are in the market to buy a laptop. Brian: Right, right. Jared: So you get to see all that. Another variant are what we call process tests. So a process test involves making sure that the process works. So for example, there is a website online that will help you expedite a passport application. And it is actually really complicated because you have to do things like you have to put in your credit card information for the passport expeditor, like you can only write a check for the US government. So you have to include both the credit card number and you have to write a check. And you have put things in a certain envelope and you have to go to the post office and they stamp it in a certain way. You have to do this stuff. If you do it wrong, as good as the expeditor is, they are not going to be able to get the government to issue the passport. So they have this very sort of complex process. Some of it is done online, some of it is done on paper, some of it is done on forms. You basically enter this information, you print out a form, you include it with your passport. It is complicated. Brian: There are lots of places [cross talk]. Jared: Exactly, there is a lot of places you can mess up and because you want to - because the reason you pay these guys the big bucks is you need to turn around the passport in 48 hours because you waited till the last minute to get the renewal or something. It is critical that you get it right the first time. So you would do a process test on this. You would have them use the application. You will be checking to see that when they are done, they have put all the things in the envelopes the proper way, they have gotten everything signed, and they have gotten everything stamped the proper way. If that end result is not the right end result, you have to go back and change the design process to change it. So that is what a process test is. And then the last thing is something we call an inherent value test and I think we have done a podcast on that too. Brian: I believe we have. Jared: We have. And an inherent value test is basically a way to get the value of the application out. An example of this is, one of our clients is in the photo uploading and sharing business. So you upload your photos, you could share them. One of the features they have is they produce these really cool photo albums. You can put comments on each page, they bind them really nice, they are really neat. And customers who discover these things think they are very cool. But what we discovered was, that people who didn't know about them, they didn't quite get what this was all about. They couldn't grog it. So the inherent value test basically tells you it is a two- phased test where you get the people who know how to use it basically show you what it is and tell you what they love about it. You say things like, "Well pretend I am a relative or a good friend that bumped into you at like a cocktail party or a family holiday party and you were telling me about this thing." I am like "I don't get it." So you take me online, you want to show me how it is, so pretend we are online and show me how that is. That's what you do in the first phase with the people who are already familiar with it and then you test people who are brand new with it. And you basically see if they looking at it and using it come back with the same value that the other ones is. And that you tell you whether they think the application is worth it and there are all sorts of value statements surrounding that. So those are the basic variants. We have talked about benchmark testing, paper prototyping tests, co -discovery, interview-based tasks, process tests and inherent value tests, so those are the big ones and they all tell you slightly different things. Brian: So great. The last thing I wanted to talk about is all these tests so far have been bringing users in tour on our tests, but what happens if we reverse that and go out into the user's natural environment, are there inherent benefits to that type of thing? Jared: Absolutely, absolutely, and yeah. When you bring people in, they are not at their desk, they are not at their computer, and they are not in their home, so you miss a whole bunch of things. So in fact, there is a huge amount of value to going out and seeing things. I mean if you want to see how someone gets prepared to do their income taxes because you are doing an income tax app thing. If you want to see what happens when you ask the question, "Have you made any charitable deductions this year?" And they don't realize that question is coming and now they have to dig around in their files to find them and maybe it is scattered. That could be a really useful exercise, but you are not going to see that in the laboratory, you are not going to see that in your conference room when you do a test, you are going to see that only when you go into their house. I remember the folks at Intuit telling me that when they were first testing versions of Quicken, they were going to people's houses, they would find people who, the only place they could put their printer was at the bottom of their closet, so they had the coats and the boots. And the printer was there and the paper was feeding in from the back. It was the craziest thing. And that is how they learned for instance what it takes to get the check stock lined up properly, so you could put it through. UPS has a web-based application that involves making labels for packages that you are shipping out, seeing where if someone in an office has to run into the hall, stick the label stock in, run back, press the print button before someone else prints something to that group printer, those are the types of things that you want to see, because you will make all sorts of assumptions. Until you go into the field, you are not going to see how this stuff works. And you'd be so surprised how different people's environments are. We were just at someone's house and she had all this high-tech stuff and then she had this old computer and this old monitor that had this tiny little screen, it must have been 640x480 and it was just shocking how high-tech she was in so many ways, but she was using this computer because she couldn't bear to throw it out, so this was her computer. Her husband had the really cool new computer, but she did everything on her machine. And so that meant dealing with smaller screens and she was scrolling horizontally, it was just crazy to watch her do this stuff. So those are the types of things that we see when we go out into the field and it changes the way that we use things, it changes the way they work. So there is an advantage to this. People are telling me that they are doing sort of almost field studies by, for instance, using video chat software, in some cases shipping a cheap video camera to the user, letting them plug it into their machine and then letting them hold the camera. "You can't see me, I have got my fingers up in the air and I am pointing my fingers as if I was a waving a camera around." This is the benefits of radio. But basically holding the camera up and showing it around their office or their home so that the person can see it. That does work OK, it is not great, you still miss a lot of stuff because it is one thing to see it on TV, it is completely different when you see it there. But it is an option. So you can mix and match. There are no rules to - there is no "This is exactly how you do it, if you don't do it this way, the usability test police will come and get you." That doesn't happen. That's basically it. Brian: Great. Well, I will just add to that that our listeners could also look upÉ we have a virtual seminar with Kate Gomoll on field testing, which is a really wonderful primer on how to get started with that. Jared: That is a good point. And she also has a report. Brian: She did. Jared: She did. Brian: And we have that as well. Jared: Yeah, she wrote a fabulous report for us. Kate Gomoll has done some amazing stuff with field tests. And so the work that she has done is really just stunning. So yeah, if you are thinking about this, it is even just entering your sort of event horizon, it is something you want to do in the future, you definitely want to listen to her virtual seminar and you definitely want to get the report, because people are telling us that those things just make it so much easier to get this done. Brian: Very valuable. Jared: Yeah. Brian: Absolutely. Jared: Absolutely. Brian: So all right. Well, we would like to thank everybody for listening in and goodbye for now. Jared: Take care. [Music interlude] Announcer: We hope you have enjoyed this Usability Tools Podcast. If you are interested in more of UIE's research, sign up for our free email newsletter. You can subscribe easily at UIE.com. We love hearing from you. Send us your comments at mailbag@uie.com, that's all for this week. Thanks for listening. Goodbye. [Ending music]