In our research, we found successful design teams have each team member spend a minimum of two hours every six weeks watch real users interacting with either their design or a competitor’s design. The most successful teams have even more frequent exposure hours.

When team members watch someone use the design, several things happen. First, they gain empathy with that person — empathy that makes them sensitive to frustrations and delights the design imparts. That empathy is critical for setting design priorities, as we try to eliminate those frustrations and amplify the delights.

Second, the team picks up the culture of use. They learn the language the users use. They learn how users approach different parts of the design. They learn the goals of the users, and how the design fits into the users’ daily life.

Third, the team develops a design language to describe the differences between good and bad. Having the real experiences of real users as a common understanding breaks each team member of always referring to their own experiences. Instead of saying, “this is how I’d use it,” they can now say, “this is how Mary, who we saw last week, might use it.”

Fourth, because the exposure happens frequently over a long period of time, the team members see how their design attempts are working. It creates a feedback loop in their design process, where they learn when their design changes created the improved user experience they were seeking.

Innovation happens when you add value for the user. Teams with more exposure to how real people use their designs can more easily see opportunities for innovation. They can try new ideas to whittle away the users’ frustration and see how those ideas pan out. This makes them smarter and more informed, so they make better decisions in the long run.

I remember always being impressed with the pictures and talking about the Colosseum and the Roman Forum. Yet nothing could prepare me for my first trip to Rome. It wasn’t anything like the pictures.

I think this is exactly the same feeling that designers have when they visit their users for the first time. Sure, they’ve heard about users and what they’re like through the various sales and support channels. Maybe they even met one or two in their travels.

But seeing your users really work with the design is a whole different world. It changes the way you work and think about what you’re building.

In today’s UIEtips, I explain why I love taking teams out on their first site visits. It’s really exciting to see the transformation the designers undergo as they open up a new world. The metamorphosis they go through is really fun and exciting.

Read the article Why I Love Taking Teams On Field Visits.

If you’ve always wanted to go on your own field visits, you’ll want to get the best techniques from Steve Portigal, who is a master at these things. His User Interface 16 Conference full-day workshop will be a ton of fun. You’ll go on your own site visit and analyze the results – an amazing education. Check out how much fun it’ll be.

You can’t ask people what they want. They can’t tell you. The answer is almost always narrow in focus, concerned with the here and now rather than the future. How do you get them to give you the observations you need to design what they will want? Conducting field research to actually learn about your users can lead to innovative new ideas. Going out into the field provides real opportunities to see what the world surrounding your product is like.

Steve Portigal, Principal of Portigal Consulting, is an expert in conducting field research. He understands the value and unique insights that can come from observing your users actually using your product.

Creating a great user experience starts with field research, that’s why it’s one of the 8 workshops at this year’s User Interface 16 Conference in Boston, November 7-9. And luckily, Steve Portigal is presenting the workshop. Get the details on Steve’s and the other 7 workshops at UICONF.com.

Here’s an excerpt from the podcast.

“… If you are trying to change the game in a certain space, that’s well entrenched, you’d better have a more interesting approach to the field than to say, ‘well what would you want to see different?’

You have to be looking more broadly at people’s behaviors and their needs? What are educated people trying to do, and how are people solving problems? What are the entrenched challenges there? You need to use techniques to gather that information and make sense of that information.

It’s not a ladle that you dip into the soup, right? Scoop. Oh, here’s what people said they want. We’re going to go off and do it. That’s never a way to do breakthrough stuff…”

Tune in to the podcast to hear Steve cover these points:

• Should you ask people what they want or how they’ve molded their world around the products?
• What are the benefits of A/B Testing?
• What do you do when teams jump from observation to design solution without taking time in between?
• Should you do exercises or activities to translate the data that you’ve collected in the field?
• How should someone deal with the anxiety of being in a person’s home or workspace for extended amounts of time?
• What is the value of doing a “pilot” test?

Do you have experience conducting field research? Share your thoughts in our comments section.

Recorded: August, 2011
[ Subscribe to our podcast via ←This link will launch the iTunes application.]
[ Subscribe with other podcast applications.]

Full Transcript.

Jared Spool: Welcome everyone to another episode of the SpoolCast. I am very excited today because we get a chance to talk with one of my favoritest people on the planet, Mr. Steve Portigal. Steve, how are you doing here?

Steve Portigal: Great. Who can not be great when they’re framed in such a glorious fashion? So, thank you.

Jared: Awesome. Steve, you probably know this but everyone else might not, you are going to be teaching a full day workshop at UI16 this year. I hope you know this because if you don’t it’s quite a shock I’m sure.

Steve: DOING. Yes, I know it.

Jared: On immersive field research techniques. I’m very excited about this because we haven’t had a field research topic in many years and I know you and I have been talking about this for a long time. It’s really exciting to see people really interested in field research these days.

The amount of interest has been growing radically. And, I was just thinking about this the other day that I think one of the reasons is because no longer can we just depend on incremental improvements, you know, just fixing this feature a little here or running the usability test and cleaning up this dialogue box.

We now, for a lot of organizations in order to really have a full burst into the marketplace or to really have people pay attention they have to have something that is completely ahead of what their competition is doing and the way to do that is to go into the field and see what is being missed by the current products and offerings and what opportunities are there. I don’t know if that’s what you’re seeing in your work.

Steve: Yeah, very much so. I think that, as you say, implementable improvements they’re table stakes. As you say the competitors are going to be doing that at least. Lots of spaces are getting crowded, new spaces are opening up. I think what’s interesting is those are hard problems to solve. There’s not necessarily obviously clear what to go do in the “white space”.

So companies have these massive trajectories, they have momentum, they’re succeeding, many of them are doing really, really well and using design to kind of optimize within that. I think people know from experience, from recent trends that that can be a short lived advantage, that they have to keep working along the “innovation”.

I’m going to put quotes along every six words I say here. Along the “innovation” vector to try to continue to open up new spaces and stay ahead or to get ahead.

Jared: Right, yeah. I mean, one of the things that I’ve been seeing is that, you know, someone the other day on Twitter, I don’t remember who it was, posted that if you are always trying to catch up to your competition you’re always staring at their ass. And I think that if you want to get ahead you’ve got to do some of this research that really gets you out there, really gets you with your customer, really gets a chance to see what the total game is that is happening with where the products today are working and where they’re just completely missing the boat and then designing for that space as you put it.

Steve: There’s a great sort of, what I think of as a great myth around that that maybe we can talk about for a sec. Just the oft repeated idea that you can’t ask people what they want because they can’t tell you so that if you’re in the kind of business and design challenge that we’re talking about where you want to break through and innovate and reinvent something you shouldn’t ask people what they want because they can only talk about what is going on today.

I love hearing that because I feel like I have a good response to that. It’s a conflation of a few things. One is, let’s just say, looking more largely, doing field research to learn about people and asking people what they want. I think if this is not an area that you’re experienced in you think those are the same thing. You think the only thing you can do in field work is to say, “well what do you want?” and then go off and build it.

And most people would say that’s not an effective technique for learning new things. I agree with that on the face of it. If you, you know, are trying to change the game in a certain space that’s well entrenched you’d better have a more interesting approach to the field than to say, “well what would you want to see different?”

You have to be looking more broadly at people’s behaviors and their needs and, you know, what are kind of educated people trying to do and how are people solving problems? What are the entrenched kind of challenges there? And so you need to use techniques to gather that information and make sense of that information.

It’s not a ladle that you dip into the soup, right? Scoop. Oh, here’s what people said they want. We’re going to go off and do it. That’s never a way to do breakthrough stuff. So yeah, I agree when people say don’t ask what people want because they can’t tell you but I don’t agree with the implication of that which is don’t do research to try to innovate.

Jared: Well it’s interesting you put it that way because I took a team out on a bunch of field studies. This is a company that had been in business for six years, they had a very popular product, the customers loved them but they’d never actually been out to watch their customers use their product.

They handle support calls all the time so they know the things people call in for and they use it themselves all the time so they know how they use it but they didn’t and they hadn’t ever gone into the field and seen someone use it.

And I asked the head of development, I said what are you hoping to learn on this? He had a very interesting response. He said, “I’m really hoping to see all the ways that people hack our product, all the ways that they use it in a way that we never intended possibly because we’ve left some big, gaping hole out there or possibly because there’s a use out there we never thought of.
I want to see all of that.”

And I thought that was really open minded for him. It was really sort of out there. And you’re right that sort of, you know… was it Henry Ford? I thought it was Bill Gates who said don’t ask people what they want because if I asked people what they wanted they all would have told me they wanted a start button. Wasn’t that the quote? I think that’s the quote isn’t it?

But it seems to me that there are real opportunities to get out there and just see what the world around your product is like. It’s not so much that you’re asking people what do they want as much as how have they molded their world around the products that are there?

Steve: That’s right. That’s right. In that example and in your story about your clients you’re kind of talking about what’s the research question? What do we want to know?

I think implicit in your story is the business question which is: “what do we want to do?” So I can imagine your client saying we want to change, you know… direct resources toward changing the type of engagement we have with users or redesign the platform to take us 10 years ahead.

There’s obviously some strategic question that’s driving that and then the research question which you created with them or they created in their brief to you is a really helpful one. Then I can imagine your method just falls right out of that. Once you understand that there’s an interesting continuum there.

But you need to surface the business question. You need to surface the research question. What do we want to learn that’s going to help us answer that? Those are, I think, really important to draw the thread between those before you get to methodology. I think that takes a lot of expertise and so asking people what they want I think is just the naivest version of that.

Jared: Right, right. And I think, it’s a cop-out type of business work, right? That sort of, “well we’ll just ask them.” It’s up there with putting 1,000 little knobs and customization things into your design. It’s this way of saying I don’t want to take the effort to learn what these people want so I’m just going to put it out there and let them, in essence tell me.

And then whatever they say I’m going to decide if it’s a good idea or not and then do it. If enough people say it it must be a good idea.

Steve: Isn’t that called AB testing?

Jared: Yeah, yeah. I think AB testing falls into that a lot. Actually, AB testing has the down side that you never get tot hear any why. So all you know is that design 37 beat out designs 36 and 34, so obviously that might be what people want.

The other downside of AB testing is that people often use the absolute wrong measure to determine what is success. So they do something clever like well we got that person to sign up without ever asking if that person will use or value the service in any way that will be long term meaningful to either us or the user.

But we got them to sign up. And you know, we promised them money and their best sexual fantasies and they signed up. Guess what? So I guess our dishwasher repair service is now going to be a hit.

Steve: Unfortunately you know, year over year returns or loyalty, sometimes those fall to a different team. That are, you know, trying too… I’m sure you’ve encountered this all the time. We have our loyalty team working on a loyalty product and we have our conversion team working on sign ups for the product.

And this, the sort of silo in their design efforts based on kind of slicing and dicing, so to speak, the way that “the consumer” is using their products. Of course, no real person segments their experience in that way at all.

They don’t have a difference between interest, conversion, usage, loyalty. So these companies, I think, are trying to divide up hugely complex problems and you know, apply resources to them to try to own every facet of the experience.

I understand that effort but certainly when you look at the whys, as you bring up, the why question applies across all these different parts of the experience and you can’t think of these different aspects of the experience in that vacuum. It becomes very challenging.

To gain any insight about loyalty without gaining insight into conversion. So AB testing sort of proves how to get people to do a thing you think you want them to. We really like when we get to triangulate across methods.

Obviously there’s no one method that’s good for everything so you can find great clues, as you say, if you’re asking the right questions in looking at log data or other sort of very observational, kind of objective measurement things. Then you can get some narratives from people through different types of methods that help you understand why.

You’re not asking one research method to solve the other problem but if you put together a whole series of explorations in an ongoing way, you know, then you’re sort of doing intelligence gathering and you can tie that into insights. Then I think you can really start pushing your designs to solve the problems because you have a more broad-based understanding of them.

Jared: Yeah. I think that’s true. I think that’s true. Now, I’m interested in your thoughts around this idea. So a team goes out and they do research and, like you said, they triangulate their research. So they’re doing field stuff and they’re doing some stuff with their analytics and some other methods too.

All of a sudden they’re producing all this data. Observations and analytical number and all sorts of things are coming in. And it’s really easy for folks to just say oh look, people who see this screen are more likely to click on this button so we should design that way or when we went out in the field we noticed that people kept asking us for this so we should just build that.

And I’m wondering if you’ve encountered this sort of immediate jump from observation to design solution without taking time in-between and what you do about that when you see it?

Steve: Yeah. I think that’s a very serious concern and I think so many times we’re setting up an engagement and we’re kind of warned. Our internal gatekeepers say we have to think about how we tell so-and-so what we’ve seen because we don’t want them to go off and start doing things, that there’s kind of trains in the station that are charging up and ready to… I’m butchering a metaphor here, but…

Jared: [laughs]

Steve: They’re ready to burst out of the gates–there I go, I killed a metaphor–that they sense that around them, and they’ve seen knee-jerk reactions. I think what we do at the outset of a research project is identify what the milestones are and what the output is and what they’ll be able to do with that and try to have that.

Either it’s explicit in the proposal as part of the conversation that we keep having, because people are hungry. They’re hungry for something. Try to keep engaging them in being in the field and sharing fieldwork stories and sharing early kind of thematic things. But we very deliberately do not say, “And maybe you should do X as part of that.”

So in terms of our structuring of our communications and our reporting and so on, we’re trying to set that frame and set that expectation clearly. I don’t mean that’s sufficient to help structure that. It’s funny. I think of actually a counter example where people did act very quickly on…it’s the low-hanging fruit stuff.

And then when we’re out in the field with something, this is a detour on the way to answering your question, which I think is an important one. They were out in the field and watching someone set up piece of…I guess it was sort of an audio/video/computer hardware product.

The instruction manual explained how they should insert the, it was L-I-O-N lithium ion battery. But I think it’s like L, lowercase I, captial O, It’s like a very weird word. It’s a technology word. The person that we were observing just kept talking about the “Lion battery,” and of course had no idea what the “Lion” battery was.

Jared: Right. [laughs]

Steve: The person that was from our client side who was out in the field with us. I think he owns the documentation process. He went back and he took the word “lion” out of it, so it said, “Insert the battery.” It was awesome. I mean I was just so glad to see because they had huge, huge, huge, huge problems that we uncovered through this longer project that required a lot of design efforts to solve. But I was just really gratified to see.

To me that was just a low-hanging fruit. It was very obvious what the problem was. There was no sort of ripple out affect to making changes. It was “Let’s just take that word out.” It was just a nice little edit.

So one thing is that that could be done very, very quickly, like for him to open up a documentation management change order, so that guy owned it. The change was very, very quick. I think it could be rolled out fairly quickly into the next printing or the next run that they were doing of that documentation. So it was an isolated solution to an isolated problem.

Now the fact that they have used that kind of language in their documentation, and of course you know just having worked on these kind of projects that there were many, many, many things like that that were much more complicated and twisted that didn’t necessarily have such easy resolutions.

So I’m not of course excited about people, you know, jumping the gun on everything, but where there were some very clearly actionable pieces that didn’t require the report from the vendor of the research process, fantastic. It was really great. We were clearly being actionable and having impact and giving them examples at a detailed level in a more user-centered way.

That all being said, this was very, very complicated, and it took a lot of work with us all together to kind of unpack the research and make sure that we understood what exactly was going on.

It’s the difference between going from stories and anecdotes and feature requests to understanding something that looks more like a model or a framework or a continuum or a diagram or segmentation. Some way that you can visually or kind of informationally, if you will, include all this stuff together.

I think that’s analysis and synthesis, and that’s a thing that doesn’t necessarily come naturally to designers who are trained to make a translation between an observed need and a design solution. I love that ability the designer’s playing, and I always just want to just slow it down. I think we often run into the consequences of the failure to do that where you know these teams are sitting on a multifaceted compost heap of anecdotes and mythologies.

Jared: I’m writing that down. “Multifaceted compost heap.” That is like the best phrase of the day. [laughs]

Steve: They are looking at all this stuff. Depending what angle you’re looking at, it’s like, “Oh, people want this” or “So-and-so said this.” These myths get created, and it’s hard for anyone in the organization, let alone as a group, to have a coherent sense of what to do. They’re just looking this pile and seeing a glimpse in the sun at that moment.

People will have these pet stories that get kind of retold. Sometimes lurking within this heap are escalated larger-than-life anecdotes. The name of the difficult angry customer that gets repeated over and over again in every conversation. “Well, so-and-so-and-so’s going to…you know how they’re going to react to that.”

We worked with a company that had circulated a video. It was a kind of transactional tool that was often being used in high-pressure market-changing conditions. There was a video circulating of the hands of this person who was using this thing at a rate that you just wouldn’t believe. You’d think the video was sped up.

And you know, so I think on the face of it they’re doing this great thing, right? They’re circulating a challenging use case, but it reaches this kind of status inside, that it’s larger than life. You know, you’re trying to design for Superman, and part of the story of the Superman is that they can’t be designed for because they are so over the top in their performance.

So these “peering into this compost heap” people can kind of, in groups and cultures, create these legends that are trapping them more than anything else. And so you have kind of a divergent mess with these spikes sticking out.

None of it is representative. None of it gives you an integrated, holistic view of what the different types of users, what the different types of users problems are, what the different design strategies are, how to prioritize.

It’s never been folded into anything and kind of elevated up a level where it has structures to it and you can say well there are these different types of folks, here’s how they interrelate, here’s the kinds of problems we’re having, here’s the kinds of issues we’re dealing with.

This is not about… sometimes you’re just trying to reframe what the challenge is. We had a client that was dealing with an easy to install product and it was all about reducing time to install but when we talked with people about that value proposition in their use context they talked about smart.

Kind of in the smart technology, smart home, smart phone use of smart. They kind of pushed back our story that it was about smart. It wasn’t about reducing time. It was about reducing errors and saving me having to go back and fix the install. That was kind of an elevated framework about what is the benefit?

How does this thing that you’re doing fit into the way these people are thinking about their work and what they care about? So, and this is kind of a long ramble, Jared, but it’s about the needs to get from this compost heap, this sprawling mess of stuff where people are grabbing on individual pieces to something that is more holistic, unified, and you know has kind of action items coming out of it.

I’m waving my hands in the air, not that that would help anybody anyway. Doing that aggregation and translation is why, this is a long answer to your question, why do we not want people to jump off and start designing things right away?

It’s because we need to put it together into this larger, generative framework which sounds really smart and hard and time consuming but it is very doable. It just means you need to allow time in that process for that to happen and timing your brain for that to happen and defer or parking lot that jump to solution impulse.

Jared: So to defer that do you use exercises and group activities to get people to take apart the compost heap, as it were, and start to look at the different things and start to measure OK we’ve got the guy whose hands are really fast.

But from a bigger picture what are the other users like and are they like that or are they something different and do we have to design for a continuum of things?

Steve: Yes. I mean the short answer to that is yes. It’s an exercise activity. So you know, I think about doing research with users. There’s sort of these big chunks of activities. One is planning. We kind of already talked about trying too figure out what your research question is, what your business question is, what your methodology is.

There’s all the planning. There’s all the field work. So this is, you know, whatever kind of methods that you’re using, doing that and that is very immersive. That gets your brain going and gets you thinking about… You know, maybe you’re thinking about solutions. I think what we try to do is think about people and get stories about people and new perspectives on people kind of in the mix constantly.

And then the next step is analysis and synthesis. So there’s a phase of work that’s OK so we’ve finished being in the field. We have artifacts, we have experiences, we have video, we have transcript logs, we have reports to go into a new activity that says let’s disassemble all that.

What are even the axes which people are engaging in? What are the factors? What are the extremes? To dump all that and then to start to collate and organize and structure and prototype frameworks. Oh, it seems like there’s a relationship between the maturity of the user and the feature sets that they’re using.

That’s not a very good hypothesis, but it seems like there’s something here where there’s a relationship between these different factors. And it looks like, oh yeah, here’s how it breaks down. Well, no, that’s not actually true. Maybe it’s a relationship between these factors.

You’re sort of experimenting in a guided way. Your gut is driving you, your experience is driving you to look for meaningful ways to organize and structure this stuff. That’s the synthesis part. The analysis is sort of decomposing these stories and these fieldwork experiences into these elements and the synthesis part is putting them back in a new way and building up this framework.

Jared: I think I have an example of this. I was talking to Kate Brigham the other day from PatientsLikeMe and she was telling me that one of the things they’ve noticed was that when people first start using PatientsLikeMe, you know these are folks typically with chronic illnesses or they’re caregiver to someone with a chronic illness, they’re very much about just trying to discover who else out there is like them.

And they’re just, you know, putting in their data about how they’re feeling and they’re getting back data about whether how they’re feeling relates to how other people have felt in similar situations. They’re very much focused on that data and sort of initial connection sort of stuff.

Then they learned that as people use it more and more their attention shifts away from the data stuff and more into the community aspects. They start to have friends who they communicate with on a regular basis through their messaging capabilities. And those people, it’s less about their disease and their complications and it’s more about having these connections and these friendships so the functionality has to shift to this other stuff.

That was something that they observed in studying their user population that there was in fact a change in functionality as people sort of matured with the service.

Steve: That’s a great example, and you can imagine folks not as talented as Kate sort of looking at you know, some — it’s the compost heap again — some set of data, some set of feedback, and not understanding what it is that differentiates user input A and user complaint C to understand what to do about it. Logic is, half of our customers want this kind of change, and half of customers aren’t using this other piece.

Jared: Oh, yeah, well their community members are very, very vocal about the design of the site and tend to discount those things that are aimed towards new members, because they’re long past that. So the types of complaints they get in are very biased towards those more mature feature sets.

Steve: And now that they’ve built this — I’m going to call it a framework — there’s a journey of a user through their relationship with us. As you say, they have different expectations and different interests, and so there’s different designs. The design solution isn’t put feature in for this use case or that use case. It’s creating a product that changes with you or that evolves or that you can find your way through that.

I mean, there are a number of specific types of design solutions that can obviously be built from that, but you’re trying to take those observations and those inputs and build that larger story, at which point there’s some strategies. We’re doing a lot of — in our hand waving conversation here — presumptively on Kate’s behalf…

Jared: Right. [laughs]

Steve: We’re still far from creating design solutions. We’re building what the problem is. I mean, it’s not “we.” Kate did this, but you know advocating for what she’s done. Her team has identified what the design problem is, which is completely different than where they might have started if they were looking at the different types of inputs they were getting — the feedback, the observations, things that weren’t being clicked on. We have to make this button bigger because people aren’t clicking on it enough. Those are naive types of design solutions, because they don’t reflect the deep understanding that you’re relating that Kate and her team have produced.

Jared: Yeah, I think that this idea of slowing down and really trying to understand the problem before you jump to the obvious solution so that you can get a more deeper perspective is really a valuable thing that separates teams that are really good at what they do from those teams that are just really trying to be reactive to the world.

Steve: Agreed.

Jared: So one of the things that I hear from folks when we start to talk about field research is that they get really anxious about having to spend large amounts of time in a customer’s home or in their workspace actually talking to them versus from behind the safety of a double-paned, one way mirror with acoustic tiling, so that they can’t hear you giggle. That idea of being right there — that’s hard for a lot of folks. Do you find that?

Steve: I find it for myself, sure. I think the first interview or first sort of thing that we’re doing at the beginning of any study — I’m just incredibly nervous. You know? And I think this has a lot to do with confidence.

It has a lot to do with personality type. I’m an introvert. I think this puts me into an uncomfortable situation. First one is very scary for me. I actually remember having breathing problems a year ago — not that long ago, and I’m been doing this for a long time — kind of like going up to the door.

But you know, by the end, I mean, it’s kind of like riding a bike for me at this point, but definitely I know what that nervousness at the beginning is.

Jared: See, I thought it was just me.

Steve: We definitely can relate to people that are having their nervousness, right? It is — and as much as you plan for it, it’s always going to go in some way that you don’t expect. You know, I hate to be flip, but I think there is a “just do it” aspect to this.

What’s the worse that can happen in these kind of situations? If the worse that can happen is you get murdered, well there’s probably some reasonable planning that you can do in terms of screening your participants to keep that from happening. I guess the other worse thing is that you piss off a valued customer, and I have worked with financial institutions where we’re not just going to consumers, we’re going to their customers, and it was really charming and eye opening to see how they kind of brought a customer service mentality to research, and I think it was really effective.

It actually helped — it gave them a framework to work within. We’re not going out there to fix their car or something. We’re not performing a service but we’re learning about them in order to advocate for them, in order to do a great job for them, we want to represent them to the institution, we want to represent the institution to them, and that I think was a big part of their culture. I think that was really kind of helpful.

So you know, they didn’t want to piss off customers and have them leave that relationship. So those are worse case scenarios that you can be prepared for.

I think you can get no information or get misleading information, and that’s kind of a risk that you’re taking. And this is, I think, is where practicing, pulling down the information that’s available out there about best practices that you and I are making available that are everywhere, I think, in our field, are ways to mitigate that.

Trying it and reflecting on it — there’s lots of learning that you can do. Watching your videos of yourself do interviews. I just cringe every time I do that, or reading transcripts and seeing the stupid things that I say or bad ways that I ask questions.

Doing it and just have the experience and then reflecting on it, debriefing with your colleagues, talking about what you’d do different. Treating it like a learning process, you know I think it’s something — and also, I’ll just say that being uncomfortable or scared or out of your zone — it’s not the worse thing in the world. If it is, 90 minutes in somebody’s living room — you can, you know, just bear down and say, “OK. In 91 minutes I’ll be done, and I’ll see what I’ve learned from that.”

Because I can think of any number of times that I’ve just been uncomfortable or confused or had my own view of the world pushed on. Some people like to go on rides or go see horror movies. Those are, you know, getting a thrill by taking yourself out of your physical comfort zone or your emotional comfort zone. This, I think, you can look at it that way. I mean, it’s more deliberate and more meaningful, and it’s not for entertainment, but we do have analogs in pushing ourselves that you might look at this in that way.

Jared: Yeah, I think that there’s definitely something to that, and I think that also planning and practice just makes it a little easier, I think, for folks. I’ve found that having a good plan as to what we’re going to ask, so you’re not feeling like you’re improvising from nothing the entire time you’re out there. Of course, you want the conversation with the participant to be natural, but knowing what the goal of the session is and where you want to hit and what points you want to touch on helps a lot.

Steve: I totally agree. I mean, preparing a plan, writing up a plan, creating kind of the archetypal interview. This section takes 20 minutes. This section takes 30 minutes. We’re going to have these props or these activities for 40 minutes here. Checklists — I mean, this, again, depends on your personality type, but what does it take for you to feel confident?

Even if you don’t do anything like that, you’ve at least mentally prototyped what you think the session is going to look like. You’ve got buy in, so you know that your colleagues are confident, because they’ve had input into this. Some people I know do a pilot — because you’re basically hypothesizing that you can have an in-depth conversation with these exercises and these topics in this amount of time, and that you know how to ask about that stuff. So, whether it’s a colleague or friend and family, before you go out into the field, do kind of a participant number zero, and try it out.

Jared: My former colleague, a guy who helped us with a lot of our statistics and things in the early days, Will Schroeder, always used to say, “If you don’t do a pilot or a rehearsal, your first session becomes your pilot or your rehearsal.”

Steve: That’s so true.

Jared: I found that to be very much the case, so there’s a lot of value, particularly if it’s a really important set of sessions and if you’re really concerned that every session go really well, because either each participant’s really important or the people who are observing are really important or you just have so few that you have to make every one work. That’s when a pilot or rehearsal really, really plays an important role, I would think.

Steve: And sometimes you know, these are hard people to get to, so the first one has to be your pilot. But if you build an iteration — I mean, we were meeting with bankers recently and had lots of aspirational methods and we had Post-It notes and we were going to do timelines and get people to rank things and one or two interviews, and we basically didn’t even succeed in deploying any of our plan in the first interview. By the second interview, we were like, “OK. We’ve got to throw a lot of this out,” but we kind of sat down and talked through, “Well, what’s the iteration of our plan look like?”

So, it’s great to have a plan, and we weren’t — you treat it like a hypothesis that you’re testing. I think those first two or three sessions were extremely valuable because we learned what the topics were when we kind of went with it and we weren’t trying to force our guide on them, but we had an overarching architecture, I think, to try to work within. We then were able to just rebuild very quickly after those.

Jared: Yeah, I find that in our research that we’ve done to be the case. Tell me if you find it the same way, that as you do more sessions, the sessions mutate because you’re seeing some patterns and you want to explore them more, because they’re slightly different than the ones you saw before. You’re more attuned to things that are new and different, and maybe they’re things you haven’t covered before that you want to see if you can invoke in the session to see if you got any new responses to it. So there is a metamorphosis that happens.

Steve: Absolutely, yeah. There’s questions that you can’t quite get answers to, so you’re trying a few different ways and you’re re-purposing bits of improv from one into the next. If there are sort of issue here to him, that this is challenging for people and how to help them feel more like, hey, they can go ahead and do this. This isn’t so scary. Maybe just acknowledging that it is iterative, evolving, “metamorphisizing.”

Jared: [laughs] “Metamorphisical.”

Steve: Yeah. [laughter] And that you want to allow for that. You want to debrief. If you had five minutes to debrief about every interview, it would be, I think, two questions. One, what did we learn that surprised us now? And how would we handle the next interview differently? So that you’re debriefing on content — what are we learning? — and on process — how are we doing this? — as much as possible. Those are the two big things to really think about at the highest level.

Jared: That makes perfect sense. Well, everyone’s going to learn about how to do this stuff and get more comfortable with it in your full day workshop. I mean, you’re all going to go out and actually do some field research and then come back and analyze and synthesize the results, and I think it’s going to be a lot of fun. It’s going to be really a very cool day.

Steve: I think it will be great. I think people will be, hopefully, excited and surprised by how far we can get in a day in terms of playing with many, many aspects of this process.

Jared: So if you all want to come and hear Steve, what you need to do is go to the User Interface 16 Conference website, which is uiconf.com. The conference itself is going to be in Boston November 7th through 9th, and we’re very excited about it. It’s going to be a lot of fun. Steve, thanks for taking the time today to talk about all this stuff. This was a lot of fun.

Steve: Thank you.

Jared: And I want to thank our audience for listening, once again. It’s great to have you along with us. And as always, I want to thank you for encouraging our behavior. Take care. We’ll talk to you next time.

In our experience, one of the most powerful ways to gather important insights about users is the field study. By making direct observations, design teams can identify opportunities they may have never discovered if they had only conducted usability tests, focus groups, or surveys.

I think it’s essential for all designers to really understand how to conduct a field study and learn how to gather critical information about users. That’s why in this week’s UIEtips we’re reprinting an article from 2007, where I discuss the unique power of field studies.

Read the article: Field Studies: The Best Tool to Discover User Needs

We are so excited about this topic that we’ve invited Steve Portigal, a world-renowned expert in ethnography and innovation, to conduct a full-day workshop at this year’s User Interface 16 Conference in Boston, November 7-9. If you’ve never done fieldwork, or want to learn the latest techniques for extracting brilliant design ideas from your customer visits, you’ll definitely want to explore Steve’s workshop at UICONF.com.

Outsourcing your user research work is like outsourcing your vacation.
It gets the job done, but probably won’t have the effects you were seeking.

I usually say this when someone is asking me to do their user research for them. This is something we did quite a bit in the early days of UIE, but don’t do any longer.

Usually, they are asking us to do this to save time, because they don’t have trained folks, or because they are afraid of bias. All these reasons are rational, but there are better ways to deal with them than hiring someone else to do the research on their behalf.

As I said, I founded UIE as a company to do just this. I felt the rational reasons where why companies weren’t conducting their own research. I thought we could offer cost-effective, inexpensive research services to help. User Interface Engineering, in 1988 (it was our 23rd birthday yesterday!), was one of the first companies to make user research services available to other companies.

However, after working with hundreds of teams and providing their research, we started to looking at how effective we were. Were the teams’ designs getting better? Were they doing more research? Were they creating better user experiences?

We were sorely disappointed with our results. While every team told us they really got a lot out of our work, most weren’t improving their designs. They were appreciative of our reports, but hadn’t read them. They enjoyed our presentations, but weren’t really adopting the recommendations. And, most importantly, their culture didn’t change — they weren’t integrating users into their design process any more than before.

It wasn’t only UIE’s clients with this problem. We reached out to organizations using other outsourced user research services and discovered the same results. Hiring the work out wasn’t getting the job done.

We realized that we were missing an important variable in user research: the team’s direct exposure to their users.

When we take a team on a field research project, we introduce the team members to their users and having them spend time seeing them use the product and doing their work. In doing this, we’ve accomplished 90% of the work of the project.

It’s the exposure that changes the way people work. The same is true for usability testing or interviewing users. The direct exposure is the most valuable part of the project.

When you hire out your user research, even to the most competent of user research professionals, you’re losing 90% of the value. The research becomes a game of telephone, where the “away team” (to steal a Star Trek term) learns all about the users and somehow has to communicate back what they’ve learned. No mount of report writing or presentations can replace that lost experience.

Some UX service companies will tell you that they’ll remain part of the team, integrating the knowledge they learned into the design as the project continues. However, that creates an imbalance, where some people on the team know the users well and others have no idea. Those others, who will eventually own the entire design, are working at a disadvantage and won’t be making their design decisions using this critical knowledge.

This is why we now refuse projects where the team wants to outsource their research. We still do plenty of field visits and usability tests with our clients, but only if they come along to every session. If the client team isn’t there, we won’t conduct the session – there’s no point.

For the folks that think they don’t have time to do their own research: You’re better off taking the money you’ll spend on hiring someone and burning it in the back yard. You’ll get the same value in your product.

Seriously, if you want someone else to do your research because you don’t have time, you’ll need to dedicate twice as much time to spend with the researchers, extracting every little thing they learned about your users. Otherwise, you won’t get the value you paid for. It’s not a time saver to go this route at all.

For the folks who feel they don’t have the skills onboard: That’s an easy problem to fix. Training on user research methods is pretty easy. This is the bulk of our consulting work these days. We use a “Watch one, Do one, Teach one” approach. (We stole it from the medical training world). Most teams pick up the skills pretty quick and do a damn good job in just a few weeks.

And for those folks who feel doing your own research introduces a bias: You’re right, but it doesn’t matter. There’s always a bias in research, even when you get a third party to execute it. There’s nothing wrong with biased research, as long as you understand your biases and how to counter act them.

If there’s anything you can outsource, it could be participant recruiting. However, make sure you work with someone trained in UX recruiting, not market research recruiting. UX trained folks (we use Usability Works – they’re awesome!) know how to deliver the information they learn about your users in the process.

That said, you should even try to resist outsourcing your participant recruiting. You learn a lot when you talk to your potential users, even if they don’t qualify for the study. When you’re outsourcing it, you’re flushing a lot of great source material down the toilet.

Once you’re in the habit of doing your own research, you’ll never want to go back. It’s just too awesomely addicting and useful.

More and more, we’re finding ourselves in situations where the design just “has to be right.” No longer, can we just have incremental feature enhancements or small improvements in the design. Our users need to be wow’d and delighted. And adding large fonts in bright colors with rounded corners will only take us so far.

To truly delight our users, we need to dig deep into what is meaningful and valuable to them. Give them something that resonates and they will jump for our design.

We can discover those resonance points by taking our research into the field. We meet the users in their own environments, observing them as they live their lives and do their work. We bring back oodles of data, which, once we analyze and synthesize, we can reveal the delightful essence of new designs.

Steve Portigal has traveled all over the world to do just that. He’s spent thousands of hours in people’s homes, offices, and the other places of their lives, just to learn more about what will delight them. His work with design teams has taught them to mine their rich data sources and uncover a wealth of value and meaning to design for.

Steve’s full-day workshop at UI16 will take you through the entire process. Prepare for a hard day of work, which starts with a real field visit. You’ll bring back observations that you’ll work with for the rest of the day. Under Steve’s expert guidance, you’ll learn the best methods for interviewing users, analyzing the data, and synthesizing the key meaning.

You’ll be ready to head right into the field the moment you get back to your office.

See the other UI16 Spotlights:

You can catch the sneak preview of UI16 at uiconf.com. (And there’s still a few of the sneak preview $1,349 registrations left. Snag one while they are still available.)

These experts will dive deep and get to the nitty-gritty details that make any designer a UX pro. Look at the intensive full-day workshops we’re putting together for you.

• Brandon Schauer: Immerse your team in an innovative design process that produces refined design ideas in record time.
• Kevin Hoffman: Facilitate productive and insightful kickoff workshops to start your projects with everything you need.
• Hagan Rivers: Employ best practices to simplify your most complex applications with state-of-the-art UI techniques.
• Steve Portigal: Drive your design with effective field research to deliver innovative results.
• Bill Scott: Discover the latest rich interaction techniques for engaging user experiences.
• Kim Goodwin: Compose compelling stories that drive a realistic design process from start to finish.
• Stephanie Sullivan Rewis and Greg Rewis: Enhance your designs with HTML5 and CSS3 without sacrificing your design goals.
• Luke Wroblewski: Integrate mobile design’s best practices and techniques into your process.

Get more information on the workshop topics and speakers at UICONF.com.

Details on the Sneak Preview Site

In the next few weeks, more details about the agenda and workshop will emerge. However, you can get a view into our special sneak preview site now.

And because the site is in the sneak preview mode, we’re offering a sneak preview price – our lowest price, of $1,349.

On Wednesday, July 27 at 1:00 pm, registration will open.

There are 100 spots available at the special low price of $1,349. Once they are gone, they are gone and you’ll have to buy one of the more expensive spots.

There is a way to get a jump start on registration, and learn about exclusive offers and the latest UI16 news. Add your email to the UI16 list at UICONF.com and you’ll automatically get added to the priority group.

Now go see the 8 different design workshops we have in store for you at UICONF.com.

Pam Rodriguez and Luke Wroblewski did a nice job of posting their notes. Thanks guys!

• Steve Portigal on Design Fieldwork: Uncovering Innovation from the Outside In – Pam’s notes, Luke’s notes.
• Kate Brigham on PatientsLikeMe: Adventures with Data Visualizations – Pam’s Notes, Luke’s Notes.
• Luke Wroblewski on Designing Mobile Web Experiences – Pam’s Notes.
• Mike Lee on AARP: Designing a Strategy for Organizational Transformations – Pam’s Notes, Luke’s Notes.
• My presentation on The Essential Principles behind Great Design Principles – Pam’s Notes.

As you can see from the #UIEWAMT Twitter stream, everybody had a great time and we all learned a ton.

There’s one more stop on the 2011 tour – Minneapolis on June 27-28. Use the promo code BLOG and get $100 off the registration price.

See you there!

UIE Web App Masters Tour:
Get $100 off the Minneapolis Masters Tour with the promotion code BLOG.
Minneapolis

Great design principles guide your team to creating designs that delight your users. They are well thought out and often based on research. Having a set of great principles will allow your team to turn ordinary design into extraordinary design. But not everyone has great design principles. What even constitutes a great design principle?

Jared Spool understands what it takes to create good design. In his talk The Essential Principles Behind Great Design Principles, from this year’s Web App Masters Tour, Jared dives into our latest research on what teams are doing it right and which are missing the mark.

Great design principles are born out of research and observation. If you sit and have a meeting to discuss “what you want to be when you grow up”, you might end up with vague or convoluted principles. You’re not basing your decisions on your actual users’ experiences. Conversely, the Windows 7 desktop team collected a large amount of data on the failings of Windows Vista to develop their principles and by all accounts, turned out a much better experience.

Creating a set of great design principles shouldn’t be about striving for vague goals like being “universal” or “clean”. They should be developed to solve problems and address real issues with your product. Principles are the guide to getting your team to the project’s end goal and fulfilling the vision you had for it. As Jared says, “if you know where you’re going, it doesn’t matter how fast you get there.”

Conducting field research and gathering data can help you determine your top priorities. Once you make that determination, you can then take steps toward developing design principles that will ultimately guide your team in making great design decisions.

Don’t miss Jared in either of his two talks at the Web App Masters tour. The tour is making stops in Seattle in May and Minneapolis in June. For more information on Jared and all 11 Masters, visit UIETour.com.

UIE Web App Masters Tour:
Get $100 off the Seattle Masters Tour with the promotion code BLOG.
Last Stop! Minneapolis

Recorded: March, 2011
[ Subscribe to our podcast via ←This link will launch the iTunes application.]
[ Subscribe with other podcast applications.]

Steve Portigal of Portigal Consulting uses the methods of synthesis and ideation to approach this crucial next step. During his virtual seminar, Steve explains that synthesis is the process of turning field data into insights and then how you move to ideation to turn insights into solutions. So many questions came up during the seminar that Steve ran out of time to answer them all. He tackles the remaining questions in this podcast.

Here’s an excerpt from the podcast.

“…If you were ranking things against something that no one cares about you’re not really deriving value from that process. One thing that I think has been important for us is to align on the factors before we go into the ranking section, so that we know what we’re going to do.

When I say align, I mean that it’s us and the people that are going to be in that session. We’re doing idea generation with a large group, but we’re doing ranking with a very, very small group. My favorite would be a maximum of three to keep it kind of tight.

So I think we’re trying to understand and facilitate discovery with our stakeholders. And then understand their measurement of success. What are their business goals? And I think there are a handful of obvious ones around feasibility and cost and kind of investment and payoff.

I think those feel vaguely standard to me in terms of, how do businesses perform and how do businesses make products. I think more what I would like to get out of that facilitation is to kind of tease out the nuance. I think I gave the example in the webinar of a criteria that one team we worked with had and that criteria is about feasibility.

So, when I hear that, to me that means, can we build it? Is this Star Trek technology or is this something that we can actually build? And when we kind of tease it out, feasibility for them was actually about regulatory stuff. It was about legal feasibility…”

Steve addresses these additional points in the podcast:

• How long does it typically take to get from the first interview in the field to the opportunities?
• How much do you involve the client in the analysis process?
• Should you deliver both good and bad statements so that it’s more palatable to your sensitive stakeholders?
• Who is the top-line report really for?
• Is it dangerous to present the top-line report to the stakeholders?
• Can the opportunities translate into specific user requirements rather than solutions?
• Should collaborative analysis happen with the end users?

Do you have experience analyzing user research data? Share your thoughts with us in our comments section.

Recorded: April, 2011
[ Subscribe to our podcast via ←This link will launch the iTunes application.]
[ Subscribe with other podcast applications.]
[ Transcript Available ]

Subscribe to our podcast via

At UIE, we receive a steady flow of questions about user research. There is a staggering amount of information out in the world, just waiting to guide your designs. Best of all, it’s nearly free for the taking. We’re happy to reveal the key to unlocking this information:

Ask the right questions.

Getting out into the world and actually interacting with real people who use, or potentially could use, your product or service is incredibly valuable. We tell our clients this constantly: the organizations who are most successful are the ones who are on intimate terms with how and why their customers use their product. But how? To answer that question, we invited our friend Steve Portigal, principal of Portigal Consulting, to conduct the UIE Virtual Seminar, “Deep Dive Interviewing Secrets: Making Sure You Don’t Leave Key Information Behind”

Steve’s specialty is informing design decisions by getting on the ground and speaking directly with customers. And sharing how you can do the same. Today, we release the interview Jared Spool conducted with Steve after his seminar, following up with a number of additional audience questions. You can enjoy this interview without first seeing the Virtual Seminar, but afterwards, you’re going to want to see it. You’re in luck. You and your team can still access the recorded seminar.  (See special offer below)

Jared asked Steve, in the end, what does the interview process really boil down to for it to be effective? Steve’s answer?

You have to really, really listen.

Jared and Steve discuss several points about the interviewing process, drawing on stories from both their experiences. Here are some brief highlights:

• Understand the why behind what people are saying. This improves designs.

• Transcribe your interviews. They can be used as a deliverable for a client and they allow you to critique your interviewing technique. Steve uses Chromolume Transcription, but there are many options (see note at the bottom of this post).

• How do you deal with uncomfortable situations, like when an interviewee’s supervisor wants to observe your interview? Steve thinks this is often an indication of a failure in the planning process. When you’re in the field, there are a couple of tricks you can use to help steer the situation towards productivity.

• How do you deal with interviewees who ramble? Try not to interrupt, Steve suggests. This may be their natural inclination and you should try to roll with it. Try to keep them on track with very focused questions.

• How do you end an interview when you discover you’re talking with someone that won’t be helpful? Hopefully your planning will avoid this, but even so, you may find later that the person’s insights help you more than you thought. Stay in the moment.

• Can you interview well into the product design stage? Sure, you can even bring prototypes. Ask, “how does this work for you?” “How would you teach your (parent/significant other) how to use this?”

• How do you deal with difficult interviewees? There’s no saving some interviews, but you should attempt to build a friendly rapport that exudes professionalism. Many times they’ll open up.

Have a listen to the podcast. You’ll certainly pick up some tips for the next time you’re planning research or are out in the field.

Special offer:

For a limited time, you can gain lifetime access to this seminar at a special price. Purchase the recording before May 21 for just $99 (that’s $50 off the regular pricing). Anyone in your organization will be able to watch Steve’s seminar whenever they want, as often as they want.

Note, as an aside,

Steve mentions having had some trouble with the transcription service, CastingWords. For what it’s worth, we use CastingWords at UIE for our transcriptions, and aside from the occasional issue, we have been happy with them. Also, they now offer time stamps, for an additional fee.

(Oh, and by the way, our last event sold out, so you’ll want to Register your team early!)

When you spend time with your customers, it’s an opportunity to learn how to move your design forward. You don’t want to leave important information “on the table”—information that can give you a more complete understanding of how to move your vision forward. You might act on incomplete detail that creates risk when it forces you to guess what the users need. Worse, the partial insight you have may take your design team in the wrong direction.

User research is an expensive endeavor. Make sure you’re prepared to get the most out of every minute that you’re with your users. Come home with a deep insight into their thinking, their lives, and how you can change their experience for the better.

Steve Portigal will show your team the art of asking the question. You might visit the user in their office or home, have them come to you for a usability test, or even have a chance encounter at a trade show or while waiting for an airplane. Do you know what to ask? Do you know what to listen for, to extract the critical detail of what they can tell you about your design?

Steve will help you prepare your team for any opportunity, be it formal user research or less structured, ad-hoc research. He’ll also give you tips on how to work with your stakeholders and executives, who may also be meeting potential customers and users, so they know what to ask and how to listen—integrating their efforts into the research team. (Wouldn’t it be great if they understood why you’re doing what you’re doing?)

Get your team asking good questions, the right questions, with this fantastic seminar. Honing this skill will be a great addition to their Toolbox. Register your team before January 19, with the promotion code TOOLBOX, and I’ll also send you the link to a fabulous webinar Kate Gomoll did for us, Field Studies: The Ultimate Tool in Your Usability Toolbox.

Are you prepared for meeting someone who could be using your next design? How do you make sure you get into their head, learn what their life is all about, and get the information you need to build something truly innovative and delightful? We’d love to hear your ideas and about your experiences below.

Naturally, our clients’ teams start by focusing on their direct audience. They look to make the best experience for these folks, to make creations come to life quickly and effectively.

It doesn’t take long, once they start researching how their direct users are taking advantage of these tools, to see that not every creation represents the tool well. Quickly, the goal of the team becomes to expand the scope to enhance the lives of what we call the downstream users.

In moving the design focus from the direct audience to the downstream users, we’ve found having robust personas and scenarios helps tremendously. The field research behind the personas involves both the direct users and the downstream users. Scenarios are often interconnected between personas, since the stories describe the direct user’s relationship with their downstream users.

The most successful of our clients in this endeavor have had great luck when they’ve put together a solid vision of what key downstream users experience, then talk about the idealized vision for the direct customers’ development process to reach that experience. The combination of the two viewpoints becomes very powerful.

It’s not clear when “quick and dirty” became a dirty phrase in the usability world. There are those that believe that testing must be scientific, and that takes time and money — luxuries not often available to many development projects.

Usability testing expert Dana Chisnell knows what it means to work by-the-book – she co-wrote “the book” (The Handbook of Usability Testing, 2nd ed.) with Jeff Rubin. In this seminar, Dana will break down the process of collecting user research data, exploring the must-haves, the nice-to-haves, and the certainly-can-do-withouts. You’ll learn how you can answer your essential design questions using methods that would make MacGyver proud.

This presentation is perfect if you have yet to conduct your first usability test. If you’re experienced with testing, Dana will show you some new ways to inject user research into those tight-on-resources projects that keep cropping up.

Register today at http://www.uie.com/events/virtual_seminars/wild/

What questions do you have about Usability Testing in the Wild? What tools or tricks have you used to maximize the resources available, and still deliver quality results? We’d love to hear your thoughts and experiences.

Duration: 22min | File size: 12.5 MB
[ Subscribe to our podcast via iTunes. This link will launch the iTunes application.]
[ Subscribe with other podcast applications. ]
[ Text Transcript ]

Each week in our Usability Tools Podcast, we will be sitting down to discuss tips and tools for improving your site’s user experience. The goal of our weekly podcast is to share some of the most important findings from UIE’s research on web design and usability.

In this week’s podcast, Brian Christiansen and I continue exploring usability techniques for web-based applications. Web-based applications are different from content-based web sites because the users are involved in a transaction. When we’re researching the usability of a content-based site, we’re focused on how users will find and react to the information. However, with web-based applications, there are many other considerations that we need to account for.

If you missed the first part of the show, you can listen to it here:
Usability Tools Podcast: Useful Web App Usability Techniques, Part 1

This week, we explore the usability technique toolbox, focusing on those methods that help us with web-based applications.

In this episode we start with the basic usability test, move onto variants, then talk about field studies. In each case, we explore the web-app specific advantages and talk about how we get the information we need to make informed design decisions.

We talked about several books in this episode:

• The Handbook of Usability Testing—by Jeff Rubin
• A Practical Guide to Usability Testing—by Ginny Reddish and Joe Dumas
• Paper Prototyping—by Carolyn Snyder

As always, we’re very interested in hearing from you. Do you have questions or comments about this episode? We love to create shows based on your questions. Please leave a comment below or email us directly at mailbag@uie.com

UIE’s Latest Research: If you’re interested in the topics we discuss in the podcasts, I highly suggest you sign up for our free newsletter, UIEtips, to read our latest usability and design research as soon as we publish it. We’ll also notify you in UIEtips when we publish new podcasts.

Each week in our Usability Tools Podcast, I will be sitting down with UIE’s Managing Director, Christine Perfetti to discuss tips and tools for improving your site’s user experience. The goal of our weekly podcast is to share some of the most important findings from UIE’s research on web design and usability.

This week, Christine was tied up getting ready for our upcoming UI12 conference, so Brian Christiansen joins me for a show whose topic came from our reader, Colin. He wrote in to ask about the statistical significance of usability or field tests.

Statistical significance revolves around having enough participants to make your findings valid. However, the number of participants necessary can vary widely, depending on what you’re studying and how.

During the podcast, Brian and I answered these common questions:

» Many people talk about statistical significance, but what does it really mean?
» How many people does it take to produce reliable results?
» Why do usability tests require many fewer participants than surveys?
» How do we know we’ve gathered enough data from our research?

As always, we’re very interested in hearing from you. Do you have questions or comments about this episode? We’d love to create a show based on your questions, just like we did with this episode. Please leave a comment below or email us directly at mailbag@uie.com.

UIE’s Latest Research: If you’re interested in the topics Christine and I discuss in the podcasts, I highly suggest you sign up for our free newsletter, UIEtips, to read our latest usability and design research as soon as we publish it. We’ll also notify you in UIEtips when we publish new podcasts.

New: Survey and listener drawing!
We would like to give you and your co-workers free admission to our next Virtual Seminar program, with full, lifetime access to the archived program as well! All you need to do to win is give us your feedback on your podcast listening experience. Fill out the following survey and each week we’ll randomly send one survey participant a free admission to the next UIE Virtual Seminar and Archive, a $169.00 value! We appreciate your input!

Participate in our survey to win!

(Over at Yahoo!, they use something similar for the personas they create.)

A Flickr set with more images of these cards

Here’s an image of a set I made here at UIE, which I modeled after baseball player cards. These don’t need to be fancy, in fact I made this set in about ten minutes. Here’s how we do it:

1. We start with images of our users. We snap these during our field visits or when people come in for testing. We assure them that the images won’t be used for anything commerical.
2. Second, we use the OmniGroup’s excellent OmniGraffle to layout the cards. If you don’t have OG, you could use something from your Office suite, for instance PowerPoint would be fine. You could simply divide a slide into 6ths to create your cards.
3. We dropped in our images.
4. We added labels our cards with as much or little info as necessary. Here we used their “user number,” which is how we tracked them through testing and their first name. We wanted the cards to be as easy to read from a medium distance as possible, so a group could discuss them while strewn about the conference table or pinned to the wall. Sometimes we’ve played games with them, quizzing each other by holding up two cards and asking what user needs they have in common, or contrast how their needs are unalike.
5. We printed them out on card stock. Snip, snip.

(Extra credit would be to make two-sided cards with more stats about the user, if that’s helpful for you. You could accomplish that by producing two cards per person, print, then cut and paste them together. )

The next step is deployment of the tool: determining how many copies of which cards to make and letting your collaborators decide how to keep themselves reminded. Some groups stick them around their displays or cube walls, for example.

We are using CastingWords.

They produce transcriptions in plain text, Rich Text Format and HTML formats. They give you access to a handy RSS feed which allows you to download your file when its ready. They also notify you via email when the files are complete.

You may find it interesting that their company is built upon Amazon.com’s technologies. The prime tool they use is the Mechanical Turk.

We use their transcription service for just about anything we have an audio file of, from podcasts to recordings we make during field research. Converting them to text makes them easily searchable, both for people on the web looking to find good content, and for ourselves to find information from a field excursion. Since we use Macs in the office, we can take great advantage of Spotlight‘s ability to catalog within text documents (.txt, .html, .rtf, .doc) to find information inside hours and hours of information almost instantly. The transcriptions themselves become both our content and metadata for the audio files.

Are you using transcriptions to try to get a handle on your audio information? Have any tips to share?

Kate Gomoll is president and founder of Gomoll Research & Design. Back in March, we hosted Kate for a UIE Virtual Seminar on Field Studies. During the live presentation, we received many questions about the field research techniques Kate discussed. To delve into these questions in more depth, we recorded a follow-up podcast where I ask Kate about many of the pressing issues associated with field study research. We thought Kate’s insights were so useful that we’ve decided to share them with all of our SpoolCast listeners.

Among the issues Kate and I discussed,

• How can design teams demonstrate the immediate value of field research to management and stakeholders?
• What are the best tools to analyze user data?
• How can design teams integrate field studies into their research plan?
• What are the best ways to put study participants at ease when you are taking photographs or video of them?
• How does tag-team interviewing work?

If you’ve never conducted a field study, I highly encourage you to listen to this podcast. Kate Gomoll is the expert we recommend clients turn to with their questions about field research.

You’ll also want to check out Kate’s new report, The Field Study Handbook: A Common Sense Approach for Discovering User Needs. Because field studies are such an essential tool for design teams, we asked Kate, along with Ellen Story Church and Eric Bond, to assemble this comprehensive guide. (There’s a promotion code that will earn you a discount, but you have to listen to the end of the Podcast to get it.)

You can also purchase the recording of Kate Gomoll’s Virtual Seminar here: Field Studies: The Ultimate Tool in Your Usability Toolbox.

The most valuable asset of a successful design team is the information they have about their users. When teams have the right information, the job of designing a powerful, intuitive, easy-to-use interface becomes tremendously easier. When they don’t, every little design decision becomes a struggle.

While techniques, such as focus groups, usability tests, and surveys, can lead to valuable insights, one of the most powerful tools is the field study. Field studies get the team immersed in the environment of their users and allow them to observe critical details for which there is no other way of discovering.

While field studies are one of the most expensive techniques to implement, the value they return is tremendous. We’ve never come back from a study thinking we’ve wasted our time and resources. A quality study can produce enough information to keep a team busy for months.

In this week’s feature article, UIE’s Ashley McKee has conducted an excellent interview with Kate Gomoll, a recognized Field Research expert. In the interview, Kate shares how she and her team at Gomoll Research & Design conduct field studies.

Read today’s UIEtips article.

Has your design team conducted field studies? How have they worked for you? As always, I’d love to hear what you’re doing. Join the discussion below.

[Whether you're new to the benefits of performing field research, or a seasoned researcher looking to brush up on your research techniques, you'll definitely want to get your copy of our latest UIE Fundamental Report: The Field Study Handbook -- A Common Sense Approach for Discovering User Needs, written by Kate Gomoll, Ellen Story Church, and Eric Bond.]

Our Product Management team at iRise is constantly reaching out to our potential customers as part of an ongoing “Customer Needs Analysis” campaign. We engage usability experts, interaction designers, information architects, business analysts and other software definition participants in observation sessions that let us get closer to our potential customers’ needs.

If you are interested in allowing someone from our Product Management team to spend a day unobtrusively observing you and talking to you over lunch, then please contact me at peter@irise.com. We are willing to consider any location in the U.S.

If we end up engaging you for an observation session, we will provide you with a gift certificate to Amazon.com or your favorite restaurant and a trial license for our iRise Studio simulation product.

I bet Peter and his team learns some great stuff.

Field studies aren’t hard or expensive. Are you thinking about them?