August 8th, 2014
Lately, Jared Spool has been mulling over what he defines as deliverables and artifacts in the design process. The idea is that deliverables are more authoritative and complete, whereas artifacts are more conversational and exploratory. Scenarios are an important part of the design process and Jared was curious where they might fit in. So he enlisted Kim Goodwin to chat about it in this podcast.
Kim is the VP of User Experience at PatientsLikeMe. She’s also an author and expert on personas and scenarios. She believes that where you are in the design process defines whether scenarios are a deliverable or an artifact. The size and culture of your team is also a factor. A smaller team has less of a need for formal deliverables.
However, in larger organizations scenarios and personas serve as a great way to get everyone involved in the same frame of mind. Bringing stakeholders to interviews with users at the start of the design research helps solidify that the personas used to inform the design are shorthand versions of real people. This gives you a solid foundation to move forward with the design.
Jared Spool: Hey, you’re listening to another episode of the SpoolCast. I am very excited about today’s episode, because we are going to have a chance to talk with the most amazing, the most wonderful Ms. Kim Goodwin, who we have had at oh so many of our events.
The reason we keep having her back is because she is the smartest, most brilliant person we know on getting good products and designs out. She is going to talk at the User Interface 19 conference, which is going to be October 27-29 in Boston, Massachusetts.
She’s going to be giving a full day workshop there on scenarios and how to design with them. We are just so excited about this. We’re going to talk to her about that today. Hey there, Kim.
Kim Goodwin: Hi Jared. You sure know how to pump up a girl’s ego.
Jared: Thank you. You deserve every word of it. I thought today we could talk about something that’s been sort of rumbling around this casing of a skull of mine, which is this distinction that I’ve been working out between what I’m calling artifacts, and deliverables, in the design process.
Let me explain what I’m talking about, and we can talk about how scenarios might fit into this, if they do or they don’t. Which is, a deliverable is something we hand to our clients, to our dev teams, to the people we’re working with, and it becomes a resource that we use for the rest of the project, at least some major portion of it to come.
It could be wireframes, it could be usability reports. Things like that. An artifact is something we put in front of our stakeholders, in front of our users, in front of our peers, with the intent that we’re going to explore some idea around them.
Deliverables have this sense of authoritative completeness, whereas artifacts have this conversational stimulus to it. First, let me start here. Does that distinction resonate with the way you think about design process? Do you think that these two things are there?
Kim: Hmm? That’s a good question. I think there is a meaningful distinction, and I think it’s one, as a professional, we maybe don’t make enough. I think there’s a lot of focus on beautiful deliverables, for lots of reasons, and maybe not enough on of the interim artifacts.
It’s interesting, though, that you say artifacts have a conversational component to them whereas deliverables are authoritative. It depends on your role.
If you are, say, a consultant from outside, something that’s just a conversation piece might be one of your deliverables. In fact, if you’re following a good process it probably is, whereas in-house may get framed less as a deliverable, right?
Jared: Yes. For instance, an early draft of something might be more in the artifact category, but in its later drafts it might be more in the final deliverable category. I don’t know. I’m still working through the idea.
Kim: It’s a fuzzy line, but I think it’s a vague but important distinction. If it’s just an artifact, if it’s just a thing that’s meant to make an idea more concrete so that other people can discuss it, then, I think there’s less pressure to make it perfect and precious, which sometimes we can do with our deliverables, right?
Jared: Right. A whiteboard sketch of an architecture, for example, or a concept diagram might be more in the artifact category, whereas that same diagram in a PowerPoint slide that is saying, “This is what we figured out we need to build and we’re all in agreement to this,” is more in the deliverable category.
It’s a concept map and, in fact, it may not revolve from one to the other very much, but the way we use it, the context of use, I think, probably changes dramatically.
Kim: Yes. One of the determining factors in my head would be that context of use. I have this little construct in my head that I think of as the avocado model of teams. Avocados are delicious, why not compare teams to them?
Jared: I’m wondering if eventually it evolves into a guacamole model.
Kim: Maybe, after a long project, it does.
Jared: Things get mushy in my head.
Kim: It’s true. In my head, the way I think of it is the center of the avocado, the big hard brown nut there, is the generative part of the team. Typically, just a couple of people working pretty much hand in glove, creating ideas, driving synthesis about the people’s ideas. Not to say that ideas can come from outside that pair or threesome, but somebody’s really got to pull that together into a coherent solution.
Beyond that, you’ve got some close collaborators. Maybe that’s a couple of core engineers you’re working with if one of those people isn’t in the center of the avocado, a product manager, a subject matter expert.
Who’s in which part of the avocado is going to vary depending on your organization. It’s a small handful of people who need to review your work, probably every few days, to make sure you’re really on track and you’ve got everybody’s expertise incorporated. That’s a great group of people to look at artifacts because they don’t need a lot of explanation, they have plenty of context.
When you start to get to that outer skin of the avocado, the more distant stakeholders, the people funding the project or folks who may be our major decision makers but aren’t that involved in the day today, need a lot more explanation than you could include in something that’s just an artifact.
Those are the people you need the PowerPoint presentation for. To my mind, how close to that day today is someone is really what determines whether you need an artifact or more of a deliverable for that conversation.
Jared: That’s really interesting. That’s not something I’d considered before. This idea, that part of what distinguishes what I’m calling artifacts versus deliverables, is how much outside explanation you need to be able to look at this thing and derive and provide insight in reaction to it. Fabulous! That’s really neat.
This gets me to the question I wanted to ask from the very beginning which is, if we talk about scenarios and personas, do they fit into either category or both categories? Maybe what you’re telling me is they are, in fact, both an artifact and deliverable.
When you have them as a deliverable, you have to put more story around them, more explanation, whereas when they’re in an artifact mode, because of the people you’re talking to and where you’re talking to them in the process, you may not need to do that.
Kim: I think you’re right that they’re both, really. This is one of those things that depend on where you are in the design process, on which part of your audience you’re communicating with, and also on the culture and the size of the organization that you’re working in.
If you’re working in some tiny little startup, you have a small team, you’ll sit across the hall from each other, around a big table, or whatever, you don’t really need a lot in the way of formal deliverables. This is one of the arguments that only new ex-folks make. It’s absolutely true. You don’t need formal deliverables when you have the “two-pizza team,” as people call it.
When you have some enormous team of 40 engineers, 50 engineers or more than that and their spread across a couple of continents, many time zones, and varying skills on the team, some people who are really experienced and people who are not that experienced, that’s when you have to get much more formal about deliverables.
They have to involve a lot more explanation and be done with more detail.
Jared: That makes perfect sense. It occurs to me, as we’re talking here, that we haven’t really defined personas or scenarios. Maybe we should take a step back and just introduce those people for which these ideas may be a little new for them.
Kim: Sure. A persona is an archetype. For any given product or service, there are several different behavior patterns that you’re designing for. If it’s an enterprise setting, chances are those patterns have a lot of similarity to roles although they’re usually a little more complex than just a role. You’ll often have several personas within a role.
In the consumer world, those behavior patterns tend to center more on attitudes, goals, and things like that. For any given product, you might have two or three personas, six or eight personas. You could have more than that if it’s a really complicated enterprise tool.
Their set of behavior patterns and goals distilled out of ideally qualitative research, you can certainly supplement that with quantitative research, represent a type of user. They do so in a more humane way than just a user profile would because they’re, essentially, the stars of stories, which brings us to scenarios.
If a persona is your character, the scenario is your plot. Imagine putting your character in a situation. For any product or service, you generally are going to have multiple scenarios, situations that your personas will encounter, that are going to unfold in different ways.
The scenario is the ideal world version of that situation. If you put a magic black box in the hands of your persona, what would happen in that situation? How would that story unfold? Then, we use that to guide the design going forward.
Jared: I liked that you pointed out that scenarios and personas were about behaviors. A lot of people fall into the trap where they make them about census data, job titles, and things that are not necessarily behavioral.
Three people with the job title “nurse” may have completely different sets of behaviors as to whether they’re a nurse in an emergency room versus a nurse in pediatrics doctor’s office versus a nurse in a nursing home.
Kim: Absolutely. I think that people will often conflate personas and roles if you’re looking at a business context or some non-consumer context because they sound like similar constructs, but context, skill level, those definitely make a difference in how many different personas you have.
For example, if you’ve got an IT guy in a small company versus one in a large company, the guy in the small company is responsible for the phones, the database, the network, probably the copy machine, anything that plugs into the wall is his problem and his level of expertise with any one of those things is probably relatively low.
Whereas in a big enterprise, that same IT guy is just responsible for one system.
Jared: Just the servers or just the routers.
Kim: Right. Job titles can be really deceptive that way, but that’s very important that personas are much more nuanced, much more about behavior and demographics are just not that enlightening in most cases when it comes to personas.
I see a lot of people with huge persona sets of 20 people for 1 fairly simple product and they’re making a bunch of meaningless distinctions based on things like demographics that don’t really matter when in reality maybe they have 3 personas.
Jared: I’ve also seen teams – you can tell me if you see this too – which have the two personas: new users, existing users, and they bundle everybody into those two categories, again, without taking any consideration that there might be different sets of behaviors to look at within people who might be exposed to the product for the first time versus people who’ve been using it for a while, but some people use it every day and some people use it infrequently and what they do with it is different and all those things.
Kim: I’ve definitely seen that one. I don’t see it as much as the overemphasis on demographics though, interestingly enough.
Jared: Interesting. You mentioned earlier Lean UX, and for people who might only have a vague notion of what that is, Lean UX is this – I don’t know what you would call it. A movement? A variation? A dance? A song? Genre?
It’s a way of approaching the design process that deemphasizes long-term deliverables and focuses on faster iterations and movement with small teams, like you said. The teams that could be fed out of two large pizzas happily.
Is that how you would qualify it? Is it something similar to that?
Kim: I think so and like any emphasis on a method, it’s one of those things that works great in some context and I think is less appropriate in other context. There’s no one right way to do UX. It’s all about assessing what’s the culture you’re in, what’s the size of team, and the type of problem you’re working on and picking the right tools from there.
Jared: Let’s get back to this idea of scenarios as discussion pieces for artifacts. Is there something that has jumped out at you recently where you were like, “Oh wow. Because we had this conversation about the scenarios of our users, we were able to move this project forward in an interesting way that we wouldn’t have gotten to otherwise.”
Kim: I think it happens all the time. I think that very often I’ll see a designer who maybe has a single image of a static screen and then we have a conversation about what’s the scenario here? How does this actually unfold? How does this screen change over time? That identifies a hole in the design, just in a design critique session.
This happens a lot with stakeholders looking at a design and saying, “Why didn’t you design it that way or why don’t you have this feature or this field or whatever?” Then we can have the conversation about what’s the scenario in which you’d use that, and by the way, which persona is that for, and sometimes people realize that’s kind of my pet idea.
I guess it doesn’t actually make sense. It’s good for containing scope and opinion and sometimes it’s great for coming up with whole new ideas, but I think really that goes back to not just the scenarios as a tool, but the research that informs the scenarios and really looking at what is that user’s whole experience? How would they frame what their experience is?
Because I find a lot of teams, particularly those who are focused on agile tools, like user stories, they’re so focused on making their scope very small that they’re not necessarily thinking about the entire breath of the flow from a user’s perspective and very often the user experience is more at the epic level in agile terms than at the story level.
Scenarios really look at the whole picture, that whole experience, part of which might even be outside of your product and that’s where I think the big wins are.
Jared: Thinking about this. A user story might be as a customer I want to tell the eCommerce system that I need to return this particular order, but the scenario could possibly talk about the larger context of why I’m returning the order. Maybe I ordered the wrong thing in the first place and maybe instead of fixing the return system, we should fix the way I chose the product.
Kim: The scenario would actually start with, “Huh. I need to buy a thing. Let me find the thing. Did I get the right thing?” And then would follow you through the return process because all of that is interconnected because when that persona arrives at your website, he or she has certain expectations about what you already know about her.
If you only start with the return process, does that persona remember when that purchase was made or what are all the things we should already know about her? What should we be anticipating and how could we, of course, ideally prevent that return?
A scenario really looks at that whole process of the relationship between product or service and person.
Jared: That makes perfect sense because in the olden days, if you wanted to return something to Amazon, you had to go find the Amazon order number for that product, which was printed on the piece of paper that was in the box that I may or may not have kept.
Now I just review my orders and there’s a return this button. I never actually pay any attention to the order numbers. That element of the design has become almost useless to me, even though it’s essential to the underlying database. I don’t need to know it because I’m looking at pictures of the products I’ve bought. It’s obvious which one I want to return and I never think about the order number.
Kim: That’s exactly right and in fact, a scenario about that purchase from Amazon would also include the fact that at some point you get an email that says, “Hey. Review your experience with this product or with this Amazon market seller,” because we’re not just focused on the one tool.
If we think about the way organizations are created, chances are – and I’m speculating here – there’s probably somebody at Amazon who’s just worried about those follow-up emails and that’s their piece of the system that they’re responsible for.
If they only thought about that part of the interaction, they’re missing opportunities to really make that a more integrated experience, to make it more delightful. To minimize my effort as a user by taking their knowledge about me from one stage of the process to another stage.
And so now you start there, and you can divide up your scenarios into much smaller pieces based on what your specific product responsibility is or what user you’re using on that spread or however you chunk your work.
Jared: So returning to this thing that keeps gnawing away at me about artifacts, it sounds to me like what you’re saying is that I could have one scenario that will actually be a discussion multiple times through my development process.
Because I keep coming back to it. I keep saying, “But what about the scenario where the person buys the thing and then returns it?” And I may be working on reviews, I may be working on the original purchase part, I may be working on the order processing, I may be working on applying credits.
I’m working off the same scenario each time. Does it happen that people might, as they dive a little bit deeper into the problem, start refining and reflecting on that scenario a little bit more?
Kim: Absolutely. I would say that scenarios happen at several degrees of granularity throughout your project. So early on, at the stage where you’re being optimistic and you’re not so worried about constraints, because you ideally want to spend at least a couple hours there on any given project.
That first scenario is totally broad. It’s end to end process, across multiple channels if you’re in the e-commerce world or something like that. Could cross multiple platforms, and it’s optimistic.
Then, as you proceed, you’re going to scope that scenario down a little bit because there are constraints in the real world, and the crazy thing you imagined that sounded great, you’re not going to be able to touch that for another two years, right? So we’ll leave that alone.
Then we might focus on just part of that scenario, and then flush it out with a bit more detail. And as we start to design that part of an interface or interaction, then we can put more details in as we start to make implementation decisions. And so, certainly we’re adding color to that.
We’re also, as the project proceeds, doing variants. So initially, we want to pick some very typical but very specific situations to use to illustrate our design, but not worry about all the edge cases. Over time, of course the design has to ultimately accommodate all of the weird things that people do, or even all of the semi-frequent things that people do.
But we don’t want to drive the core of the design. Gradually, we get more and more into those as well. So it’s interesting, now that I’m thinking about your deliverable versus artifact distinction.
And I would actually add a third in there, which is that sometimes scenarios are deliverables, where we write out the prose, and we have beautiful storyboards, and we put it in a PowerPoint or a document, and we use it to educate lots of people about the design.
Sometimes this scenario is just what you’d call an artifact where it’s some bullet points we’ve outlined on the whiteboard and some storyboard sketches. But sometimes, there’s even a lighter weight version, which is not even an artifact. It’s just a conversation.
So very often, somebody will have a sketch on a whiteboard or a set of storyboards on the whiteboard and say “OK, that looks like it works pretty well for the scenario we were talking about, but what if this happens? What if this is the situation? How does the design adapt?”
And we talk through that as we sketch, but we don’t ever even document it. We’re still using scenarios as a design tool, but there’s not even a written scenario artifact of any kind. Does that make sense?
Jared: Yeah, yeah. So it’s just an oral discussion of what happens for this thing. And maybe that thing is a story we keep hearing from the support people. I would imagine technical support is a great research resource for those edge condition scenarios.
Kim: Yeah, sometimes. We don’t get to those until later in the process. The early scenarios are much more informed by, I find ethnography to be a really great tool for starting those.
Jared: Yeah, I could definitely see that.
Kim: You have a lot of breadth from people, and you really focus on those key situations and just spending time with real users, getting them to tell you their stories, that’s the perfect input to scenarios as design tools.
Jared: So, let’s explore the deliverables for just a few minutes here. At some point, using your avocado model, which I really like, we need to make that sort of broader group, the folks who are on the skin or outside, maybe they’re the onion and the tomato that goes into the avocado, right?
Kim: That’s a tamale.
Jared: Yeah. So, these people need to figure out who have we been designing for, and what do we think these people do, and how did we map? So now, we’re sharing deliverables, I could actually imagine a presentation where I’m actually there to present the design that I’ve created, but I start by talking about the personas and the scenarios to give context to the design.
So in some ways, the deliverable at that moment isn’t truly just personas and scenarios, but it’s this combination of here are the people, here is what they need to accomplish, and here is the design that we have imagined that will help them imagine this.
Maybe we compare it, the same scenario to the old design so we can see where the pain and frustration is. And we can compare and contrast, so that you can see how this new design has improved it. Is that something that makes sense?
Kim: In general, yeah. I think it varies a little bit depending on the type of design you’re doing, whether it is more green field design or more iteration.
Jared: By green field, you mean?
Kim: Inventing new stuff. That’s a little bit different from iterating something that exists right now. So, the first thing I do before I say “Here’s a persona in a scenario” is “Here is the real humans we talked with. Here’s what we heard from them.” Ideally, you all joined us for those interviews so that they’re real to you.
And that way, the personas are not this thing that we made up out of whole cloth, but instead they’re a shorthand for the real people we’ve all met. So, one thing that I like to do is drag stake holders and team members along on user interviews.
Because then, if they join three or four user interviews and you’re doing more than that, they will see enough similarity and also enough difference in those interviewees that when you talk to them about what you heard in other user interviews, they believe you. Just for a way that they don’t if they didn’t participate in any of the interviews.
Jared: Right, and if they’ve only gone on one, then they think everybody is just like that one. So you have to actually do more than one, right?
Kim: Exactly. That’s why I always push people to join, say three to four interviews, because that’s enough to give them a flavor without them latching on to that one person that, by the way, happened to be a little bit odd and think that that’s the whole world.
Jared: Having them see three or four interviews, doesn’t that give them a healthy respect for the complexity of the problem? Because the odds are those three or four people are going to be so radically different from each other that they’re going to go “How the hell are we going to design anything for this variation?”
Kim: [laughs] Well, I think actually three or four interviews is enough to see both similarities and differences. Because they’ll say “Oh, I’ve heard this theme come up over and over again.”
Jared: Yeah, people keep talking about that.
Kim: Yeah, when we start to say “Here are the big trends.” They go “Yep, I can see that” or “I didn’t hear about that one, but I believe all the other ones so by extension I’m going to believe the one you’re telling me about now.”
Jared: Adds a lot of credibility to that, I could see that completely.
Kim: And they see the differences, so they certainly do gain a respect for that. So ideally, you want to get people bought into all that research before you ever show them personas. I see a lot of teams that say “Ta-dah! Here’s our finished persona set.”
Jared: The posters on the wall.
Kim: And posters on the wall are fine, but they should be reminders, they shouldn’t be introductions as much as possible.
Jared: That’s a really interesting point, yeah. So I guess you’re doing it wrong rule. I guess I see the personas, the first time I think about these things I haven’t had any research, so I’m getting introduced into the personas that way.
Kim: Yeah. I’m cautious about the phrase “You’re doing it wrong” because I think there’s a lot of different circumstances out there, but I would say that if you’re a close participant in the team’s decision making, then yeah, you probably are going to be a lot better off if you have had some first hand experience with the research.
If you have a giant team of 100 developers, you can’t get everybody out there to do the user research right away. And so sometimes inevitably, people will need them for the first time. But as much as possible, that core team, that creamy center of the avocado, you really want to get them involved in the user search.
Jared: But I’m wondering if, going back to the deliverable thing and looking at the outer skin of the avocado, if it even makes sense. Before I start talking about personas, I actually start talking a little bit at least about the research. We went out, we visited this people, let me introduce you to some of the real people we met and the real problems they had. Now, let me show you the personas we created based on those.
Kim: Exactly. So, if you are trying to get people bought into your design for the first time when you show the design, you have already lost.
Jared: Interesting, OK.
Kim: Because buy-in starts, not just with the design. It doesn’t start with the personas. Actually, I don’t think it even starts with the research findings. Buying, in my experience, starts with defining the research plan. So, when you get the stakeholders around the table and you say “I have some thoughts on who we should talk to, what do you guys think?”
That’s a really important conversation to have with all the right stakeholders, because if you come back with research findings and someone says “Oh, but you didn’t talk to this kind of person”, you already lost the argument.
Jared: Right, right. Those are not our real users, you need to talk to our real users.
Kim: Exactly. So you’ve got to start buy-in with the research plan. Even introducing personas and scenarios, you need to start earlier than that with checking in with the stakeholders, making sure that everybody’s aligned.
Jared: Yeah, this all makes crazy perfect sense. I’m loving this stuff, Kim.
Kim: It’s good stuff. It’s always fun to talk with you about this, Jared.
Jared: Good, it’s good. Well, I’m very excited about UI19 because you’re going to come in and you’re going to do a full day workshop where you’re going to walk us through exactly how to build these things out, and how to use them in our work. That’s really exciting.
Kim: It’s always a great conference, I look forward to it.
Jared: And that workshop that you’ve done, you’ve done it a few times for us now, and every time it’s one of the highest rated workshops we have, so I’m so glad you’re coming back. So I’m looking forward to it. So for those of you who would like to attend the workshop, it’s going to be in Boston in October, October 27th to 29th.
It’s called the User Interface Conference, you can find out more information at UICONF.com, and we’d love to see you there. It would be really awesome to have you there. Hey Kim, thank you so much for spending the time talking to us today.
Kim: Thank you Jared, my pleasure.
Jared: And I want to thank our audience. Of course, you can check out the conference site, you can learn more at UIE.com. And you know what? If you have a chance and you really like these podcasts, if you could take a moment and visit the iTunes library and leave a little rating there for us.
And even if you don’t like it, you can go ahead and do that too, but particularly if you like it we would love to hear your feedback and see what’s going on there. So take a moment, do that. Thank you for listening today, and as always thank you for encouraging our behavior. We’ll talk to you soon. Take care.