Jared Spool: Welcome, everyone, for another episode of the SpoolCast. I am very excited. We have Todd Zaki Warfel with us, who did a great virtual seminar for us a little while back called "A Practitioner's Guide to Prototyping." Todd, how are you doing?
Todd Zaki Warfel: I'm doing well today, Jared. How about yourself?
I'm doing fabulous. So, during the virtual seminar, you went through your eight guiding principles of prototyping. Do you want to refresh us as to what they are?
Todd: Sure, sure. So, there's eight guiding principles for prototyping. And the first one is understand your audience and intent. Second one is plan a little, prototype the rest. Third is set expectations. The fourth one is yes, you can sketch. Number five is, it's a prototype, not the "Mona Lisa." Six is, if you can't make it, fake it. Seven is prototype only what you need. And number eight is reduce risk, prototype early and often.
Jared: I really liked how you sort of summed them all up. And as we were going, we got a lot of really, really interesting questions, and we didn't have a chance to answer all the questions during the session. So, I thought what we would do today is, starting with those principles and going with what you talked about, answer some of the questions that came up.
And the first one that attracted my attention was from Bruce, and he asked about the intersection between prototyping and specification. One of the things that was sort of a theme through a lot of the questions was this whole notion of how you plug prototyping into the process. And maybe you could talk a little bit about when you prototype and when you specify and where do they meet.
Todd: Sure. For us, we actually use the prototypes as our specification, for the most part. Now, there are some things that you're not going to see or that maybe won't be self-evident in the prototypes. Some of the business rules or back-end functionality may not really be clear in the prototype.
And this is actually one of the reasons why we turned to prototyping and away from an older, traditional method of wire frames with written specification documents. What we found is that, since the prototype, basically, is show and tell, and allows you to see the story as well as tell the story and actually play around with the systems, it's much more tangible. When we do prototyping, we find that, actually, any specifications that have to be written are dramatically reduced.
So, for example, in my book, there is a case study from a gentleman over in the UK, and their old, traditional system was do some wire frames, write a 200-page specification document and deliver it out to the development team. And they shifted over to using more of a prototyping model. And for similar systems that they used to have to write a 200-page spec document, they found themselves delivering the designs, with specifications, about three times as fast and that the specifications went down to 20 pages instead of 200.
And so they're essentially using the prototype as the bulk of the specification and then writing some supplemental documentation to describe things that aren't self-evident, like back-end business rules and maybe some technology-type stuff. And we've done a very similar approach.
So, a lot of times, what we'll do is prototype out maybe a core flow, plus maybe some air sessions and maybe some success screens and that type of a thing. But, we won't typically prototype out every single scenario. We'll kind of do the 80-20 rule. So, here's 80 percent of it prototyped out. You can pretty much see how it works. And then any additional, supplemental information that may not be self-evident in the prototype, we'll either write some documentation, or, in a lot of cases, actually, our clients just take the prototype and then their internal team basically writes that spec.
So, for us, it's actually replaced the specification, in a number of cases completely, and in cases where it's really, really complicated systems, then it's replaced probably 80 to 90 percent of the specification.
Jared: That's interesting. Have you played with using screencasts or videos as part of your specification process?
Todd: Yeah. In fact, actually, just recently, so probably a few months ago, we started doing this, where we'll take some of the prototype, and actually, we'll mock it up, we'll build it, and then we'll actually go through and screencast ourselves going through one of the flows or one of the scenarios, then basically take that video file and either send it to the client or post it up on Basecamp or some type of a project site and show it to them.
And the interesting thing about that is we can send them over the prototype with a description, "Oh, and then if you go through this scenario," or we can walk them through it through a virtual screen-sharing session with Adobe Connect. And that can take like 10 minutes. Or we just send them the 15-second video clip that actually shows the demonstration of the thing working. And we've found that video is incredibly effective for that type of thing.
Jared: Yeah, we've been doing that. We also, for some clients where the video is a little hard to work with, once we've made the video, take screen captures out of the video and then insert them in a page with some text annotations, just basically the equivalent of sort of captioned screenshots, the way you would do in advertisement storyboard or something.
So, if you've ever used the Amazon affiliates program, when you're surfing Amazon and you've signed in to the affiliates program, there's a little tool bar that kind of slides in and out? So, this is a very similar thing. There's a little tool bar that basically slides out across the top of your browser window. And once you display the tool bar, then all of your Post-It notes basically display on top of the prototype. And so you can put notes on there. I can put notes on there. Anybody who's viewing it can basically put notes and then show and hide them. One way that we've accomplished that is with a clickable prototype.
Sometimes we'll do things in Fireworks. And in Fireworks, what we typically do there is just add an additional layer. And the same thing: we actually have this little Fireworks symbol that is a Post-It note that's 80-percent transparent so you can kind of see through it. And so we'll add a bunch of Post-It notes on top of some layers, and then add some notes and then just toggle those on and off.
So, those are some other ways that we've kind of built in some supplemental documentation or specs into a prototype.
Jared: Yeah. I think these tools are getting so easy to make these prototypes document-able that specifications, I think, to some extent, are going away. But, Amanda asked an interesting question, which is, "If you're prototyping only some of the functionality, how do you talk about the rest of the functionality that isn't in the prototype such that the team knows how to fill in the gaps?"
Todd: That's a good question. I would say - and this is probably more of a process thing than it is a prototyping thing. The way that we operate is we evolve our prototypes.
So, to answer that question, if it's at the very beginning, where that functionality is not built in but we intend to build it in, typically what we'll do is we'll talk about it a little bit with the development team or whoever else is on the product team, so it might be a product manager or another designer or somebody. And we might talk about what that functionality might look like, we might sketch it out really quickly with them, and then discuss how it's going to go eventually into the prototype, and then we'll probably schedule that in at some point. So, we keep a backlog file where we'll add those types of additional features of functionality in and set a date for when they're going to go in.
If, on the other hand, it's something that we just don't ever intend to put into the prototype, either maybe we don't have enough time to, or we've discussed it with the development team that's actually going to implement it, and they're comfortable with what we've got in filling in the gap, then a lot of times we'll do that through sketching.
So, anywhere where there are holes, then we'll have a discussion with the development team, and we'll typically sketch out and typically do like these storyboarding, but we'll storyboard out where those gaps are and handle it that way. But, again, it depends on where we're at in the process.
If it's early on in the beginning, and we intend to put it in, then we'll sketch through that with the development team and schedule to go in. If it's something where we're near the end of the prototyping process, and there's really no intention to put it in, then again, we'll just sketch it through with the development team, and if they have some guidance at that point.
Usually what we'll do then is follow up with them once they've implemented it, and they'll shoot us an URL to look at to just validate that they took the right approach or not.
Jared: What's sort of coming out here is that the prototype is not just this sort of end result deliverable. It's not this thing that you go off in secret, and you build, and then you back and you say, "Here it is, chaps. Build this."
But, instead, it's this interactive artifact that, as you're building it, you're working with that development team, and you're having the conversations that tell you how to design it, and because you're having those conversations, the things that are not covered in the prototype, they're learning how to design through your conversations.
Todd: It was interesting when I first started writing my book, I was focusing on the prototype and kind of prototyping with the [inaudible 0:09:32] being the prototype, and it was fairly early on that I stepped back and took a look at what we were doing with prototyping.
And to your point, while there's this artifact that's there, really for us it's more about the process. So, really it's prototyping is part of a design process, and the prototype is just... honestly, it's almost like an ancillary artifact that just happens along the way.
So, we do a lot of sketching through the prototyping process, and we try and involve the development team very early on in that process. I really liked the way you described it as far as they learn along the way, and that's really what we're trying to do.
So, whenever we engage a client in a new project, one of the things we say from the very beginning is, "If we do our job properly, then we're not going to be here two to three years from now. For us, and as much as we would love to work with your guys, for us this relationship is kind of like raising a kid."
"We're going to send you off to college someday and hope you go out and explore the world and make something of yourself. And you can check back in from time to time, but we really don't want you moving back into the house."
Jared: [laughs] I need you to tell that to my kid.
Todd: We tell them, "Some kids do move back home, but we're hopeful that you don't move back home, but that maybe you just come back home for the holidays," right?
So, for us, what we do is, for all of our clients, we use prototyping as a way to not only create a design, but also to teach them a design process, and to teach them how to continue to do this stuff and how to have better design practices and improve the overall usability of the product to make it more consistent and much more predictable far beyond anything that we can reach.
I mean, one of the web apps where we're designing for a client right now, I think it's 250,000 lines of code. There's no way we'll touch every single corner of that product. The development team will. We might touch maybe in total 25 to 40 percent of that product.
But, what our intention is, along the way we're going to create a number of patterns, and then their dev team can learn from those patterns, and then they can take those patterns that they've learned through this prototyping process with us, and then essentially spread them out across the rest of the application.
Jared: Yes, but do you think that part of learning those patterns, right... you know, it's one thing to publish a list of patterns, and say, "OK, we've got this wiki. It has these patterns in it. Go off, design something," right?
That's sort of like saying, "Hey, here's this place called, 'Hope Depot.' It's has everything in it you need to build a home. Go build your home."
But, because you're in the prototyping process, and you're working through the patterns, and you're trying things out, and they're failing, and you're trying other things out, and they're succeeding, that act of being part of that trial and error learning process, that's more than just learning the design process.
That's learning what about these patterns works and what doesn't, and that stuff, it's really hard to communicate in the pattern descriptions themselves.
Todd: It's funny, because with this client I was talking about, this big web app that we're redesigning right now, we are entering the third iteration of the design process for them, so we're tackling a third part of the product.
With two rounds behind us, we're just now actually putting together a pattern library for them, and in doing so, we're doing fairly standard things like having an image plus a description and what is it, when to use it, when not to use it, and there's only so much that you can do through that written documentation.
And so we're actually considering, not initially, but longer term, actually incorporating some video in there so they can see the demonstration, which is something you and I have discussed. Video can be really powerful.
But, again, that'll only get you so far. It's not until you start playing with that pattern and implementing it and actually try to put it into something where you discover what's working and what's not working.
The interesting thing about this company is, the product that we're working on right now, I think their total dev team was maybe five or six people, like their core development team was maybe five or six guys. It's a mid to entry-enterprise application. So, in their core team is around five people that have been building this thing for the last decade, and they're quadrupling their investment in the number of development people they're putting on staff.
So, they're going to bring 10 to 12 more developers in who have desktop application experience but no web app experience, so for those guys who've never done this stuff, that pattern library is only going to be marginally useful.
The five guys who've been with us for a year working through these rounds and working through the prototyping process, although they're both going to have the same documentation, the guys who've actually been down in the trenches and gone through this process and been there with us side-by-side through the whole prototyping process, they're going to be light-years ahead of the 10 to 12 new guys that they're bringing on.
Jared: Yes. I think that's really a key thing about prototypes is, really it's an under communicated message as to this benefit, which is the immersion in the prototyping process is really where 90 percent of the benefit comes from. It's not the actual final interactive deliverable.
Todd: Yes, not at all. We're actually prototyping some internal applications right now that we're working on, and it's funny, because we sketch all this stuff out, and we think, "Oh, yeah, this is totally going to work!"
And our sketches are probably the equivalent of what other people would use as wireframes. They serve the same purpose for us, but they're sketched on paper. For us, well, I would actually argue it for just about anybody - those sketches and those wireframes are honestly, just theoretical at that point. It's all theory. It's, "I think this is going to work."
It's not until you actually get in and start prototyping that functionality where you're playing around with things that you really discover what things are really going to succeed and what things are going to fail.
And a lot of times those failures that we discover fairly early on in the prototyping process lead to what we call, "happy accidents," which is, this thing failed, but it basically creates this new opportunity for me to go try something else.
And for some of these internal applications that we are prototyping right now, just today actually, we were going through some HTML prototypes we're working on, and then started looking at some of the sketches, and we discovered some things that really were some gaps in the functionality, and those gaps were opportunities.
So, we're like, "Hey, this is awesome. Let's go back to HTML prototype and let's add this piece of functionality. Let's hide this, and then we can just benefit that way."
Jared: Yes. Amy Cueva, the principal over at MadPow, she just tweeted to today, a really interesting thing. Just said something along the lines of, "A design is just hypothesis until you put it in front of users." And I think that prototypes let you take that design out of the hypothesis stage and give it a chance to actually see if it's proven.
Todd: I totally agree. Like I said, for us, and I think really you could argue for just about anybody in our own practice, that those sketches, which, again, are kind of like wireframes for other people, but for us, in our practice, it's the sketches.
It's all theory at that point, and it's not until we actually build something tangible you can play around with, we're putting that theory into practice and seeing if it's really going to work.
Jared: Well, that brings me to Bill's question, where he asks, "How are you able to do usability testing when the prototypes are not refined, or they're missing pieces?
Todd: I love that question. It's funny, so, in some cases we'll intentionally leave holes in the prototype for testing. So, since they're not complete - I mean the very nature of a prototype is, it is complete - we actually use that to our benefit.
If we bring a participant into our office to do testing in person - or nowadays we do a lot of testing remotely - we'll actually kick off the testing session letting them know, "By the way, today we're going to show you a prototype. The prototype is incomplete, so there are going to be some areas of the prototype that aren't working, and if you come across those, you might click on something that either doesn't go somewhere, or might create an error."
"That's OK. Again, it's a prototype, so there's probably going to be some problems with it, or some holes, some gaps. If we come across one of those sections, then what I'll probably do is pause you for a second, and let you know that OK, you've discovered one of those areas where there is a hole or where it's not complete. At that point, I'd like you to actually walk me through how you think that would operate."
So, we actually use that to our advantage, where if there is a hole in it, that's a great opportunity for us to say, "Well, OK, that part's not fleshed out, so maybe you could tell me how you think that actually would work."
And then we watch for patterns through the participants to see if, hey, five out of eight participants said they would expect "X," then that gives us some pretty good design ideas.
Again, the very nature of a prototype is that it's not complete. Every prototype that we test is broken or incomplete in some way.
Jared: I'd be afraid in those holes, though, that the participant in the study would come up with this sort of sci-fi fantasy ultimate design, but I guess if you're listening for the right thing, you don't so much pay attention to what they think the design would be, but so much where their mind is going in terms of expectations for the next steps.
Todd: The way that we typically approach it... I think it was maybe four or five months ago, we were doing some usability testing on one of those "private members only" sales shopping sites, so we were doing some testing on a prototype for that, and part of the shopping cart piece wasn't flushed out. There was just a hole.
And we knew somebody would clicked three steps into the shopping cart process, and the last piece was just dead, like it wouldn't go anywhere. And we knew that going into it.
Participants would hit that section, and we'd say, "OK, so you've just discovered one of the areas that's actually not flushed out in the prototype, and so what I'd like for you to do at this point is maybe you could just describe to me what you would expect to happen next."
And so we really try and focus more on from a functionality and flow standpoint, and really we approach it with, "If you could maybe describe to me what you would expect to happen next." That's really what we're after is their expectations for "happen," not, "What do you think it would look like?" So, it's really not about the look and feel, it's really more about from a functional standpoint, what would happen next, not how it'd look.
Jared: That's pretty cool. It reminds me of a technique that Steve Portigal taught me, which is, in some of his studies, he will have pairs of people build a design for something, and he doesn't pay so much attention to the actual design they end up with as much as he pays attention to their conversation as they negotiate which things are important and which things aren't. And this feels very familiar to that same sort of thing.
Todd: Yes, absolutely. When we're basically tracking our notes and observations in a usability test like that, when we go back and look at them, what we're looking for are behavior patterns, and we try and... I hate to use the term "interpret, " but essentially, we're really not looking so much at what they say as really what's behind what they're saying, like what's the goal of what they're saying?
So, it's not so much really the actual words that the participant is saying, but we try and look at the words they're saying and also try and go back and look at the video recording, and what we're really after is the intent behind what they say and what their goal is.
Jared: Along these same sort of lines, the folks over at SmithBucklin, they have to trace their requirements, right? So, they've got this list of requirements that have been produced by their business analysts and other folks in the organization, and they actually have to document where in the design those requirements are being met. Is that an example you use Protonotes for, or do you have other techniques for that?
Todd: So, two things, and this is by no means meant to offend or upset anyone who uses that older school method of requirements and then design. We try and go the other way around, where we're designing something, and then actually the requirements are written off of the design. And again, a lot of that has to do with, because we're using the prototype as the documentation, then that method works a little better.
So, one thing I would probably turn that approach around, and instead of designing off of some requirements that a business analyst wrote, I'd encourage them to try and get a designer to work with the business analyst and actually design the thing, and then have the analyst write the requirements off of the actual designer prototype.
If, on the other hand, you're in an environment, and some of us are going to be in this environment, where you just can't do that, or while you're trying to get to that approach, you're still kind of going along the old method of, "I'm handed some requirements. I have to go design something," then we may use something like Protonotes for that.
We also borrow an approach from the Agile Community, which is basically, keep a backlog that tracks all this stuff. So, a lot of times what we would end up doing in a case like that is maybe pulling those requirements into a backlog, which basically can be anything from a proper backlog tool to just something like a spreadsheet that just has a list of all these pieces of functions that have to get built in, and then you schedule them out in one, two, three, or four week cycles. So, you basically build a project plan.
So, what we do in a case where we're handed some requirements, we'd put those into a backlog, we'd schedule them out, and then with each release, we'd review the prototype, and we would show in the prototype, "Requirement 1.6, let me show you where that is. Requirement 1.7, let me show you where that is." So, that's an approach that we might take for something like what they're talking about.
Jared: Just to be fair, it's not always business analysts who are just generating requirements. Sometimes it's like regulations change, and you have to adapt a system to meet the new regulations.
Todd: If you're doing and working the financial industry or for government, Sarbanes-Oxley is out, "whoo-hoo!" Yes, there's all this new stuff that we have to go comply with, so a lot of times we take those requirements and fold them into a spreadsheet, create a backlog out of it, and then we go in and bite off 10, 12, 15 of those at a time and put those into release, and then we'll show in each release where those have actually been captured in the prototype.
Jared: One of the interesting questions that came out of the this was from the folks over at Evantage Consulting, and they wanted to know what you thought about the idea that prototyping allows you to take risk. I mean we often talk about how prototyping lets you reduce risks, but does it give you an opportunity to actually take risks?
Todd: Oh, absolutely. In fact, it allows you the opportunity to take risks and at the same time reduce risk. So, what you're doing is, in prototyping, you can go out there, and you can take risks. I don't know about this... you can kind of take risks with very small risk, right?
So, you can go out, and you can try something, and if it fails in a prototype, the amount of investment, time, effort, money, that total investment that you've probably put into the prototype is really small and insignificant compared to if you had done that in production, right?
So, I don't remember where I've seen this, but there was this graphic I saw at some point over the last couple of years, and it maps out the cost of fixing something in production versus the cost of fixing something in design. And it's something along the lines of, to fix it in design is maybe 10 percent. To fix it in production, the cost is 100 percent, right? And I think it was Frank Gehry that said, "You can either fix it on the whiteboard with an eraser or on the construction site with a sledgehammer." Right?
Todd: So, it's the same idea. Essentially, the great thing about prototyping is, and again, the approach that we use is the rather agile approach with very small incremental pieces that we're building out.
The advantage we have with that approach is that I can take a week, or maybe two weeks, put it into a prototype, go show it to some stakeholders, or maybe do some musical A-testing on it. I'll know really quickly whether that thing failed or if it was successful, and my investment is maybe two weeks.
If it failed, and I'm going to do a release another week or two from now, then I've really only lost maybe... thrown away two weeks of production time versus, once it hits production, that thing is probably in there for six months, nine months, a year, two years. So, the risk of not doing it or basically, trying to try it out in production, to us, it's pretty stupid.
The other thing is then what happens is people tend to be very risk averse, because since you know that if you put something in production, it's going to be in there for six to nine months to maybe a year and a half, you tend to be much more risk adverse and not really try and innovate much.
If you're doing it in prototyping, you're like, "Yes, you know what? It's not really going to take me a whole lot of time and effort. If it blows up, no big deal. I can replace it quickly and easily, and it's not like the company is going to lose any money. Nobody is going to ban the shopping cart if it's just in the prototype."
So, it's one of those things where prototyping actually allows you to take risk and at the same time reduce risk by taking them in prototyping instead of taking them in production.
Jared: Sophie asked a question about production. She wanted to know if the prototypes have to be made with the same technology that you're going to use in your production system, or are there actually advantages to doing them in something completely different?
Todd: That gets back to the very first guiding principle, right, which is: know your audience and intent. From the very beginning, if you know your intention is that this is going to go into production later on, that's really going to drive the approach you'll take, it'll drive the language that you're going to use - and by language I don't mean English versus Spanish - I mean that drives the prototyping language.
So, are you going to do it in HTML, are you going to do it in FLEX or maybe you're going to do it in Rails, or whatever, versus maybe do it in PowerPoint, right?
If you know, and if you've established from the very beginning that, "Hey, we want to do some prototyping, but our goal is actually that prototype that we're going to try and recycle some of that from production," totally drives your approach. There are some cases where that's entirely appropriate. There are environments where you can do that.
That's the approach that we take pretty often. I'd say probably 85, 90 percent of our prototypes actually end up making a good portion of their way into production. We know up front that that's a goal, and so we actually will take a little more time in the prototyping process. Our code is a little cleaner. We tend to actually try and write more production code than something that's going to get thrown away.
Now, on the flip side, if we're working with a team and they've told us, "Look, we have absolutely no intention of using any of the stuff you guys are going to write in production," we're like, "Sweet!" We don't have to debug stuff as much. We can go through a little quicker.
Depending on what we've established up front with the product team and the development team, then either we will go more towards production code, or not. To be honest, I don't recommend that for most people, and the reason I don't is that, it can be very tricky. It can take some significant amount of time and effort to do that, and frankly, it can be fairly dangerous if you're not really, really good at writing code.
In those cases, that's what we do recommend for people that want a prototype, even if you're going to use the same language, just have the guys that are prototyping talk to the dev team, and make sure that the development team knows, "Please do not use any of this code in production." If there's any concern that they will, then your best bet is go use some entirely different tool so that they can't try and put it into production.
A lot of the developers we work with are notorious for trying to write the least amount of code possible, so if you give them some prototype code, they're like, "Aagh, well, it's close enough."
May not be the way I would do it, but then your prototype code works its way into production, so if that is a concern then what we would recommend is using a tool set that would completely be some other language or would produce such really bad code that you would never put it in production.
So, Axure is such a great tool. It does kick out HTML code, but you would never, ever, ever, ever put that code into production. So, it might be a way where, hey, Axure is a tool I might want to use, or maybe do it in Fireworks, or PowerPoint, or something like that.
So, again, just the nature of the clients and the guys we work with, we establish that early on, and the guys that we work with, typically they're comfortable with the level of the code that we write, and so we'll go that route. But, for most people, again, we don't really necessarily recommend that, because it takes a lot of time and effort.
Jared: I would think that one of the risks is that if you're doing a prototype, if you're doing lots of iteration, and you're trying out ideas and some of them are working, and some of them are not working, that if you're going as fast as you can, architecturally, the code you're going to end up with is going to be a complete bowl of spaghetti.
Todd: It can be. Let's say that we've put maybe six iterations into our prototyping process before it's going to get handed off to development team, and let's just say that that development team, we've established with them early on that, "By the way, we want to try and leverage a good portion of the code that you're writing for production."
What we'll typically do is - we write fairly clean code anyway - but what we'll typically do is, the first two to three rounds, it is not worry about making it fully production, just get it kind of close to production.
And then probably two rounds before we're going to hand off, we actually go in and re-factor our code, clean things up, and actually get it ready for production so that the last two rounds, we're really being much more careful about how we're writing things. And then those last couple rounds, just write it like we were going to write a production code application.
If you don't do that, then, yes, what you're talking about, you have thousands of lines of extra code that just aren't needed in there, and it can just be a royal wreck. And that's why a lot of times I don't necessarily recommend it for somebody unless they've already practiced this, and they're comfortable doing it on a regular basis.
If somebody's trying to move from wireframing to prototyping, one of the first things we'll recommend to them is, "By the way, if you ever want to get to where you're writing production level code, don't even think about doing that right now. Wait until you've got a few projects where you've prototyped under your belt, and then maybe think about moving to a different style of prototyping where you're writing for production code. But, I would never, ever attempt to do that from the beginning."
Jared: OK. Well, Todd, this has been excellent, and, of course, the virtual seminar was really a lot of fun.
For anybody who missed the virtual seminar the first time, there is a recording. You can get it with lifetime access for you and your entire organization. You just go to uie.com and look for the virtual seminar on "A Practitioner's Guide to Prototyping" with Todd Zaki Warfel. Todd, thank you again for spending time with us today.
Todd: Thanks, Jared. It was great. Appreciate you having me.
Jared: OK. And I want to thank our audience for sticking with us, and once again for encouraging our behavior. Until next time, take care.