This a transcript of the UIE Podcast "SpoolCast: Interesting Moments with Bill Scott" available at This podcast is brought to you by The UIE Web App Masters Tour, an inspiring, intensive two days with today's leading web app designers. Coming to San Diego, Minneapolis, Philadelphia and Seattle. See the Master, Topics and Dates at http://UIETour.com ----- Jared Spool: Hello everyone. Welcome to another episode of the SpoolCast. Today I have with me -- I'm very excited -- I have with me Bill Scott, who is an old friend of mine. He's the director of UI engineering at Netflix, and he also is the author of "Designing Web Interfaces,", which he coauthored with Theresa Neil. And he is going to be presenting at our UIE Web App Masters Tour which starts in San Diego in March. Bill, welcome today. Bill Scott: Thank you. I'm glad to be here. Jared: I've been working on the materials for the Web App Masters Tour, and I was going through your presentations. The presentation your giving is called "Designing for Interesting Moments." And what your doing in this is interesting. You're slowing down time so that we can see the actual interactions that happen when somebody does something like drag and drop, all the different things that happen. Where'd that idea come from? Bill: I think it kind of goes back to being fascinated by the nuance of design. I started back in the early '80s with a Macintosh, and I was always fascinated by just how much attention was made to the detail of an interaction. And then as we came to the web and finally had richness that we could do on a website, it was apparent to me that a lot of people didn't have that background appreciation or experience of having to wrestle with the interactions. You know, rich interactions. So as I thought more and more about it, I realized that any way that I could tease out that understanding to people would be really important. Then it came to the fore when I was at Yahoo, and the Yahoo user interface library team, where I was working closely with them to help think through their drag and drop library from a code perspective. And I was looking at it from a design perspective. How could we create this library so that from a design arena we could do anything we wanted with drag and drop and those sort of things. And so what happened was I began to document it and try a lot of state diagrams, drawing things out, and here's a drag and drop interaction, And all of the possibilities of permutations just began to explode on you. As the scribe at the meetings, I would go off and try to document all the possiblities that we had talked about, all the nuances, and before long I had 20 or 30 pages of notes. And I realized quickly that that would be a silly way to come to any conclusions. Then it occurred to me, as often is the case -- the cobbler isn't always aware of his shoes -- that maybe I could apply a little bit of design back to the way we were capturing this information. It occurred to me the thing I saw over and over was that there were a lot of little events, and there are always these objects and elements involved. By putting those into a simple grid I could really point out the time and slow down time and start putting all this under a microscope and bringing out the nuance. Jared: Yeah. What it brought back to me was, I was programming window systems in the day, before they were part of the operating system. So if you wanted a scroll bar, you had to create all the interaction that the scroll bar had. If you wanted a drop-down menu, you had to store the bitmap that would be below the menu, you had to write the white box that the menu would exist on, you had to populate it with the text that you wanted it to have, you had to do the different hover events as the user moved the mouse up or down to deal with it. When they selected you had to invert all that. You had to take the bitmap you stored and put it back and get rid of the white box. You had to do all those things and you had to be very conscious of it. You had to do them in a very specific order and your code had to be really tight to get it done small. And that was how we put a pull-down menu up there. Kids today, they don't have to do that because they operating system takes care of it for them. Bill: That's right. Jared: And I think that's a lost art, because we had to sit down and think about all those interactions to do anything simple. And of course the faster processors you have, the faster displays you have, the more of these interactions you can do. But at the same time, you have to slow down time to be able to understand what is really happening in these little interactions. Bill: And we take the phenomenon of the web as it came about -- the Web 1.0 -- where you only had two events most of the time. And that was just a click on a link and a submit button. You know, submit a form. And maybe you had a hover. Maybe you had two and a half events, I don't know. You really didn't have this problem nuance with what you could do from an interaction perspective. But as soon as we opened that up with Ajax and newer versions of Flash and all those other things -- Silverlight and all this other stuff that's happened -- the same problems we wrestled with back then come back, and come back to the developers and engineers. And coming from an engineering and a design background both, I'm just like you, that I had to wrestle through all those real, real subtle nuances, and saw that as we built a library we had to do that. It is interesting, because design teams in Yahoo appreciated the interaction grid. We call it "interesting moments grid." But the engineers really appreciated it, because they could just look at it and they didn't have to wonder if the designer had forgot something. Because when you put this into a grid with your events going along the top and your elements on the page or actors or objects on the Y axis, then you purposefully leave some things out and you purposefully put some things in. It's real obvious the decisions that you make, both with what to do and what not to do. Jared: Yeah, I found the grid to be fascinating because like you, when we did all this stuff we struggled with state diagrams. And the more complex the tool, the more difficult it became. And it was really difficult to communicate how you get from where to where. So the grid acts sort of like a transition matrix, right? Bill: Yes, it does. Yes. Jared: And so any given place on the grid, you can then look at the columns or look at the rows and figure out where else you're supposed to be. And so each column represents something like a cursor or the object that you're dragging or the menu item you're clicking on, right? Bill: Yeah, each row can be. And the way I did it, each row was one of those objects and each column was a different event. Jared: Oh, right. OK. Bill: It may be like you hover, you click. It could be that you start dragging, you stop dragging, you drag over an object that is a valid drop target versus one that's invalid. All those different events are interesting moments come into play there. It is funny, because when I put this together I realized, "Well, this is actually a good way to do this." And it sure got rid of 20 or 30 pages of notes, so I was pretty happy with that. And it also communicated well. Then one day, I was in Flash doing something and I realized, oh, OK, I just stole the idea from Flash." Because, if you get the Flash grid at the top, it's basically your objects or the rows and the timeline is running along the X axis. [laughs] Jared: I haven't done that much Flash stuff, so I didn't realize. Bill: Well, I haven't either, but I think I probably was subtly influenced by -- we're all a product of all the things we've seen. Jared: That's true. That's true. I know I'm a product of the things I've seen and the things I've eaten. Bill: [laughs] Yeah, me too. Jared: Your book and your presentation are strewn with these little diagrams. So, people who are familiar with this are going to see it. If you want to see an example of this, you just have to start thumbing through the "Designing Web Interfaces" book that Bill and Theresa wrote. You can see these things. What's interesting about the diagrams is, like you said, the empty boxes are as fascinating as the filled ones. So the filled boxes say what happens when that event happens on that object. Right? Bill: Yeah, it's just the same thing, that a rest in music obviously just as important as the note itself. Right? Because sometimes the rest can lead to some kind of point that's going to come in the music. And the same thing, if you try to do too many things, you can get too chatty with the user. So, you may want to save the punch for one of those moments when you actually drop something. But generally speaking, where I see a lot of problems happen, especially with what I call "anti-patterns" or what are called anti-patterns, these are the things that you really shouldn't be doing in designing interfaces. It's because people miss those moments, they're actually missed moments. A good example of that is the old Yahoo Photos site, which is no longer up, but I do have it saved for posterity in a Screencast, and I show that in my talk. But, you end up dragging several photos into an album, and because there's no indication that the photos actually dropped into the album folder on the left-hand side, and there's no feedback that says, "Oh, there was three, but now there's six items in the folder." The designers had dropped in two extra "idiot boxes", which is a great anti-pattern. The first idiot box says, "Do you really want me to drop these items into the folder that you so carefully managed to use your mouse dexterity to get to?" Not quite that message, but that's gist of it. Then there's another pop-up that says, "Hey! Guess what? We did what we said we would do. We actually put those items in the folder." It's sort of this, as Alan Cooper calls it, "stopping the proceedings with idiocy." So, the missed moments, though, were just those little, subtle feedbacks that could have been done, instead of the hammer approach by having those boxes pop up and interrupt the user. Jared: So, when you're creating these charts, how do you figure out what is key? Are you doing it along side the actual prototypes, or is it one of these things where you'll write some stuff in the chart and then you'll create a prototype to do that, then you'll play with it and you'll go, "Wait a second, we've missed a moment here, we've missed a moment there." Then you go and change the chart. Or is it... Bill: We've done it different ways. I know at Yahoo we sat down and sketched it, just printed out a big grid, like a huge maybe three-foot by two-foot chart. And we just took a pencil and sketched in it. And said, "Well, this is what's going to happen here and here and here." And we actually used that to drive our usability studies, because we were trying to understand what kind of... For example, at the moment, when you render the item on the screen, what could you do at page load time to indicate that you could drag and drop like on the My Yahoo site? So you'd add textured dimples there and see if that helped out. You could capture a bunch of different variations of trying to get a message across, trying to communicate with the user at that very moment in time, just when the object was on the page whether it was draggable or not. Then what happens when you move the cursor over an inch, should it be turned to a hand, should it not? Do users see that or not see that? And that was the kind of studies we were doing. Then, in other cases, we actually used it to kind of go back and document what we had done from prototyping and what worked. Once you got it bootstrapped, you tried a bunch of stuff. We didn't try to document it. Then we captured it back into some diagrams to share with other properties within Yahoo, as they started looking at drag and drop. Instead of them having to go look at our living example and try to reverse engineer it, we just sketched it back into the diagram. So, work is a good, a reverse engineering tool to how the design works. And I used it as a debug sort of thing sometimes to just see what somebody's doing when I see an interaction that looks good. But, I'm kind of nerdy like that. So... [laughter] Jared: I think this is really neat and I like the idea that you're thinking in terms of these very small-scale interactions, but then blowing them up, because a lot of the devil is in the details on this. Things like… a lot of applications will put up a message with a bright yellow background and then they'll fade out that background over a period of time. How do you decide where that background appears and when it fades and how long it should take? And by slowing down time and understanding what those transitions are and when they're going to happen, it makes it easier to build it. It probably makes it much easier to go through your QA process. I would think it simplifies things in all different ways. Bill: Yeah, and there's an interesting side effect of this too, though, that can happen to you that I've watched on the other end of this extreme. And this gets into animations and transitions. What I see a lot of times, people wanting to do is over emphasize those and slow those down too much. And this is an artifact of the design and engineering process that when you focus in on some sort of special effect, let's say, you tend to over-emphasize that, because you're not thinking in full context of the whole flow of the application. So, you have got to be real careful as you get the microscope out, but you also back up and look at the whole landscape. I was just talking with a designer's husband who happens to be in motion graphics. We were discussing this and he said, "Well, that's really interesting because in my area of motion graphics when I'm doing commercials and trying to figure out luminous effects or transitions and these sort of things, I have a simple rule that I follow. I call it the 'cut it in half' rule. "So whenever I do an effect, I usually cut that in half or cut it in half again. If I've got this amount of luminance, I may cut it down half way to what I think. Because, when you put it in an overall context, you don't want it to be the primary actor on the stage. It's just a part of the supporting cast." So there's the other side of the whole hazard, that if you focus too much on this micro-interaction, you might overemphasize it. Although you have to dive down there and think about it, but you also have to come back up to the top level and think about it. Jared: Right. I remember one of the examples you showed in a presentations that I saw a couple years ago. In the middle of the interactions, there was a drag-and-drop or something, and there were dancing gerbils. Bill: Yes, dancing hamsters. Jared: [laughs] Dancing hamsters. Bill: I still show that one, because it's still popular. Jared: Well, everybody loves dancing hamsters. Bill: [laughs] What's there not to love about dancing? Jared: [laughs] But, yeah, I think that's the downside of making the interaction a little too present. Bill: Yes. Exactly. Jared: So, I was very intrigued, as I was reading through your book -- which is just awesome -- about having seen you present, I was concerned about the book, because there is a lot that you do in the presentations that involve animated screen shots, animated screen captures, where you show an iteration. I wasn't sure how you were going to pull that off in the book, but you actually do a great job. Did that process take a lot of effort to figure out exactly what you were going to do, or did that just come really naturally? Bill: It did take some effort, because that was what I worried about. I worried about two things about the book. One was, you have to show lots of examples and the examples will soon date themselves, right? Jared: Yep. Bill: And secondly, I'm looking at these real fine-grained interactions and discussing the nuance of those. How do you actually show an animation in a book? We need new books that you can have animations embedded in. We don't quite have those yet. Maybe some future of the Apple iSlate or whatever, I don't know, or a new Kindle. But we don't have that, so I just went back to Scott McCloud's work on "Understanding Comics " great book, and just basically using key frames to tease out those interesting moments, and calling those out, and pointing out the salient points. Which, in a lot of sense, is actually more powerful than the presentation. Because if you take those key frames in the right way, the human brain is really good. As Scott talks about this in his book "Understanding Comics," of doing the tweening, understanding what goes in between those frames. I slowed down time a lot in the book and look at each one of those key frames and say, here is the tradeoffs that are happening here. That's one of the things I tried to do in the book a lot, was not say, OK, well this is the only way to do something, because obviously it's not. But, given the constraints of the design here, if I do know what was going on, I'll say, because I may have been involved in the design, or know the people who were, or otherwise I'll say I'm conjecturing. This is probably the things that drove that. I think that's what people have appreciated. As I've heard feedback, positive feedback, is that they appreciate the peek into the nuanced thinking that goes into these rich interactions. Jared: Yeah, it's really quite neat, the way that you did this. Do you think that this is a technique that designers could use when documenting the interactions that they're building? Bill: Yeah, and there are a number of folks who have done this in their interactions. Dan Saffer, over at Adaptive Path, and I have talked about this in the past. He's done some of this with key frames. I think he was calling it "microstates." which is another good way to think about it. A few years back, I still have people who use this framework that I wrote with Visio. It was interactive storyboarding. I did an article for "Boxes and Arrows," maybe four or five years ago. The idea was to take Visio and make it as interactive as possible, so you could simulate these events. In essence, what I ended up having was little embedded panels that you could click on and it would change the interface down below you, to show the interesting moments of time in a flow or interaction. Jared: Oh, neat, yeah. Bill: Yeah, it actually was a pretty nice thing. I was one of those, over the holidays, I decided I had a little time and I thought I would try to tame Visio. Because I was tired of trying to do... Rich web applications at Sabre, which were very complex airline management applications, running the whole operations center, or crew planning, or flight scheduling, which were not trivial applications, we were moving some of those to the web. Trying to wireframe those was just ridiculous. So I came up with a templating system in the stencils. You can create stencils in Visio and they were storyboarding frames. This is all kind of ties together with - even on the documenting side, I've given talks about that perspective. In fact, I did a workshop for you a few years ago. We did that in the talk. We talked about how do you document these things. Part of that was this whole thing of using key frames and trying to make those interactive. So things have moved forward in that arena and other people have been doing, and are doing, some of the same things. Jared: Now, when you were putting together the book, this book wasn't always a given, right? This wasn't a, you sat down, wrote it in a weekend, and you were done thing? Bill: [laughs] You can ask my wife. It was a weekend, and another weekend, and another weekend. 52 weekends I think. What happened was I was at Sabre and the Ajax revolution kind of popped up. I was being invited to come up to Yahoo to be their Ajax Evangelist. I had been doing an Ajax framework at Sabre. We launched Rico, which was one of the first Ajax frameworks. So I had a publisher who came to me to write - they were just putting out a book on Ajax, it was the first book on Ajax. It was a bestseller and one of my good friends was a co-author in that. They wanted me to do a book on designing for Ajax. I decided to that and started working on it. I went to Yahoo and though I would have plenty of time, the weekends, but every time I sat down to write it was like, oh my god, it was the most life draining experience. It just sucked the living... I got to where I couldn't even imagine ever writing a book. I decided, at some point, I just couldn't go on. I kept taking to the publisher and they kept talking me back into writing. So finally, John Battelle came to the Yahoo campus and spoke on his book on search, you know, on Google, and I discussed this same phenomenon with him. He had the first version of the book done. "We read it and just thought, "This is a piece of crap!" It was too academic. It was dry and it wasn't interesting." So he ditched it. He says, "You know, I am journalist, I am story teller, so I tell it as a story." He went back and he wrote as a story and it's was a bestseller. And I am sitting in the audience trying to decide whether I should continue writing this book. And I realized, "You know what, I need to not write this book, because it feels the same way. And if the book comes out later, it comes out later." And like the next week, after I decided not to do this and cancelled my work on it, I started getting invited to speak. Then there was an interesting thing. Nate Koechley, who used the lead the front-end development at Yahoo, had pinned me an email and asked me to give him a few tips. He was going to give a talk to Design Engineering. And -- you know these sideways moments, you know, where you're not thinking about something? I kept getting guilty about not responding. And finally, a few days later, I shot this email off and I wrote like nine... little tips came about. How you should think about designing for the Ajax and the rich experience and those sorts of things. And when I got down to the email and sent it, I went, "Oh my god! That should have been the book." And Theresa, who is my co-author, I sent her a note and said, "That's what we should have used," and she said, "Yeah." And so, we didn't do anything else with it. But then I went and spoke. The first year I spoke about 40 or 50 times, variations in that talk. And what happened was I started hearing audience resonating with it and the way I framed the talk changed as I got more and more feedback. So when I finally sat down to write the book for O'Reilly, there was a different audience in my head. Before it was these imagined peers who were going to critique my work. And so I was always writing very defensively in that first version of the book. I over-researched things. The language was too stiff and formal. I got more comfortable in own skin in speaking a lot and realizing that I didn't have to have it all right. All I had to do was just share my experiences and say, "This is what I understand," and express those limits, you know, and just give back to the community. So the audience inside of my head as I wrote now was all those people; I could see their faces. And generally, Jared, you look out and see the smiling faces, right, so that's the people I thought of. So it was just a totally different experience. You know, it's funny; you're writing a user experience book and you forget who you're writing for. Jared: Oh, yeah. Bill: You do a conference and you forget. All these kinds of things, we have to reminded who the real audience is. So that I think changed the whole tone of things and made it a fun experience as I started writing the book. And then the challenge was just how to get it into a format that wasn't as interactive as a presentation, and I couldn't rely on my down-home Texas accent to make people feel comfortable. Jared: Right. [laughs] Well, the book is really a great read; you and Theresa did a fabulous job, and the examples are a killer. And I am really pleased that O'Reilly did such a beautiful job of printing it. Bill: They did. Jared: The full-color images and stuff. This is not a cheap book to produce. Bill: Mary Treseler was my editor there, and she is incredible. What the turning point really was, as I was trying to design together was, I happened to be in Boston - I think it might have been for one of your events. But anyway, I was in Boston and I went over and met with the designer for the book. And I just went through the presentation with him when I got there, and he goes, "Ah! I get it now." And it was like, from there on out, there was just no trouble at all. They just did an amazing work with this, and they caught the same spirit that I had in trying to produce the presentations. Jared: Yeah. It's really a great piece of writing. I'm wondering if your lesson about sort of knowing your audience plays into things, like the documents we have to do for the people we work with? Bill: Yes. It's constantly. Because there's so many design artifact documents. And you can start creating different artifacts whether it's wireframes or they are full-on prototypes, or detailed visual comps, these sort of things. They always to keep in mind who is the audience, what's even the process here where I work, you know, what's the right optimization to do. That's what we look for actually when we look for engineers and designers, is do they get the business contacts and the players and what their roles are, so when they're in design or building user experience from an engineering perspective, do they do the right thing? It's a real important skill to have beyond just the technical aspect of design or engineering. Jared: So would you say that since you wrote the book, it's changed the way you do work documents for Netflix? Bill: I think so. Yeah, it definitely has. One thing the process did for me was teach me to edit, because it's so easy to write so much. I still have that problem when I do presentations, because there's so much material. I hate to leave anything on the cutting floor, but it's like having the rest, points you don't do and the interesting moments diagram. But leaving out stuff, sometimes you make the other message much more valuable and important. But I still wrestle with that. But that's something that I have learnt from doing the book was edit, edit, edit. Jared: Well, you're famous for having 200-slide decks for an hour of presentation. Bill: [laughs] Exactly. So don't listen to me. I am not the expert. Jared: Right. Bill: But I can talk a good story. Jared: Yes, you can. You can. And I want to encourage everybody who's listening to come and hear him talk a good story at our coming UIE Web App Masters Tour, which he's going to be speaking at. Bill: Yeah. We'll even play a few games, too, and people get a chance to win some books, too. Jared: Oh, fabulous! Well, that sounds excellent. And, of course, the book is "Designing Web Interfaces: Principles and Patterns for Rich Interactions, " published by O'Reilly, written by Mr. Bill Scott and Ms. Theresa Neil. It's a fabulous book. I think it'll be one of these things that will sit on your desk and you will keep turning to it. If you buy the book, I am going to highly recommend that you write your name big on the inside and potentially chain it to your desk, because it's one of those books that the rest of the team is going to want to get copies of. So that's awesome. Bill, thank you very much for spending time with us today. Bill: Always a pleasure. Jared: And I want to thank our audience for listening to us once again on this episode of SpoolCast. And as always, I very much want to thank you for encouraging our behavior. Take care, and we'll talk to you next time. --- Don't forget Bill will join us at the UIE Web App Masters Tour. Find out more about the whole show at