Transcript of Designing Powerful and Interactive Web Applications, an interview with David Malouf and Bill Scott
The following is a mildly edited transcript of Brain Sparks Audio Library #6: Designing Powerful and Interactive Web Applications, an interview with David Malouf and Bill Scott.
Jared Spool: Hello. I want to welcome everybody. We have today to talk on the line, we have Bill Scott.
Bill Scott: Hello.
Jared: And David Malouf.
David Malouf: Hiya.
Jared: And my colleague here, Josh Porter.
Josh Porter: Hello.
Jared: And I’m Jared Spool, and we are going to talk today about webbased applications and rich Internet applications and AJAX and building applications and all those types of things. Bill Scott, who works for Yahoo, and David Malouf, who works for Symbol, soon to become Motorola, are speaking at the UIE Web App Summit, which is happening in Monterey in January.
Bill: Beautiful Monterey.
Jared: Beautiful Monterey. We are very picturesque Monterey. We are very excited about this. The registrations that are coming in, the numbers are better than we expected so it looks like it’s going to be a rocking good event. They are giving a full day session at the conference. They’re speaking on the topic of rich Internet applications and AJAX, and this was a presentation that they gave at the User Interface 11 conference. It was really a wellreceived session. Josh, you attended part of it, right?
Josh: Yes. It was one of my favorites. In fact, I blogged about it a little bit. My big takeaway from the session was that, even though a lot of it’s so Flash and AJAX and there is a lot of new technology that you guys were talking about, you really focused on the interaction side of things in talking about, well, will people be able to understand this if only part of the screen refreshes at a time, or something happens when you hover, or if something gets submitted and updated immediately. I got a lot of out if myself.
Jared: Very good. Let’s start at the beginning here and talk a little bit about rich Internet applications and AJAX. Is AJAX, at this point, sort of the predominant form that we’re seeing, or are we seeing other tools like Flash becoming (whatever happened to Curl?) really sort of becoming in their own? What do you guys think is happening out there?
Bill: I can take it from Yahoo’s perspective. AJAX is probably predominant although we do a lot with Flash, also. It’s sort of the emergence of AJAX as a cohesive set of technologies that have a name and the resurgence of Flash being used a lot for things. For example, Yahoo Maps, our new Yahoo Maps is a Flashbased product. But I think the thing that’s appealing about the AJAX technology is it allows web developers to insert interactivity into the page at a very low cost, and incrementally. Instead of having to learn a completely new technology, AJAX is really sort of an incremental technology on top of what they already know.
David: Yes, I would just say I totally agree with that. There’s something about the cost of entry of AJAX. I mean, the first major AJAX item that can be noticed was what Netflix did with its ratings. If you look at it, it was such a minor change to the concept of a commerce site, but it also changed everything about the user experience, and it was such a small bit of code to add. Then, over time, they’ve been able to scale more interactivity and more robustness and more richness into their user interface, to what it’s become today.
I think that’s one of the huge advantages that you’re just building on top of the existing platform. It’s not a new platform it’s just adding more robustness to the existing platform. I do want to add that the environment’s changing. Vista’s coming out at the end of this month with all that it promises in terms of rich Internet applications, in terms of keying to Windows foundation platform, and all the other goodies that are coming out with that that they are also going to bring into Windows XP and even into Mac OS.
Then, let’s not forget about what Adobe is doing way beyond Flash, that they’re bringing their Engagement Platform together so that PDF and Flash work together, both online and offline. Those are going to be technologies that the developer base is not going to be able to ignore because they’re advanced and more compelling.
Bill: Yes, Dave makes a really good point. I was just speaking with a couple of companies at Enterprise Space. They were actually rethinking their strategy of moving some of their complex apps to the web, which is what they were beginning to do, and looking at things like WPF or Adobe’s thoughts coming forward with their Apollo project.
I think AJAX sort of just kind of opens up the door of this discussion to happen, and these other technologies are getting ripe, too. I think it’s just going to be a really fun time going forward, not just with what’s happened with AJAX or Flash, but also what happens to the desktop as these desktop apps really become rich desktop Internet apps. I think we just haven’t even really begun to see what’s going to happen in that space.
Josh: You guys were talking about how the big names are kind of jockeying for position in rethinking some of their strategies. What do you think is the golden path for end users? If I’m using an application and it has pieces of it online and pieces of it offline, perhaps, what do you think the kind of golden path is for end users, that there’s no distinction between online and off? How do you see people being affected by that?
Bill: I think, for example, a mail product. People expect to be able to, when they’re on the flight and don’t have connectivity, they like to be able to get to their mail, go in and compose messages and have an offline experience. In that case, they do have some understanding that they’re not connected. Ideally, you’d want to be able to store that stuff off and make it truly transparent, and the next time you’re connected up the messages just get sent, which is what most desktop mail clients do, although the webbased mail clients aren’t there yet, because the storage mechanisms, offline storage mechanisms, are just now being formed.
It’s kind of a rough experience right now. The ideal experience is that there’s not that it becomes more of an implementation detail that’s not really part of the user’s mental model and they can just undo their work, but we’re not quite there in the onlineoffline experience. Dave, you might have some thoughts on that.
David: Yes. I think the first major take on this is going to be what Adobe’s coming out with, when they try to have basically their own new browser which works online and offline. You can call it an evolution of Central, but one thing that Central didn’t do very well, and I would say that even email applications don’t do very well, is communicate the state. I think that a lot of users don’t really understand what is going on. Business users do, because they are used to traveling with their laptops if they are in that mode but most users I don’t think believe ever that they are offline that they are always connected when they turn their computer systems on, especially home users. So I think it is going to be interesting to figure out what does it even mean to be offline?
Where does the ubiquity of network connectivity come in to the even questioning of what is offline? If I fly Lufthansa I am not offline anymore because they have WiFi on the plane, or at least they used to, but that may come back or that may change again. So, that’s the other equation, these states are becoming more and more or less and less concrete, the communication of status is becoming more important.
Jared: So I’m wondering, I’m a user experience manager and I have a bunch of applications that are strategic to our business and it’s not going to matter, they are all things that require connectivity to our server in order to be able to focus, maybe it is an insurance company thinking about online quotes or underwriting applications and making those things more selfservice.
When I look at the technologies that are out there and the ones coming down the pipe, do I say, well I should just do HTML for now because I understand that and skip Flash and Java for now because Adobe Apollo and WPF are going to replace those technologies and therefore it is a wasted investment for me to try and get in deep with them. Or do I just do small stuff with Java? What strategy would you recommend tackling this problem with?
Bill: I think you answer from two similar perspectives: one is of course the business perspective. One is the user experience and one is from the engineering perspective.
But the first thing to understand from the user experience perspective is this more of a lightweight kind of application, is it a transient? Or is it something more like Cooper likes to call them sovereign applications that you are going to use every day. Are the features fairly complex or does it behave more like just content pages that have a little bit, a few forms and such? And you move from that sort of traditional weblooking kind of application to something small like a single page application, something that is a for off an email product or some advanced visualization system. If you are going the more lightweight route or the more what I would say looks more like a pagebased application, then HTML is really good and suited for that.
If you are going more for a single page, very, very rich application, Flash is certainly a consideration. Flex is a consideration as a better programming environment for most programmers for Flash. From an engineering perspective, if the skill sets of the folks doing Java want to use the Google WebToolkit. The problem is, there are a lot of dimensions: the skill sets of the engineering staff, there’s the inventory of what you already have, as far as the code assets, the business perspective on how you want to deliver this application that comes into play.
Some of the time these things are so strong, in fact most of the time it is a fairly simple little reduction of I can’t do this, I can’t do this so I have to go this way. I think the key thing from the user experience perspective is really crafting what that flow is going to be for the user and then seeing what technologies fit that within the space of what your company can afford to do.
David: I think the term that jumps in my head when I think about this is disruption, disruptive technologies, disruptive change, the disruptive experiences. One of the reasons why AJAX jumped to the fore was because it was nondisruptive. And it didn’t even know that it was able to happen a few years ago before it did happen. We had to wait for things to be in motion. We had to wait for a certain level of ubiquity. We had to wait for certain browsers to not to be within the legacy anymore.
I remember when I was doing this work I was pushing and pushing for Flash originally, because it was ubiquitous, it didn’t have legacy, it did not require a browser and it got pushed back for the programming language reasons, but this was before Flex. But then I couldn’t do anything with AJAX technologies because I still had to program for Netscape four and AJAX doesn’t work within Netscape 4. So there was something about how disruptive will my business users going to and how disruptive will my end users allow us to be? How much change can we ask them to go through? I think that whether it is Apollo or WPF or WPFe, the reality is that those are going to be disruptive to a certain extent, initially, until it gains a level of ubiquity.
What is going to be the install rate of Vista? How much can we force Windows XP users to add WPFe or Mac users even? What is going to be the install rate of Adobe Apollo? How is Macromedia, Adobe going to push those forward technologically? So I think that disruption is a really key word when we think about end user and business stakeholder experiences there.
Bill: What is fascinating there is that I would actually call AJAX subtly disruptive because it wasn’t disruptive to begin with from the point of entry but it is actually fairly disruptive to the web space from many, many perspectives. For example most of our revenue is built around the fact that we have a page refresh, a lot of ads that are out there. At Yahoo, obviously that is a huge concern. I think we have weekly, seems to be daily meetings around this topic around AJAX and how to make sure that we don’t destroy the company with AJAX. [laughter]
Fortunately we have some good ideas in place and we’re trying to purchase in the same manner. The same thing from a search engine optimization perspective from a site like Yahoo and even our ability to crawl applications that are AJAX based as well as being able to expose things that we have that are AJAX based, it is actually a fairly challenging problem beyond just making something accessible to search bots, there are a lot of algorithms that are in place that that are built around that pagerefresh model. In fact there is a whole host of things that are really built around that 10year special case of pagebased applications that are now being disrupted. Now I know last year when I first got to Yahoo we were raising the flag about revenue issues as well as search engine optimization. It took about a year for it to become a hot topic in the company, even though we were missing it from our team over and over again.
I think that this is what is going to happen is a lot of this stuff…people have only thought through what the impact of that really is. So now we are just starting to really process that.
If you had to give advice to a developer who was working on a pagebased web application it is builtin pages and it is modeled after the page model where would you tell them to start? For example, if they the interaction benefits of AJAX and they realized that they could probably use it somewhere, but they might not be able to completely change their application. Where would you tell them to start? How would you get them thinking along the lines that Netflix thought about when they said, “Well, you know what, if we just did ratings that would be great idea?”
Bill: One of the ways that we have done it here, and taught it to our engineering staff as well as our user experience staff, is through the use of design patterns. Because we actually released publicly the Yahoo design pattern library and the idea of that patterned after Jennifer Tidwell’s work. She wrote the book “Designing Interfaces” as well Martan Vanweelis’ work out in Amsterdam is to describe a gallery of idioms or techniques that are being employed and discuss those in terms of best practices. They are a little more bite size chunks.
When I’ve given these talks to engineers, I know they really appreciate the perspective of having a name to a technique as well as a sensitizing example, then some kind of best practices around it. It gives them a toolbox of solutions. Obviously they have to understand the context in which those solutions should be applied. We then can hang actual code examples and code techniques off of those. I think that’s a really good place to start from understanding what’s available from an engineering side.
David: I think it could be both. I do think the incremental approach is the best way to win the minds of those people in your company that are the stakeholders that are going to let you continue to do these sorts of innovations. Having small successes usually makes the most sense. The best way to find those is to identify what the users pain points are.
Most the time what happens is the things that the users do most commonly we’ve usually erected the most barriers for them to be able to do those kinds of things. If we can identify those paths and just remove those roadblocks. If you look at what Netflix has done, if you got a chance to hear Sean Kane give a great talk at the AJAX Experience in Boston. The theme of a lot of what he was saying was during their user testing and design research they would just identify pain points that the user had and then determine what innovation, usually in the area of AJAX, not always, sometimes it was just a visual design change, that would enable the user to get their work done, to actually select movies quicker and have a more delightful experience in choosing those movies.
David: One of the things about this question to me is the way Josh started is very interesting. He started from the point of view of the developer. I found that really interesting. I realize that a lot of people who come to UIE events are developers and they are important part of this but I think that this is a design question. This is a question for the designers.
Josh: I’m sorry I didn’t mean to rule out designers. That was just a…
David: [interrupts] I didn’t think you were ruling them out, but it says a lot. It says that still especially those who are doing AJAX more than Flash are still thinking of AJAX as a development problem instead of a design solution. Even the way Bill answered the question, thinking about what you need to do to rethink yourself as a programmer in doing this work says a lot. It says one: you can’t completely disregard the developer component, because they are the stopgap for any designer. If a developer doesn’t want to do it, it isn’t going to happen. If a developer can’t do it, it won’t happen. They are important, but when you think about, what are the elements that need to change for the user experience, the first person, the first group that needs to be engaged there are the designers.
Bill’s second thing about talking about Netflix at the AJAX experience conference was where you do your testing and you do your research. And you find out the problem areas that people are facing to complete the tasks or the goals that are in front of them. You figure out well, if I turn the volume control from one to two, what does that mean? If 11 is where we’re headed. What are the mini steps in order to help those problem statements? I think that’s really important.
The more specific thing I would say is a quick guinea probably for anyone, at least in forms, is using AJAX for validation purposes. You can do so much with AJAX so that the user experience submitting a form is so much better. That is usually a quick hit that anybody can just add to an existing page such as form without changing too much else. You are still only working with a net page and you are doing calls that you will have to do anyway. You are just doing them in a different flow.
I think that would be something that I would point people toward who are saying “Where can I test this out to see if one, I could do it, and if two, it scales within my service or how it scales within my service and three whether it is actually going to solve problems users are facing.”
Bill: That is actually a great lead in for what we are going to be talking about in Monterey. That gets kind of the heart the principles for designing rich web experience. Whether it is Flash or AJAX or whatever your technology is.
What I have found interesting here at Yahoo and teaching to a number of our engineers, and some really smart engineers I go in and have one talk that is a lot of what we’ll give in Monterey. It deals with just the high level principles from design. Engineers are really hungry for that. Even in a place like Yahoo where those engineers are not doing much design we have a whole design team doing design. Out of the web space in most companies, people are doing double or triple duty. They are making decisions that are design decisions as well as engineering decisions, just because of the fragmented nature of our business.
I think, getting back to the very basic principles of interaction, things we’ve learned on the desktop do apply to the web. There is a nuanced twist to some of those. There are things that users kind of expect, that are familiar to them in the web space, that you sort of have to apply almost as a scaffolding, as training wheels to get users to understand how to interact with in a slightly new manner.
There are lots of little tidbits that we talk about, that we’ve learned in our consulting at Yahoo and other places that we’ve talked to about the properties of animations and cinematic effects and when is a delay good, when is it not good, you know. What kind of invitations can you provide the users to cue them on knowing what to do? A lot of practical examples like that.
I think that’s the kind of thing that I’ve experienced over this last year, especially giving lots of talks around these topics, is that people are hungry for seeing the examples and understanding the context of how to apply those solutions, and really usimg technology right. It can be used in a fairly goofy manner, and we do talk about some of those examples, as well.
Jared: What are some of the examples of some of these cinematic effects and stuff that you’re talking about?
Bill: Some of those are just as simple as having something slide out from the side, so it’s like a creative use of real estate. I only have so much room on my screen, my web page, and if I can have slideout panels, that helps. We saw those, of course, early on in menus, but they’re applied more to panels and those sorts of things now, obviously having things that establish your spotlight and show up and then fade out over time, or when things change somewhere on the screen and you want to draw the user’s attention just having a light coloring effect that fades over time.
It kind of turns out that with animations and transitions, the brain is wired to respond to those. It’s wired in such a way that things that move rapidly get your attention, obviously, more than things that move slowly. Color changes are less noticeable than movement changes. A lot of people haven’t really thought about that sort of hierarchy of processing in the brain. That’s important when you’re doing that, because all these things are really about communication and engagement communication so that you’re actually using them not just to show off that you can do a special technique, but that you’re actually communicating or reinforcing a concept or idea in the interface.
It’s also sort of like a language of an interface. Whenever you do any kind of technique or idiom, you want to be consistent throughout the application, so take some thought to be careful about how you craft those. A lot of stuff in even the animation area right now is, we’re seeing people overemphasize those sort of things, focus on those techniques beyond where they need to be focused.
David: Yes. I think that one of the things to think about is, what we talked about is the discoverability. Two things one is, your content may be so complex that it can’t all be seen at the same time, but it all requires the same context, which means that I am in the midst of doing something or I’m in the midst of reading something, and not all the information has the same level of priority, but all the information is related to what I’m currently working on. I’m going to need to see it juxtaposed to the same context, but I can’t see all of it at the same time.
A great example of that would be the way G-Mail allows you to expand and collapse and even reply to messages all within the same page view. The threading is that context, the conversation as they call it, and then the reply as well, I’m replying within the context of that conversation. Instead of popping open a window where I completely lost the context of my conversation, the reply or even the forward are done within the context of that existing space.
Another type of discoverability is when you have things that are happening that are new bits of information that weren’t available before within the same mode of operating. The best example that I have of that is for Zimbra, which is a groupware system. A web-based groupware system where it will recognize a person’s name that’s in your address book, or recognize a string as an address, or a string as a phone number, and when you roll over those recognized strings, it will give you something new. In the case of an address, you click on the address or you roll over the address and it will show you a Yahoo Map for that address.
That is different from what we expect to happen in email programs. If I didn’t do that in a discoverable way, it would overcrowd my page. I wouldn’t have the ability to really focus on the first priority, which is the message itself. The other thing for animation that’s really important is things that are moving get our attention. One of the examples we show is this really subtle example of a Laszlo System shopping cart, where it’s a Flashbased shopping cart and when you click on the Add To Shopping Cart, you see the shopping cart move. It’s an accordion tab but you see it jump just a little bit. That little bit tells you, “Oh, that’s where I need to go.”
Bill: Yes, just a little bit.
David: In my shopping cart. It’s a subtle thing, and it works so well. I know that I use an online grocery shopping thing, and I’m always hunting for the darn cart, because I’m not always aware that it’s all the way minuscule in the top right. Like, if I get that reminder every once in a while, that would be really useful. Is animation required? No. Amazon takes a different approach, where it builds your shopping cart for you on the right side, always making you aware of it, but some might say that’s a little heavy handed, because I don’t need to see the shopping cart all the time. It’s just different ways of thinking about it.
Josh: So you guys have been discussing what I believe you would call a rich design pattern, kind of a transition that happens, a movement that people can see visually and kind of identify with that. Bill, you had mentioned design patterns earlier in your work at Yahoo. Could you say a little more about how the library came to be, and kind of the theory behind the design patterns? Is it a development tool, is it a tool for developers to use and communicate with, or are there other reasons for having them?
Bill: The original Yahoo design pattern library began about 2004. The idea was to collect together best practices from really a threepronged approach. One was interaction design patterns, taking those things that really interaction designers or information architects would be putting together as far as best practices. The other was visual design practices, which were more like the traditional style guidelines except in smaller chunks, that tells you exact spacing and typography and which font to use and all those sorts of things. The other was to have some code associated with it and then all formed together a toolkit so you could satisfy the interaction design community, the visual design community, and the engineers. And so that’s been going pretty well. Internally, I think we have about a hundred patterns in that internal pattern library. It’s available on a content management system under Drupal. Anybody can basically add a pattern and it goes through a vetting and review process to get added in and then it gets rated as far as, you know whether it’s the Yahoo way or not. Earlier this year, in February, I took the work to expose some of that externally but also begin to document new patterns that we had not captured yet internally. Those were the rich kind of web design patterns. And that manifests itself out on developer.yahoo.com/ypatterns, which is our pattern library, public one. So from a perspective of that public one, I’ve actually gotten a lot of feedback from engineers who like what we’ve put out so far; it’s still a small set, because it’s given them the ability to think about it at a little bit higher level. And yet we’ve also associated code with those patterns. So people can kind of get a jumpstart on some of the concepts there.
Joshua: OK, now from my understanding, the original form of pattern invented by Christopher Alexander or at least popularized by Christopher Alexander. There are several parts to them. There’s a title, there’s kind of a problem context, there’s a constraint section, then there’s a canonical solution. Could you explain how those pieces work?
Jared: OK. I’ve got one last question here, which is you gave your seminar at the User Interface 11 conference and it went really, really well. I’m wondering what were your favorite parts of it?
Bill: I think Dave’s favorite part was hearing me cough. I had a bad cold that day and I think he just really enjoyed that. I do plan to be in much better health in Monterey.
Jared: Oh good. You’re planning for that?
Bill: Yes, I’m definitely planning for that. I’m feeling much better these days. I can actually talk. I think it was really fun. Dave and I had never done one of these together. Doing it together and our different perspectives really blended well together. Point and counterpoint was kind of a fun aspect of it. Where I would have nuances from Yahoo and he would have nuances from his experience, which I think made a more complete picture.
David: I think my personal favorite moment was this exercise that we did, which was an exercise that a really smart audience helped create together. The audience got to create a problem statement. Then in groups, make assumptions. Then do two or three screens just with pencil and paper. Then really discuss those solutions through and really learned from each other. We learned, I think, as much as they learned through that process. It was a really great exercise that I think highlighted the design issues and how to do design for these types of rich solutions. And it was fun to constantly comment on Bill’s coughing.
Jared: What was the product that you guys created?
David: The solution ended up being an MBTA kiosk that would reside at a bus stop to allow you to manipulate some kind of token card, RFID based payment system for the bus.
Bill: And we actually have the beta version coming out next week, by the way.
Jared: Oh good. Excellent, because the buses and subway tokens here in Boston, the MBTA is the subway authority, and their little system definitely has all sorts of room for improvement. That is for sure.
Bill: Well we found you’re supposed to do a beta. If you do a beta it makes it good, so we’re going to do that.
Jared: Well, OK. That’s the thing. If you do a beta, you have the excuse of things not working I guess.
Bill: That’s right.
David: I’m hoping that in Monterey, whatever the class comes up with will be either about canneries, aquariums or kelp.
Joshua: You don’t think it’s going to be about the Monterey subway system?
David: Hmmmm. Maybe their little shuttle system that they have…I guess there is the Wharf that we could also add in there. We’ll see what they come up with.
Jared: There will be sea lions. Sea lions. Well, I want to thank you, David and Bill, for taking the time today to talk to Josh and I. David and Bill. Bill Scott and David Malouf will be appearing at the User Interface Engineering Web App Summit. There’s going to be giving both a full day tutorial and short presentations. The full day tutorial is on the 21st of January. That will be on rich Internet applications and AJAX. And on the 22nd, Bill Scott will be talking about design patterns and principles. And on the 23rd, Dave Malouf will be talking about doing rich applications in a presentation called, “Why is Rich and why do Rich?” I want to thank them both for taking time today and I want to thank you for listening. So until next time, this is Jared Spool signing off.