User Interface Engineering

The transcript for SpoolCast #2: Facebook Becomes Anti-Social is now available here.

This was a really fun conversation. Some of my favorite bits are:

Kyle Pero’s idea for her dream panel at a conference

I think we need to ask ourselves if we’re growing and improving our service based on what the clients want - or what we think they want - doing a little bit of our own user research I guess. I believe that our clients, and not the industry, should definitely be setting our standards, and I don’t know if we’re doing that. A panel of clients discussing their needs would be quite interesting to me, this is just my opinion. I know every one of us works on different products - some of them offline, some of them online - but basically, the need is the same. Can the products be used easily to accomplish the task? I think if we just take a moment in one of these conferences and just stop listening to each other for a change and start listening to who we are servicing I think it would be pretty interesting and different.

Rashmi Sinha on whether usability studies are scientific or not

Rashmi: But anyone who thinks that this is science, hasn’t done science! I have done science, and this ain’t science, by any stretch of the imagination. I disagree with the whole notion of trying to make it scientific because it isn’t scientific. Second, you have a big problem…

Jared: Making what scientific? The study itself, or the field?

Rashmi: The field is not scientific, and now people have this big notion from experimental psychology, but really it’s not. Experiments have - there’s a way of doing them, and it’s very hard to do that in the field.

Lyle Kantrovich pipes in on why Rolf Molich’s CUE studies are really interesting

Jared: But this brings us back to what Kyle Pero were saying. Because, I think, what is a key element of this is that the client doesn’t realize that they have different results depending on which team comes through. We’re not pitching it that way. Let me put it another way. When Ralph ran CUE-4, he had 17 teams look at, I’m trying to remember, oh it was…

Kyle: Hotel reservations.

Jared: Hotel reservations system, the U Penn system. Not the Penn, but the Hotel Pennsylvania or whatever it was, I don’t remember what the hotel was. But he had people look at this hotel thing that was done by the iHotelier folks, and the 17 teams, approximately half of them, did heuristic evaluations, half of them just happened to work out to do usability tests. They found, across all the teams, they found 61 errors, 61 problems with the design, that the client, the iHotelier people, thought were critical designs, critical problems. Things that they said, these are things we definitely have to fix. But each one of those 61 problems was only reported by one team. So in order to have collected all 61 problems, if the iHotelier people, Jim Whitney and the folks at Webvertising, were to hire a group to do it, they’d have to hire all seventeen teams to get those results.

Lyle: Well, this is why I think those studies are pretty interesting: I think that, by doing these kinds of studies, we will learn what helps us find more problems, what helps us find more critical problems, what we mean by those things. But also, frankly, if you’re iHotelier and you hire only one team, and you find, say, ten critical problems, aren’t you still in a better place? So I don’t think it brings in a question of value or the validity of these studies; but these will help us refine our art, our craft. I don’t think it’s a science: I think what we do is craft; it’s part science and part art. I think it gets better.

Kyle: What the studies really show is that it really does matter whom you hire.

Jared: Exactly.

Rashmi and Nate Bolt discuss more about the scientific nature of our work

Nate: Well I was just thinking about the discussion we were just having and it almost sounded like there was an implicit assumption that if you were a scientist that you follow certain rules and that you should come up with the same results as everyone else. But that would also lead to the idea that all scientists are created equal and we know that that’s not true because scientists like Darwin and Kepler, Galileo, they all had something that no one else had. They had this sort of insight to translate what they were seeing and the math behind it into something new. I would just add to that, before we ultimately kill it, I would just add that all scientists aren’t created equal either and we really can’t get away from the person that’s doing the process in the discussion that we’re having.

Rashmi: Correct. When you teach courses and you do even basic scientific experiments and you have every student in the class replicate that, you do find deviations. They might be deviations around some kind of thing that might form a normal distribution that there’s this one most likely result and then there’s deviations from that. But the method and the way that people carry out the method is different every where. And also in science there’s an explicit concept of the meta kind of research where you look a bunch of the different studies on a topic and you make some logical conclusions. So that once again acknowledges that not every result done addressing the same exact question is going to come up with exactly the same answers.

Josh Porter on the Facebook Controversy

Josh: Well I think Rashmi makes a great point, that this is kind of a new sort of social design issue. The really ironic thing that I just read about this morning was that the actual implementation of the feature itself caused its own downfall. Because as you were looking on your Facebook profile, you could see that all the other people, all the other people in your network, were signing up for the protest petition. So that’s how that petition was signed by like 700,000 people in 24 hours, because the new feature itself propagated it so fast, so I found that really interesting. But one thing that I don’t think any of us have mentioned as well is that the privacy settings of everyone’s information actually stayed the same. So when, if you were seeing information in your news feed, that was information that was already available to you. The only difference is you had to work much less hard to see it. It’s kind of like moving from looking at HTML home pages to going to RSS. There’s just that much less effort in seeing information.

Jared: The MoBuzz TV people, said that this is like people who, somebody who stands naked in their window every morning, who gets upset because someone handed them a photograph of them standing naked in their window.

Read the entire transcript here.

This entry was posted on Friday, September 29th, 2006 at 12:51 pm and is filed under Design, Events, Experience Design, Experience Management, Podcasts, SpoolCast, Usability Toolbox, Web 2.0 . You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.