Rolf Molich’s Comparative Usability Evaluation
August 9th, 2005
In my work, I’ve seen that usability testing is an extremely valuable tool. It guides the design of sites, provides information on the expectations of our users, and it gives a way to assess how close users are to achieving their goals.
But there still isn’t just one right way to conduct a usability test. Every usability team has their own unique testing method. They have their own techniques for creating tasks, recruiting users, facilitating the tests, and disseminating test results.
Over the past several years, we’ve been excited by Rolf Molich’s research at Dialog Design. Since 1998, Rolf’s conducted four Comparative Usability Evaluation (CUE) research studies to investigate the many different usability methods employed by teams.
The CUE studies are the first of their kind. Usability practitioners from all over the world are asked to evaluate the same interface, using their standard practices. Rolf compares the different results of the study participants, looking to see which practices are most effective at discovering and reporting usability practices.
The most famous study, CUE-2, had nine teams conduct usability tests of Microsoft’s Hotmail interface. Last year, CUE-4 had 18 testers (using both expert inspections and usability testing) looking at iHotelier’s Flash-based hotel reservation system.
Through the CUE studies, Rolf Molich has collected the methods, reports, and results of dozens of usability teams, pulling the best practices. At UIE, we’ve found this type of research to be very informative. We learn so much when we compare our own methods against our colleagues and peers.
This October, members of UIE’s usability team are looking forward to participating in Rolf’s CUE study at our User Interface 10 Conference. I expect we’ll learn a great deal comparing our own methods to our colleagues in the field.
January 29th, 2006 at 9:02 pm
Can someone help me understand what frame work they used for research. Same as what variable they used and how they analysis the data for comparative usability evaluation. I could not understand just reading the article what kind of research is that and how they evaluate the data.