To summarize: I think that eye tracking is not a simple tool to allow for a direct measure of usability. But properly used and in combination with other usability testing techniques it can give insight into usability issues which would otherwise be hard to find out.

Now that it’s less intrusive time-wise and, I imagine, possibly a bit cheaper, is it any more useful to the usability profession?

I don’t think so. We still don’t know how to read the data. It depends on (what I believe to be) sketchy interpretations to bring meaning and you still don’t learn anything you couldn’t learn without the device.

It’s certainly still new technology, and even now I think we’re not sure what to truly make of what it can do. Perhaps it’s up to universities to do some research with it, particularly in concert with psychologists, to tell us what we are and aren’t likely to be able to take away from results.

Bah! I’m ranting. But I’ll stop now and say that if IA could come up with something that looked cool to clients and actually delivered useful results then we’d have cracked the toughest nut. Maybe we should register brain waves.

this view sounds like the same view the industry has been espousing of eye-tracking for the last five years. Things have definitely moved on, but often this industry’s views dont! We’re now looking at methods of interviewing with eye-tracking that combine the best of what the eye tracking data has to offer with the best of protocols like talk-aloud have to offer. Most reputable companies selling eye-tracking technology will suggest combinations not JUST eye-tracking so let’s not evaluate it as a standalone.

Eye-tracking can be useful in ways that extend beyond the company web-site. It can be useful to also evaluate communication material… In a large corporate do users really read the instructions on their system, if they are struggling with a screen… and it has instructions on it… how did they fare with the e-learning training that was deliverd when the software was rolled out… tracking eye movements across screens can help you understand if users are really reading these elements. that is one specific example…

the equipment can be used to test materials that are web-based, paper based (advertisements, magazines), any kind of software at all. The thing I like about this is that it takes your client’s business as a whole and, through one tool, you are able to gather very interesting data. Think about evaluating someone’s use of a consumer goods web-site, magazine advertising AND marrying that data up with their movement through a grocery store as they scan the shelves for their purchases…

Is it a GREAT tool for web-site design… it depends how good your designers are… the more user-centred you are in your design and methodology the less you’ll need to do testing full stop… but if, like I do, you’re dealing with the whole business, not just a web-site, tools like eye-tracking can really expand the range within which your user-experience expertise can be used…

What’s more… the technology is fast to set-up (5 minutes), calibrate to a user (2 minutes) and learn to use (me… from scratch… 20 minutes)… As an outright purchase in a lab I think can show it earning it’s money back in about a year through the expansion of applicability, meaning that it’s use in the standard website tests is covered.

all the best

Martin Dowson

Of course, you’ll need to do user testing to see what works and what doesn’t, but it is not in tracking the visitor paths. The matter is in visitor and site interaction.

Like any tools, its use is limited by appropriate application. I believe Eye Tracking adds value IF used appropriately - that is, with the context of other quantiative and qualitative data.

Keep in mind that:

-Calibration with a new user takes less than five minutes

-Eye Tracking can be done simultaneously with other usability methods (e.g. think alound protocol); they are not mutually exclusive

Only in the fuzzy world of Web usability would a tool that actually provides valid (accurate) and reliable (repeatable) measurement be considered a potential impediment.

With eye tracking, you have to schedule more users (because not every user works and you can’t tell until you sit them down in front of the device), you have to shorten your test times (because calibrating with the machine frequently eats up 15 to 20 minutes), you have to lengthen the data analysis (because the current tools for analyzing eye tracking data are crufty and open to wide interpretation), and when you’re all done, you don’t know a whole lot more than you did before.

That’s my take.

If you want to buy a nifty new toy, I recommend the wide selection at Think Geek. You can get something wonderful there for under $50 that won’t affect your testing costs.

I’d need to compare eye-tracking to the alternatives to see what level of relative performance it provides when. It’s hard to imagine a making a decent cost-benefit analysis without at least an estimate of that. But, yeah, make some attempt to look at the costs and benefits. Don’t buy a gadget just because it has stirred a buzz.

There are many other ways to discover if the reason people don’t click on something is because they aren’t “seeing” it (such as oral interviews or retrospective analysis). These are much cheaper than the added expense of eye tracking.

I know these devices are all the rage. I just think our client’s money should be spent on better things.

On the other hand, eye-tracking combined with other data can be very helpful. For example, if users are not using a specific menu option that they should for their task, you can look at eye tracking data to see if they are even looking at it. If not, then one solution is to move the menu item to where users *are* looking. If users are already looking at it, then the more likely improvement is to give it a more meaningful label.

Eye-tracking is just another dependent measure. Whether an eye-tracker is worth the expense and hassle depends on what questions you’re asking and whether you are getting satisfactory answers using your current techniques.

When someone is gazing at something, is it because they want to look there? Or because the page made them look there? Or because they are resting their eyes there?

If they are just resting their eyes, or focusing on something- does it *always* matter? I would use eyetracking for the “hunt” that goes on -it seems most useful in during the part where the user is actively moving around looking for the next step to accomplish the task. Where they are resting or focusing is less important than where they decide to go next. I will admit though- I can see how this would skew the heatmap generated at the end.

Participants often would acquire the scroll bar without looking at it.

ok- but scroll bars are a bad example to use here. We all know it’s “somewhere to the right.” Assuming the window is maximized, we could probably all get there without looking right?

Participants would orally tell us they couldn’t see something their gaze was focused on.

But isn’t that useful to know as well? Sure you could space out and just not *see* but I have to imagine that’s a “less than often” occurance no? The other aspect could be that your using a image, word, etc. that is not universally, or culturally known. So say- you have a graphic of a mailbox for your email link (assuming this is 1995 of course ) but your testing it with a user that has never seen a “mailbox” before but knows what email is. So in that case, the user could be looking right at it, but not see it. Isn’t that a useful piece of information to know?

Participants often would click on objects they barely gazed at.

Like the scroll bar? ..the back button of a browser? …the “home” link they’ve already clicked on six times this session? In other words, are they barely gazing at things they are already well familiar with?

I’ve only seen a handful of eye tracking studies, so I’ll certainly take your experiences over mine. But your points from 4 on (I don’t think) paint a strong enough picture against eye tracking.

The folks at Wichita State came up with a much cheaper solution to collect that data.

And it’s not clear knowing expectations are that useful anyways.

That is, we have confidence to place the component on this section of a design because we know people look there regularly for that component type.