Comments on: Beware of Poorly Formed Inferences

By: Jared Spool

Jared Spool — Thu, 21 Sep 2006 11:45:47 +0000

Hi Anne,

Sorry I reported it wrong. I thought I’d read the article correctly. Didn’t realize there were two studies.

However, it still doesn’t explain the wacky inference about users being trained by search results to skip images. That’s more what I was commenting about.

There’s no evidence in any study we’ve run that suggests this inference is remotely true.

By: Anne Holland

Anne Holland — Wed, 20 Sep 2006 21:04:30 +0000

Jared — Glad you liked the Case Study. However, your blog is a tad confused. The Case Study actually talked about two sets of design tests. The first was a graphical horizontal design (you do not show in your blog) versus “version B” above. In that test version B (the textual vertical column) won. And that’s why the CareerBuilder folks were talking about in the quote you mention in results.

However, next they conducted a SECOND set of A/B tests — in this case between the two versions you show in blog. In that case the longer verbiage won for text links. And in that case it won because people’s eyes are skipping around so you need to be ultra clear in your click link…

So, the CareerBuilder guys weren’t quite as clueless as you make out — the quotre you quoted was their conclusions from a test you did not picture.

By: Terrence Wood

Terrence Wood — Mon, 04 Sep 2006 00:43:24 +0000

The visual design has little to do with it IMHO. Sure version ‘C’ is denser which may aid in scanning, but I’d wager it is actually the link text that makes it more successful. As an added bonus ‘C’ is more accessible due because it less ambiguous than version ‘B’.

By: Kelly B

Kelly B — Fri, 01 Sep 2006 15:33:46 +0000

I agree with Graham, in that there are not extra links either. So not sure what Jared is referring to in this statement, “he probably would’ve come to a different conclusion about why users prefer the page with the extra links.”

Perhaps the wordiness of the links gives the illusion that there are more?

Regardless, I do agree that A/B tests are not as effective as when combined with user testing because you are already forcing the user to choose the best of 2 scenarios, where as maybe either aren’t great at all. Nothing beats talking to the users first, creating a layout based on user needs, then watching the user perform a function/task with the presented, and getting their direct comments/feedbacks, and revising based on this feedback.

By: Karl N

Karl N — Fri, 01 Sep 2006 14:03:07 +0000

When you stack links on each other like that your eye can look in one spot and see parts of all three links without moving their eye. When you put them on a single line then you have to read the whole line with your eye before you get an idea of what’s there.

Stacking text works much better visually because text lines are so thin. That’s the obvious reason why “C” prevails for me — you can read it faster and they links look more associated with each other.

By: Graham Storrs

Graham Storrs — Thu, 31 Aug 2006 05:32:50 +0000

I agree with your comment that, if he’d done usability tests he’d have had more information to work with but I’m confused about how you picked ‘C’ as the winner based on the number of links. As far as I can see from the two images, there are exactly the same number of links in each case. The only differences are the wordiness of the links and the horizontal vs vertical layouts. There doesn’t seem to be any reason to conclude – on a count of links alone – that one would be better than another.

Or did I miss something?