UIEtips Article: Watch and Learn: Recommendation Systems are Redefining the Web

Jared Spool

December 13th, 2006

UIEtips 12/13/06: Watch and Learn: Recommendation Systems are Redefining the Web

Whenever I’m in an unfamiliar city, I always ask someone at the hotel, the bellhop, doorperson, or receptionist, what their favorite restaurants are. In my experience, this is a foolproof way to find the best restaurants. Instead of going online and searching for some place, or looking in the phonebook, I ask somebody. I don’t want just any old answer. I want a recommendation.

Instead of spending hours sifting through a myriad of data, people look for recommendations in order to save time and frustration while researching a certain product, place, or service. How many times have you been asked what you think of your car, a vacation spot, or cell phone provider? Valuable, and more often than not, reliable information is only one question or click away.

Recommendation systems are becoming an extremely important business and marketing tool for many web sites. These systems use sophisticated algorithms to record user behavior, find correlations among the data, and produce recommendations based on them. What better way to entice a user than to predict their tastes and preferences? Netflix, who rents two-thirds of their movies through recommendations, has even gone so far as to offer a $1 million prize to anyone who can improve the current version of their system by 10 percent.

In this issue of UIEtips, Josh Porter dives into the fast-emerging world of recommendation systems. You’ll discover what Josh thinks are the most important benefits of these systems, what their serious drawbacks are, and where recommendation systems will be going in the future.

Read today’s UIEtips article.

What recommendation systems have you encountered? Have you been delighted, offended, surprised, or unfazed by the recommendations you received? Let us know what you think. Leave us a comment and join the discussion below.

[If you find this article interesting, I encourage you to join us in Monterey, California this January for our UIE Web App Summit. Josh will present his short-talk, Learning from Social Web Applications, and Rashmi Sinha of Uzanto will be presenting Design Strategies for Web-based Recommender Systems. You don’t want to miss out. See the summit website for more details.]

11 Responses to “UIEtips Article: Watch and Learn: Recommendation Systems are Redefining the Web”

  1. Adam Smith Says:

    The problem with recommendation systems is the same problem that exists with Server Log Analysis – it measures and acts upon the effect, not the cause.

    The system is dependent on the frequency of an act being directly related to a specific intent. People have a tendency to make the leap from “this happens a lot” to “this happening because….”

    Look at the the last.fm example. The Beatles are going to come up as being popular in pretty much any tool that measures popular musical acts. Now if last.fm has a tendency to be popular with the demographic where Radiohead has their strongest representation then they will also appear in the top ten list. This does not mean that, in general, people who like Radiohead will also like the Beatles. It means that amongst that sampling there is a coincidence.

    Look at the rest of the list. It reads like a shopping list of the standard classic rock acts. Now is it possible that there’s another reason why Radiohead shows up as number two? Occam’s Razor would suggest that the reason Radiohead is number two in that list is -not- that they are the second most popular rock music act in history. Starting with that assumption, I think you need to dig a little deeper to understand the results, rather than taking the assumption that it’s all correct and then making another leap to create a reason for their presence in that list (particularly a reason that assumes that the system does what you’d like it to be doing.)

    Recommendation systems do nothing to understand why a person chose one thing over another or what the connection is between two things that they’ve chosen, they simply report on the frequency, in a given situation, of that happening.

    As such, recommendation systems are really only good at reliably telling you what the majority of people do most of the time. Sometimes that’s all you need. They will be good at recommending Pearl Jam if you like Nirvana, but they are, by definition, incapable of understanding -why- you like Nirvana and making any kind of intelligent recommendation.

    That’s the kind of thing you still need a human being to do.

  2. Dan Says:

    Take a look at Pandora.com for an interesting and relevant take on music recommendations!

  3. Stewart Walker Says:

    I’m with Dan. Pandora is fantastic.

  4. Joshua Says:

    Adam, I agree completely. Thanks for the excellent analysis.

    You’re right about Beatles and Radiohead. It could be that this is simply an odd correlation that happens to exist because lots of Radiohead fans use lastfm. But, as more bands we expect to see show up, such as Bob Dylan, Led Zeppelin, etc., the more Radiohead stands out.

    Now, as I mentioned in the piece, recommendation systems aren’t always right. And this might be a case where the system is somehow skewed in the wrong direction. But maybe it isn’t, and maybe Radiohead fans do have something in common with Beatles fans. That’s why it’s a recommendation, not a declaration that you have to like them. Just a suggestion.

    Also, I’m not exactly sure what your claim is about the Beatles always being popular. What Last.fm is saying, and *all* it is saying, is that for those people who play the Beatles, they also play a lot of Radiohead. This doesn’t have to do with the overall popularity of the bands, but more to do with the frequency with which each are played by the same person.

    In the end, the aggregation is about behavior, as you say. And that’s actually the strength of recommendation systems. It only looks at behavior, so isn’t skewed by a person’s biases about what they *think* they like. (social psychology is littered with biases that humans exhibit). There will always be anomolies, but for the most part, past behavior is a fantastic indicator of future behavior.

  5. Michael Grossman Says:

    Yesterday my wife said she got a babysitter for our 2 young children and we could actually go out and eat dinner alone like adults. Now all we needed was a destination. Chowhound.com has been around for a while, and comes in handy at times like this. It is a website where ‘foodies’ congregate and share thoughts on restaurants (although they HATE being labeled foodies).

    You can never tell if you will like a restaurant by a typical Yahoo search or menu portal, just by seeing what food they serve. Chowhound.com lets people publish details of what makes places special, or a great experience. So you can search your region for a type of food and find out what place other people think have the best fried dumplings or tastiest gnudi for example.

    So there is a person on Chowhound.com that raves about this special spicy beef noodle soup and other dishes at a local Vietnamese restaurant. We go and end up enjoying it a lot, but I know for certain that I would have never gone there by looking at the menu or seeing the décor. It was the story. It was the experience. It was the anonymous recommendation from a fellow, ahem, Foodie.

  6. Lindsay Ellerby Says:

    I love the tablethotels.com recommendation system. The thing about it is I know, for certain, that I can trust the quality of all the hotels posted on their site. The recommedation system just augments that, getting into the nitty-gritty of what’s specifically great at each hotel.

    The people scoring, or recommending, the hotels are Tablet Hotel guests who have stayed at the hotels, not hotel General Managers or concierges.

    User trust in the recommendation system is key.

  7. Jim Burrows Says:

    The problem I have with automated recommendation systems is related to Adam Smith’s comment about the Beatles being popular. the issue is that major hits will be often be recommended regardless of whether they have any causal connection to the source item.

    Suppose, for instance you go to Amazon and look for the DVDs that correlate with the book “You: On A Diet”. You’ll find that they include The Da Vinci Code, Cars and the second Pirates of the Caribbean movie. Now look for the DVDs that correlate with Barack Obama’s The Audacity of Hope. What you’ll find is that it correlates with The DaVinci Code, Cars, and Pirates of the Carribean. And in fact, if you look for books that correlate with You on a Diet, you’ll find Obama’s book on the first page. Grisham’s The Innocent Man also correlates to the same three DVDs.

    So, what’s common to the three DVDs and the 3 books? They are all big time hits at the moment. Being interested in “You on a Diet” probably doesn’t really indicate that you’ll be interested in Obama’s book or the DaVinci code except to the extent that any one of these indicates that you are aware of American popular culture.

    The difficulty is that unless you are very careful in how you set up the correlations (and Amazon actually does a pretty good job of this, so their correlations are better than many, and why my examples above are cross-media, within media they do quite well) you end up saying “If you like X, then you’ll like “, solely because a noticible fraction of any random selection of buyers of anything is likely to by the huge hit.

    Systems that allow users to tag each item or which have a built-in classification scheme can often avoid this, by providing multi-dimensional correlations. Thus, for instance, Pandora makes some very interesting recommendations.

    If recommendation systems don’t take this issue into account, they just push people towards the hits, but if they do, they can really help individuals find the little gems that delight them and bring them back.


  8. ido levran Says:

    whenever i think of going to a movie i check out http://www.imdb.com (internet movie database) .
    on the site, people can grade movies they saw on a scale from 1 to 10.
    since the site collects thousands of votes, i always find it to be a reliable source of information on how good the movie will be. i have a rule of thumb that if a movie recieved over 7.0 it will be good.
    I think the power of recommendation sites comes from the amount of users giving their 5 cents and the accumulation of all the opinions.
    until now IMDB hasn`t failed me…

  9. Adam Smith Says:

    Just to reiterate and clarify a little, I’m not suggesting that recommendation systems don’t work. I just have a concern that they are being imbued with more power and cleverness than they deserve.

    Like so many things, the more typical one’s request, the more accurate they are likely to be. They reward common behaviour. As such, the further one is from being in the middle of the bell curve, the less effective they are.

    And they are a reflection of effect, not cause. So comments like “the intimate knowledge of what we like and don’t like” make me nervous. There is nothing intimate or insightful about the process.

    As others have pointed out, recommendation systems will tell you what most people do, most of the time. And most of the time that’s useful. Sometimes there will be patterns in those data that reveal interesting things, but is the story always what it appears to be? As the saying goes… it depends.

  10. Daniel Szuc Says:


    makes reference to recommendation systems – “In an effort to help users “competitively produce” an answer, Sun’s Labs have come up with a lingusitics analysis technology that the company’s researchers affectionately refer to as the “Blurbalyzer” and the example given cited the way Amazon recommends books on its Web site today.”

    “If for example, it suggests Holy Blood Holy Grail to buyers of The DaVinci Code and those customers act on that recommendation which in turn helps Holy Blood Holy Grail stay on the list of recommended reading, does that mean you’ll like the book just because you liked The DaVinci Code? It’s hard to say, but Sun thinks there’s a better way based on a linguistic analysis of the reviews that people have written for the The DaVinci Code and other books.”

  11. Joshua Says:

    Thanks for the clarification, Adam. One point that I couldn’t really fit into the piece was the distinction between implicit and explicit data.

    Implicit data is that which you refer to as “effect”. Observing what people do, essentially. And you’re right, this doesn’t really get to intention, or likes and dislikes very well.

    Explicit data is the type in which people make their preferences explicit, like a ratings or a review system that is aggregated. Many of the movie recommendation systems do this. This type of data does get to intention, as it explicitly maps what some likes and dislikes. For example, Netflix knows not only what I watch (implicit) but what I like (my ratings).

    Taking this distinction further, and agreeing with your observations about implicit data, I think explicit data can help solve the issue you bring up. With both implicit and explicit data, we can get real insight into both the subjective opinions of users as well as the objective observations.

Add a Comment