Originally published: Dec 17, 2007
UIE’s Jared M. Spool had the chance to talk with expert user interface engineer, Sean Kane, to discuss the benefits of a truly usable web site, measuring web site changes, moving from a large corporation to a start-up company, rapid iterations during product launch, and hiring the right people.
Sean Kane is co-founder of GetListed, a new start-up company that is set to revolutionize the way people find and fill jobs in the employment marketplace. Before starting GetListed, Sean was the former Director of User Interface Engineering at Netflix, where he led the development of Netflix's award-winning web user interface, which was rated #1 in customer satisfaction by independent researchers five consecutive times.
UIE: During your time working as Director of UI Engineering at Netflix, the service grew tremendously. How many users did Netflix have when you first started?
Sean Kane: I was at Netflix for five years. During my tenure, Netflix grew from 500,000 to about 7 million members. I saw the company move from its old little building that was falling apart to its new glamorous headquarters. There was a lot of change over those five years, a lot of it for the better. I learned a ton about the way to build really fabulous web sites.
UIE: Netflix is a business that runs entirely off its web site. If the site isnít working, is the company completely incapacitated?
Sean Kane: To some extent. But thereís also an offline component to the Netflix experience, which is the shipping portion of the business.
With Netflix, users build a queue of movies they want to see. Then, in a separate system, the movies get shipped off to the user through postal mail. If the site is down because of a technical problem or update, the shipping portion still carries on. People still can receive and return their movies. They just canít pick them out on the web site. Unlike Blockbuster where people can go into the store, Netflixís website is the only interface users can interact with. Users have the option of calling customer service or using the website, and thatís it.
UIE: Did you find that the management team at Netflix really understood the value of the website and why it had to be usable?
Sean Kane: Absolutely. In my experience, Netflixís management is as good as it gets. Reed Hastings, the founder of Netflix, is a very smart guy who focuses on data mining and letting data prove everything. As a result, we were driven to create a compelling user experience across the board, with everything from picking a movie on the website to getting it shipped within a day to your house.
We wanted to create an incredible experience where brand new people show up to the Netflix site and think, ďOh, this might work, this might not.Ē But then, when the movie appears on their doorstep the next day, itís always an amazing thing.
We not only wanted to create that experience, but we also wanted to make sure that we deeply understood the ďwhyĒ of everything that was happening. We monitored and measured any user interaction on the site to ensure that the experience was a better one than any of the alternatives we worked with in the past.
UIE: Did Netflixís user experience team measure how design changes performed on the site?
Sean Kane: Definitely. Often times, someone would suggest a feature idea that involved a set of sub-features. There are often several different ways to implement the feature idea. When we faced situations with differing opinions on the best way to implement an idea, we took the approach of building each of those experiences out.
We would build four, five, or six different implementations of the same experience and try to determine which one would best achieve the anticipated business goals. We also tried to balance the business goals with our usersí goals. We had a great research lab where weíd recruit people and watch them use the Netflix site. We used the research to inform the subtleties of how we go about building experiences.
We focused on the quantitative data, such as looking at how people were interacting with site features, whether users were adding more movies to their queue, and if they got their first movie choice more often.
Typically, one team member would come up with an idea that everybody else would disagree with. At many companies, that might become a political battle where people angle to get their idea accepted. But since we built all of the different implementations of an idea, we could see which implementation was most effective. Thereís no room to argue when the data makes it quite clear. Itís been a huge learning experience watching how people interact with web sites.
UIE: You recently left Netflix and moved over to GetListed, a brand new startup. What is it like to be in a startup after being in this organization where you had all these resources? Has it been a shock?
Sean Kane: To some extent. The part that Iím really enjoying is that I have to wear many different hats in my new role instead of focusing on just the UI engineering of the site. Iím doing everything now and itís forced me to learn about many things I havenít given much thought to in awhile, including writing out product specifications, talking to vendors, and getting people in the door to review technologies. It takes a lot more personal time and I have to manage my time a lot more carefully because weíre still really trying to crank that big wheel.
The biggest challenge is handling all of our ideas. We have two years of features we want to build. We have to prioritize the most essential things to get out the door. Once out the door, we need to iterate so that weíre constantly evolving the product.
When we launch what weíre doing, itís not going to be polished. Itís going to have some quirky elements and some features that arenít fully realized yet. It will be very important after we launch to iterate constantly and show that the siteís improving. We want to keep our members engaged and interested in what weíre up to. I think that when users find a gem of a new site and constantly see it improving, they feel like theyíre a part of that experience.
UIE: Do you have a vision that goes out several years in terms of what your product is going to be like? Have you segmented your product down to the small piece of whatís going to be the initial launch?
Sean Kane: Yes. We want to prove that weíre on the right track as quickly as possible. This approach is similar to how Netflix thinks about things. We try to implement things as fast and cheaply as possible because we know our design decisions will sometimes fail. The more we can reduce the time and cost of those failures, the more we can attempt to get things right. If we spend six or eight months developing something and it doesnít work, weíll have wasted a lot of time and money, with nothing to show for it.
There are a lot of advantages to evolving an experience over time. If we conduct a bunch of rapid iterations on things, users arenít caught off guard by the fact that the entire siteís different and they have to re learn it all. The experience evolves over time versus just doing something thatís not going to change at all.
From my perspective, itís fine to release something thatís not 100%, because 100% is going to cost a lot of money and take a lot of time. If we launch something that does what it needs to do, within a few quick iterations it starts to do what it does really well. Weíll then start enhancing the product and making it much more robust for down the road. It allows us to get to the final result without spending as much money and we learn a lot along the way.
UIE: Is being in a smaller organization giving you any sort of flexibility that you didnít have before?
Sean Kane: Yes, definitely. At a larger organization, itís harder to get some personal, passionate stuff out because of the business priorities, unless I did it as skunkwork projects. But right now, I know what needs to be done. Itís up to me to set those priorities and figure out how everythingís going to happen.
UIE: As youíre choosing the people to be on your team, are you thinking about the functionality youíre building and trying to match people to the specific functionality? Or are you trying to build a more general team that is going to be capable of all sorts of things that you havenít even imagined yet?
Sean Kane: I learned at Netflix to always hire great, super bright people, because they have this innate ability to try new things. And itís not really that big of a challenge. They enjoy learning new things.
We want to build our product in Java using the Tapestry framework. We have some other things we may want to do with Adobe AIR or some other technology thatís out there, and so we have ďmust havesĒ for the technology components. If we decide to change direction in the future or try new technologies, if I recruit bright folks, they can change on that dime with us. Because one thing is certain: we are a startup so we need to adapt very quickly to any changes.
UIE: How do you find people who meet that criterion of just being really bright?
Sean Kane: When you post a job ad, thereís a lot of luck in play. Some of it is on instinct and some of it is when somebody actually starts proving themselves in their ability to deliver. You can get at some of that though references and through the interview and just getting a general sense about somebody. But at the end of the day, itís all about how somebody actually performs.
I also try to keep in good contact with the people I have interacted with in the past, or Iíve worked with before because your network is your best resource. So itís always important to keep in contact with really great people. You may not work with them within the first couple years that youíve met them, but you never know when things are going to change.
Iíve had some success stories with people that Iíve talked to for many, many months, and sometimes a year or so. And finally, when the situation was right, because that warm tie was there, I was able to get a great person in the door.
UIE: As youíre building out this team, are you thinking of it in terms of specific roles? Or are you going for generalists who are just really bright and can basically do everything?
Sean Kane: We have an alpha prototype built right now that was created by a designer and a few external consultants. We have designs, our logo, and some technology to help us prove our model and to make sure that itís going to work like we think it will.
We also have a secondary platform that weíre going to build to scale, which is our beta product. I have the luxury of leveraging the designs that have been put into the alpha prototype. At the moment, Iím not looking to fill designer roles. Iím working on the information architecture piece.
There are specific roles where Iím looking to place somebody that really knows the technology. However, thereís always room for a really great generalist that can get everything done and help everybody else in their various areas.
Down the road, weíll start looking more seriously at hiring designers and product managers and start growing things. Thatís when weíre going to start looking at the various components of the site and think more deeply about how we can implement the various features on our various pages.
UIE: At the Web App Summit in March, youíre going to talk about your experiences at Netflix and GetListed. Can you tell our readers what youíre hoping to share?
Sean Kane: I will be giving specific examples of the things I learned from Netflix that Iíve successfully applied at GetListed. Since we havenít launched a product yet, we canít do a lot of testing at GetListed yet. Instead, Iím looking at how we can achieve our end goals and get the site up and running before we have a lot of traffic.
One of the common questions I get when I give presentations is, ďSure, at Netflix, you guys have all this traffic, so itís easy for you to do the quantitative testing and get statistically significant results back. Iím a startup. I donít have very much traffic. How do I deal with that?Ē
I didnít really have an answer to this question in the past. But at GetListed, Iím figuring out a way to answer that question. At the Web App Summit, I hope to have that question answered and share what Iíve learned with attendees.
Read related articles: