The Freedom of Fast Iterations: How Netflix Designs a Winning Web Site
Originally published: Nov 14, 2006
"We make a lot of this stuff up as we go along," the lead designer
said. Everyone in the group laughed until he continued, "I'm serious.
We don't assume anything works and we don't like to make predictions without
real-world tests. Predictions color our thinking. So, we continually make this
up as we go along, keeping what works and throwing away what doesn't. We've
found that about 90% of it doesn't work."
We were talking with the design team at Netflix. Netflix.com is one of the
most successful web sites in the world: the sole customer interaction point
of their home DVD-rental business. Over the past 9 years, the site has grown
from nothing to serving almost 6 million customers who use the site to prune
their rental queues, rate movies, and handle any billing and transaction issues.
The designers of Netflix.com have a smashing success on their hands, but we
didn't find them resting on their laurels. They want to get even better, and
for them that means iterate, iterate, iterate.
Netflix isn't the only company using a fast iterative design approach. Google has also gained attention for their unorthodox design methods, with many people
complaining that they have a huge stable of products, but only a few they've
designed well. Yet, Marissa Mayer, Google's Vice President for Search Products
and User Experience, explains how trying out lots of ideas helped Google design
a better toolbar:
"In the case of the Toolbar Beta, several of the key features (custom
buttons, shared bookmarks) were prototyped in less than a week. In fact, during
the brainstorming phase, we tried out about five times as many key features
-- many of which we discarded after a week of prototyping. Since only 1 in
every 5 to 10 ideas work out, the strategy of constraining how quickly ideas
must be proven allows us to try out more ideas faster, increasing our odds of
This "try and see" attitude is taking hold. The designers at Netflix
told us they try out many new features with every site iteration to keep pace
with the rapidly changing landscape of the Web, as well as their customers'
increasing comfort with the current site. Much of what they do try doesn't
survive to the next iteration.
So how often does Netflix update its site? Every 2 weeks.
Every 2 weeks they make significant changes. They understand that some of
the changes will work, but most won't.
At first, this sounds like a frustrating design constraint. In talking with
the team, we realized that it doesn't frustrate them at all. Instead, it frees
them up to be flexible and adaptive, so they can react effectively to customer
needs. As a result, they don't deal with the many "when we redesign" issues
that so many of us deal with in the design world. They're building for the
present -- all the time.
Besides living in the now of their web site's design, The Netflix team gets
other benefits to fast iterations. Here are a few our research has uncovered:
Benefits of Fast Iterations
- Fail Fast
A major benefit of fast iteration is you also fail fast. Failing fast means
you invest less time in the things that don't work. If you quickly find out
what works and what doesn't work, then you take action to turn it into something
that does work.
Ironically, teams that fail fast improve as fast, if not faster, than those
who try to get it right the first time. The reason is simple: Teams trying
to get it right the first time fail as often as everyone else does. However,
when they fail, they fail really slowly and struggle to pinpoint problems because
they've changed so much at once, making it harder to identify solutions
- More Experimentation
The faster you fail, the more experimentation you can do. You can try out
ideas that might not have a lot of support, but could be potential winners.
This allows for an innovative environment.
Perhaps you've heard of Google's 20% time? They expect their engineers to
work 20% of their time on a personal project -- an experiment they find personally
interesting. This program has the effect of bootstrapping experimentation,
so it will happen more often.
- Learn Quickly
We've all had the experience of sitting in meetings arguing whether something
will work or not. Usually, both sides just don't have enough data to go on,
and they end up going with their gut or with the loudest arguer (for better
or worse). Fast iteration helps solve this problem by giving developers a platform
on which they can test quickly, helping to collect data about any outstanding
questions instead of resorting to opinionated arguments.
- Provide Continuing Interest
In addition to improving your design, fast iterations may have a psychological
effect on users. Those users who use your app with any frequency will notice
the changes, and if the good ones stick, they'll appreciate your ongoing efforts
The best teams not only design the changes, but design the process for introducing
the change. They experiment with methods to overcome the users' natural resistance
to change, providing migration paths and clear benefits for each improvement.
- Reduced Risk
Quickly iterating helps reduce risk during design. If teams can make many
small changes instead of a few larger ones, they mitigate risk because they
know which changes have what effect. If a design team makes many changes at
once, they have a harder time knowing which parts work and which parts don't.
When you make only one or two changes at a time, you know immediately what
effect it has. Reducing risk is a valuable outcome of moving to fast iteration.
These benefits don't come easy. There are significant changes design teams
have to make to their core process to iterate quickly. It's not a switch a
team can turn on or off.
- Culture Change
Most designers are accustomed to long release cycles. Fast iteration and fast
evolution of design creates a different kind of design environment.
Gone are the grand visions of the redesign, where teams spend months retooling
vast areas of the site. Replacing it is the idea that the site is a living,
breathing design that needs constant care and attention. The team at Netflix
calls themselves "compulsive data wonks". They rarely dream very
far in the future. Instead, they're concerned with what's happening right now.
- Design Determinism
When teams make the switch to fast iteration, it changes the site's testing
methodology. Testing becomes ongoing. After a release, you test for a certain
period of time to determine what to keep and what to throw away. Then you start
the process over again immediately. And repeat.
To some designers this sounds overly deterministic: Doesn't this take the
fun out of design? If all the decisions are cut-and-dried, what does that say
about creativity? What about longer-term effects? Is it possible that some
features take longer to catch on than others, and that an early flop might
not mean it's not a valuable feature? With fast iterations, if the feature
doesn't work now, then it's not right for the site, no matter how creative
- You're Either With Us...
Netflix's Chief Talent Officer, Patty McCord, told us their process of fast
iteration causes uncomfortable situations for some designers. Once, a designer
had spent time and energy working on a feature that testing showed didn't work.
When it came time for the team to remove the feature from the site, the designer
was distraught. He had become too emotionally invested in his design, and it
got in the way of his job. He ended up parting ways with the team and moving
on. Unfortunately, the process of fast iteration affected more than the product
Release Early, Release Often
Eric Raymond, in his famous work, The Cathedral and the Bazaar, wrote the
words organizations like Netflix live by: release
early, release often. At the time, Raymond was referring to software
created in the open commons, by thousands of people, with little or no delivery
This dictum now applies equally well to web-application development. Fast
iterations provide the freedom to innovate because teams can test more features.
This reduces the risk of each tested feature. However, it takes getting used
to and not everyone adjusts so easily. As with most design processes, teams
get the most benefit from quick iterations when they fully embrace the process.
Join over 25,000 subscribers to UIEtips, our free email newsletter
- Original articles by Jared Spool delivered to your inbox
- Podcasts that help to improve your UX skills
- UX Insights from the brightest minds around
- Awareness of all things UIE