October 5th, 2012
Thanks to Marco Dini for translating this post to Italian.
“What are some good KPIs I should use to measure our UX?” This is a question that comes in quite frequently and, at its heart, it shows that they are confused about what a KPI is.
A KPI is a Key Performance Indicator. It’s a metric that helps the business know if things are on track or if something is about to change.
A metric is something you can measure. You can measure and track the time a user spends on the site or whether a particular page has a high “bounce rate” (which says that it’s often the last page users visit). You can measure how much revenue the site makes. You can also measure how many times the letter E is used on the home page.
All of these things are measurable. However, just because they are measurable, it doesn’t mean that those measures are informative. (Most people would agree that counting the times an E appears is probably not useful for making decisions. In my opinion, bounce rate is about the same, but others believe it tells us “something.”)
Key Performance Indicators use metrics, but not every metric is a good candidate for a key performance indicator. Let’s take the term apart.
At the center is “Performance”, where the metric talks about how we’re performing. Knowing that the letter E appears 45 times doesn’t say how the business is doing. Knowing that sales were at $542,842 last week could tell us how the business is doing. We want metrics that talk about our business. However, just doing that still doesn’t necessarily make it a KPI.
What we’re measuring needs to be important — that’s the “Key” part. If we’re a business that’s all about selling things, then sales are probably important. It’s likely it’s more important than the number of new visitors to our web site, because unless those visitors buy something, we can’t pay the bills. (Visitors are important — no visitors, no sales — but they aren’t as important as sales.) There are many metrics that aren’t key to the business, so we need to pay less attention to those.
Finally, to be useful, the metric needs to predict the future — that’s what makes it an “Indicator.” If sales is the thing we’re most worried about, it really isn’t a KPI because it’s change doesn’t give us any warning. We need to find something that will predict what sales we’ll get. Hopefully with a decent lead time, so if it looks like sales will go down, we have time to do something about it.
One subscription-based business we worked with had two surveys they used as their KPIs. In one survey, they asked new subscribers how they heard of the service. When we were working with them, 83% said they heard from friends and family members. The other survey asked existing customers how often they recommended the service to their friends and family members, which was coming in around 92%. Nice high recommendation numbers.
More importantly, the business folks had done a lot of work to prove that when these numbers changed, they saw a change in their recurring subscription fees about six weeks later. If these numbers went down, the fees decreased in six weeks. If the numbers went up, the fees increased. Like clockwork.
Solid KPIs give us time to react when something really important is about to happen. You can’t use just any metric for this.
It takes a real flexing of statistical muscle to figure out what a business’s KPIs should be. It’s not picking metrics off a list. You need to collect years of data (to account for seasonal and cyclical variations), then have to do massive analysis to see what truly and consistently predicts the important changes in the business.
KPIs, when you get them, are truly wonderful. However, don’t think that just any metric will do.Tweet