Stop Doing Survey Research

Jared Spool

September 24th, 2015

Recently, Erika Hall published the article On Surveys where she emphasized the danger of using surveys as a research tool. She suggested that they’re often misunderstood and misused and frequently poorly done.

Here’s an excerpt from her article:

In my opinion it’s much much harder to write a good survey than to conduct good qualitative user research. Given a decently representative research participant, you could sit down, shut up, turn on the recorder, and get good data just by letting them talk. (The screening process that gets you that participant is a topic for another day.) But if you write bad survey questions, you get bad data at scale with no chance of recovery.

What makes a survey bad? If the data you get back isn’t actually useful input to the decision you need to make or if it doesn’t reflect reality, that is a bad survey. This could happen if respondents didn’t give true answers, or if the questions are impossible to answer truthfully, or if the questions don’t map to the information you need, or if you ask leading or confusing questions.

It’s worth reading the full article.

You can also hear Erika’s interview with me on team-based research and the need to shift away from an academic way of thinking about it.

Using a team-based research approach and skillfully analyzing the data is key to getting your team on the same page when trying to figure out who your users are and what problems you’re trying to solve for them. Erika Hall covers this in her full-day workshop at UI20 on Cultivating Shared Understanding from Collaborative User Research.

Add a Comment