July 15th, 2014
Understanding problems are common when trying to visualize data. Designing a layout to effectively communicate complex or even simple data can be a challenge. If the visualization isn’t immediately apparent to a user, it requires a level of understanding to get the most out of their experience.
Stephen Anderson has been working to unlock these understanding problems. He says that often times really simple changes can have dramatic effects on a user’s ability to interpret data. He cites the many examples of designers taking stabs at airline boarding pass redesigns and the evolution Target’s Pharmacy prescription bottle went through. Presenting the information in a much clearer way reduces the cognitive barrier.
In this podcast with Jared Spool, Stephen outlines what he calls the 7 Problems of Understanding. These range from problems of comprehension to problems of discovery and more. Each of these problems is usually brought about by a design or display decision. Looking further at these issues, simple changes can greatly increase the experience for users.
Check out Stephen’s UI19 workshop, The Architecture of Understanding, now in our All You Can Learn Library.
Jared Spool: Welcome, everyone, to yet another episode of SpoolCast. I’ve got to tell you this is going to be a lot of fun today. We have the amazing Stephen P Anderson with us. Stephen, last year, gave a topic on re-thinking the way you communicate the data in your applications in such a way to be more engaging and more exciting.
He just blew away the audience. He was a big hit for the day, and we were so excited to have him there. We have been crazy enough to invite him back for this year, for UI19, which is going be October 27th through 29th. He’s going to do the same workshop, except a year better. I’m very excited about that. We’re going to talk about that today.
Hey, Stephen. How are you?
Stephen Anderson: I’m doing great. How are you, Jared?
Jared: I’m doing fabulous. Last year, when you did this workshop, it was practically the first time you’d ever done it. No one had been talking about this stuff, and it really changed the way we thought about how we present all this stuff in our apps, how we get users to interact with that data, and the core content that is in a lot of the applications and designs we come up with.
In that year, you’ve been really studying this, talking to people about this, doing more workshops, and working with clients. Has anything changed? Has stuff crystallized in your head about this topic?
Stephen: Oh, yes. [laughs] Quite a bit. There have been several things that have really helped a lot of the ideas that…Even as useful and practical as the workshop last year was, some things have crystallized and gotten a lot less fuzzy for me, at least a couple things in there.
One, I did an iteration of this workshop at the IA Summit Conference. I really knew that I needed to disrupt it and disrupt my thinking in some fundamental way. Also, I’m learning about this whole topic of how humans come to understand things.
I invited Karl Fast to join me on the workshop, just because I needed something to [laughs] change the flow, change the activities, and deepen my own knowledge. Karl Fast is, of course, a professor at Kent State University, where he’s been teaching, speaking, and researching all things UX, but especially around data visualization, data interaction, for coming up on a decade.
That was a big catalyst and the conversations that we had. At the same time, I’ve been working on my book, which is “Designing for Understanding,” which will come out next year. As I’m writing through the chapters and doing the research, a lot of the ideas that have been fuzzy in the past which…As you know, you can get away with some inspiration in a talk, but when you sit down to write a chapter or do a longer workshop, you really have to have your thoughts be crystal-clear.
There are some thoughts that, since the summit, have really crystallized for me, particularly some basic thoughts around talking about these understanding problems. Once we know about how to talk about these, what happens next?
Probably the biggest change is in previous years I would open with lots of examples of understanding problems that we’re surrounded by. One I used in a TEDx Talk was about my son’s diabetes chart, his medical form, and how broken that was. Other ones I’ve used are more what we think of when we talk about visualizations — shopping for health insurance, for example, or shopping for a digital camera.
I talk about this breadth of examples, but then, in the afternoon…I was aware of it, even though I couldn’t put the right language around it. A lot of the activities centered around one of those visualization problems, one where you have lots of things that you’re comparing, like the digital camera and the health insurance options.
I realized I was neglecting these other kinds of understanding problems. That really sent me down the path of trying to explore, “What are some things or labels we could use?”
Jared: When you say “understanding problems,” you’re talking about things we have in our design, where, in order for the user to get the most out of the experience, they have to understand some pretty sophisticated stuff.
Stephen: Yes, absolutely — sophisticated or simple. There are simple examples. This is one that’s popular with a lot of graphic designers, redesigning the airline boarding pass.
Stephen: Then, of course, the complicated ones being data vis and you hear about the Internet of Things and how we have trillions of pieces of data collected every second — I don’t know if that’s right, [laughs] whatever the numbers are — and how do we make sense of those?
So, it’s all of the above, and the data collection is one half, but then the sense-making of the understanding is the other half that, for good reason, isn’t talked about a lot right now, but which is becoming the most important focus. “I have all this data. Now what do I do with it?”
Jared: When you were talking about the understanding problems getting broader for you, what are some examples of understanding problems that you missed that first time?
Stephen: I think I was covering the examples. What I had not done was really call out explicitly or name them. Where are the boundaries? Where are the centers for these different types of understanding problems? If you look, whether it’s change-management or any type of problem-solution approach, you name or label the problem to begin with.
What I set about doing was saying, “Let’s name or let’s label or classify, if you will, all these understanding problems, so that then I can go through each and say what’s the best way, what are the best skills to have to respond to each of these.
I came up with seven problems of understanding. My hope is that at least labeling or identifying these, we’re better oriented to know how to respond or what skills to use to respond to these.
This is very different from what you see in the industry today where people are still talking about data visualization problems or information visualization problems or information design problems.
I think that’s maybe the wrong way to go about it, because it’s defining the solution ahead of time. The moment you say this is a data visualization problem, you’ve already defined the solution. Maybe it’s not a data visualization problem.
Jared: So if it’s not a data visualization problem, what is it?
Stephen: [laughs] If you’ll indulge me, I’ll run through these seven problems of understanding.
Stephen: Yeah, just really quickly. I mentioned the boarding pass as an example. Just Google “redesigned boarding passes” and you’ll see lots of graphic designers who have done that.
If you remember a few years ago, the Target prescription bottle, medicine bottle. That redesign changed the experience of that moment when you’re interacting with your prescription pill bottle. Really small example. A very small data problem.
Jared: That was a pill bottle that, actually, a student did.
Stephen: Mm-hmm. Deborah Adler.
Jared: Yep. She had been working with the folks at Target. They loved it so much that they actually put it into production. Really, what it did was it solved some basic problems. Particularly for people who had vision problems, who had lots of medications.
She was mostly concerned, if I remember correctly, about her grandmother and her grandfather who, when she opened their medicine cabinet, had all these medications that looked identical.
They couldn’t figure out whose medication was whose, so they did simple things like using colored rings to say this is grandma’s medication versus this is grandpa’s medication. Then, making the print bigger, making the dosage more obvious. Stuff that no one had considered before.
Stephen: Yep. Really simple changes, but simple changes that had a dramatic effect on understanding. Some of the problems that drove Deborah to create that were some problems of taking the wrong bottle which is actually quite common.
I think for everyone, but, I think, in her case, she noticed it with her grandmother taking the wrong prescription bottle.
That classification of problems I would call problems of comprehension. Where all the information is there and it’s not very much, it’s just not been presented the way where it’s easy to understand, easy to know what to do. Or what to do next, in the case of the boarding pass.
The nutrition labels are an example of this as well. People are always redesigning or trying to figure out how to redesign a better nutrition label. That one is tricky in that there’s not one better solution, because our dietary needs are so different.
That one, when we talk about solutions for the nutrition label, is going to require some kind of interaction or digital interface. We can talk about that if you want, but there’s still a problem of comprehension where there’s a very small set of information that’s just not been presented in a way that’s easy to understand.
That’s type one. The second type of understanding problem would be probably the one that I’ve harped on historically, because I’ve seen the fewest people talk about it.
That’s the problem of comparison where you have a lot of options to choose from. Usually dozens, maybe hundreds. Definitely, I’d put it in the category of a small data problem, but the way the options are presented just doesn’t help you make a decision.
The end goal of a problem with comparison is to make a choice or narrow your options. That would include the health insurance plan, shopping for a new TV, picking a doctor, finding a mechanic to fix your car. All those types of things.
Even at the IA summit, the workshop activity we did was choosing a UX conference to attend. I think there’s what’s clocking at over 500, 600 UX conferences a year nowadays, and how do you find the very best ones like UI19.
The goal was to create an interface that would help people make comparisons. Not just based on the content and the workshop itself, but also rolling in things like how far is this from me, how much will my travel costs be, what are the hotels in the area. Things like that.
That’s a problem of comparison. That’s the second type. The third type would be…Bare with me on this one. This is a little bit different. This one took me a while for me to articulate.
I’m calling it problems of causal relationships. If you followed any of the stuff that Brett Victor has done where he’s shown, essentially, simulations from math, for example. He’s talked about how programming should have more of a tight feedback loop where, as soon as you program, you’re seeing changes. These are problems of causal relationships.
Jared: It could be as simple as adding to the checkout process. “Put your zip code here and we’ll tell you the sales tax and the shipping costs.”
Stephen: It is that. Although, if it’s really small and one dimensional like that, I would call that a feedback loop. But it’s when the feedback loops are spread over time, like you want to learn a math concept by playing with a simulation and interacting with it.
You see hundreds of tiny feedback loops, and so you see that cause and effect over time that reveals patterns. That would be the difference there.
Jared: Let me see if I understand this. I’ve been working lately, because I’ve got this project that’s happening in Chattanooga in Tennessee. We’ve been inviting people to come and move to Chattanooga.
They have to figure out whether they can afford to do that. We’ve been playing with these costs of living calculators where they put in all these different factors.
Then, they see where they in Brooklyn and then, what it’s like in Chattanooga. What’s the relative salary? What are the relative costs of this stuff?
There is a lot of playing with things. “What if I do this? What if I do that?” It’s very much tweaking causal stuff. That calculator, I would think, is a simple form of this.
Then, the math exercises, there’s a program that one of my kids is loving called Kerbal Space Program where you have to go out and build a space orbiter and then, figure out if it can complete the mission. Can it stop an asteroid from hitting the earth?
There’s all these causal relationships between the thrust of the engine that you put into the space orbiter and whether it can maneuver finely enough to be able to catch the asteroid.
Stephen: That is a perfect example. In fact, a lot of the examples I came up with are education-related where you learn through those interactions in that constant frequent feedback loop.
Jared: Don’t tell my kid this is education-related. He’s still thinking it’s a game.
Stephen: I would put Minecraft in the same category.
Jared: Absolutely, don’t tell them. They’ll freak out. The fastest way to learn to hate a classic book is to get it assigned in school.
Stephen: Exactly. [laughs] You had mentioned calculators. I would consider, for example, a mortgage calculator to be a crude form of this where you go in and there’s a set of five or six variables, or numbers, testing for everything from the cost of the house to how much you’re going to put down to the APR you think you’ll get.
It’s through playing with that that you actually get a sense of what you can actually afford. What will your monthly payment be?
Or, you could start backwards. “Here’s the monthly payment I can make. How much house does that afford me?” Even though, it’s very crude and small, that’s a perfect example of this same type of causal relationship, where through the interaction and through the play, you see the patterns.
You, basically, get the understanding that you don’t have if you have to enter the information, hit “Submit” wait a minute, or seconds, and get that back. It’s the difference between hitting the “Submit” button a hundred times versus dragging a slider for a second.
Jared: I remember one of the first programming projects I ever had, back when I was working for Digital Equipment Corporation, was doing product configurators for sales people for mini computers. These were machines that had thousands of options, but not all the options worked together.
People wanted to know what it was going to cost them, so figuring out, “If I choose this option, then I can’t have that option. But there’s this other option, and, oh, the price goes up $50. But if I do this other thing, it goes up $100,” that sort of stuff.
Stephen: That would fit, as well. Going back to your question to kick things off, you said, “Where does data viz fit in?”
It would fit in really well into the fourth type of problem of understanding that I’ve labeled, and that would be the “problems of discovery.” Often with data vis and Pure Data vis, we’re talking about cases where people don’t know what question they want to be answered.
This is very different. In the workshop, I say, “What’s the question you want to have answered?”
In the previous three examples, we can identify a question to be answered. “We want kids story about math,” or, “We want to see what I can afford. Which medicine bottle is mine?”
With problems of discovery, you have some ideas or hunches that you really don’t know what you’re going to learn. It really is data mining. It’s really about looking for insights and patterns that you weren’t anticipating.
That’s why I call this “problems of discovery.” Most of the examples are data vis. Although, there are some examples that we would not consider data vis, but definitely problems of discovery.
If we go back to — What is it? — 1850, maybe it’s ’54, I believe, John Snow created the cholera map, which was basically, by today’s standards, we’d call it, information visualization. But he basically mapped all the deaths due to the cholera outbreak and mapped where they took place.
He saw this pattern, when he put them on the map that, “Wow, a lot of deaths seem to correlate with this particular well.” That well, eventually, was covered up, but he was able to see this discovery.
He didn’t know what to look for. This preceded germ theory by about six or seven years, as well.
Jared: Yeah, it did. It was one of the things that really set that off.
Stephen: That’s a problem of discovery. The problems of discovery we have nowadays are much more complex and require — not just visual representations, but interactions.
An example, I mention in my book, is DNA and genetics, where we’re talking how [laughs] hot our genetic code is. Scientists are working away, looking for patterns.
The tools they use, and the tools that we, as engineers and designers, provide them, or create with them in collaboration, are going to make a huge difference in whether these geneticists are able to find the patterns they’re looking for or not. There’s an example of a product called Pathline where it was custom designed working with these scientists.
It was so much better than the office shelf’s charting tools they had been using previously. In fact, there’s a story, when I was researching this, where when they first shipped out Pathline, they were watching the scientists use it.
They jumped up and got excited, because they were seeing patterns they had never seen before, because of this new tool, this new visualization and the new interaction possibilities. Again, there’s a huge roll in discovery that product teams can play.
The more we know about these skills of visual representation, these skills of interaction, and the data behind it, the better prepared we are for this uncertain future [laughs] when we have more of these discovery problems.
Jared: I would think that a big piece of this is what’s behind what people are doing with analytics. Even the tools we create for ourselves where we’re trying to create analytics tools to understand what’s happening in our own designs, the more toolkit capabilities we have.
I don’t know if you’ve noticed, we came out with this service, we call, All You Can Learn. It’s, basically, a video watching service. All the seminars and videos we’ve ever made live in this thing.
We’ve been trying to understand, internally, at UIE, how are people using this? When people come in and they watch videos, what happens? What lead up to that point? What happens after that point?
Do they watch a lot of videos in a burst? Are they binge watching? We’re trying to understand these things.
We’ve found that the analytics tools that are out there, things like Google Analytics, really don’t help us with that. Being able to do things like slice off really active users, and see if their behaviors are different than the average user, is really hard to do.
We’ve had to come up with our own visualization tools. We’re really struggling with this. It sounds like this would help us a lot coming up and being able to start pulling apart these basic things where we don’t actually know what question we’re trying to answer.
We have some very vague ones like, “If two percent of our audience are our most active user base, how do we get the next two percent to be more like them?”
Stephen: This would definitely fit in there. Analytics are curious. You can approach them as problems of discovery where you’re looking for patterns that maybe you don’t know. You can approach them as problems of comprehension, relationships, comparison. They can fit under a lot of these depending on whether you have a question to be answered or not. That’s the dividing line there.
You see some packages that are designed to answer specific questions but maybe they’re not the questions you want answered. Then you find yourself hacking it to answer the questions you’re looking for. Or they’re complete abstract, off the shelf data tools that don’t really answer the question specifically so you have to figure out how to roll your own.
Analytics in particular are really a curious entity. Rarely have I seen, out of the box, an analytics package that answered everything that the specific client app I was working on needed to know.
Jared: I rarely see an out of the box analytics app that actually doesn’t leave more questions than answers.
Stephen: [laughs] Yeah, or leads you to the wrong conclusions because there’s missing information.
Jared: It just implies things. My favorite are sort of eye tracker heat maps that imply what users are looking at when, in fact, the eye tracker doesn’t measure what users are looking at or what they see. It measures where their eyes point, which, it turns out, doesn’t take in account peripheral vision. It doesn’t take into account a lot of factors.
It doesn’t take into account that I can stare at the shelf in my refrigerator for five minutes and still not find the ketchup that’s sitting on that shelf.
Stephen: Yeah. [laughs] There’s a problem right there.
Jared: It’s what my wife used to refer to as male refrigerator blindness.
Stephen: I have children who can help me find my keys when they’re gone. I have one particular child who has a knack for that. “I cannot find my keys. Can you find them for me please?”
Jared: This child will be very useful as the years go on. Do not get rid of this one. This is a keeper.
Stephen: These first four problems were pretty easy to identify. Then it was over a period of about a week, it happened pretty quickly, that the final three rolled in. It was just really looking at some of the examples again and some of the things that I hadn’t accounted for.
The fifth one is something we’re all familiar with. I would call it problems of relationships. This is where there’s really not things to be compared. There’s not a lot of data, necessarily. Any time we go into a new space where there’s a lot of learning going on, there’s that, I think psychologists call it the initial schema formation, where you go in and are just trying to get oriented to a space.
If mobile app design is new to you, the first time you go in there you’re just trying to separate what’s the difference between this phrase “progressive enhancement” and “responsive Web.” What are the mobile devices? A quite common exercise here is to create a mind map with circles and lines.
What you’re trying to do there is trying to orient yourself to the space. You’re trying to see the relationships between the different topics so that you can then drill down and actually go deeper into a specific topic.
Jared: These aren’t necessarily causal relationships, right?
Stephen: Just flat out relationships. How do things relate to another? One is subordinate. One is a peer of. One is a parent of, these types of things. It’s just orienting yourself to the space. Concept models, mind maps, are a perfect response to this problem of relationships.
In many ways you can say a sight map attempts to solve that problem of relationships. How will these pages relate to each other? How will people navigate? That would be the problem of relationship. Not a lot to talk about there other than to acknowledge that, yes, that’s a type of understanding problem that needs to be accounted for if this catalog I’m coming up with, is going to be truly comprehensive.
The other one that, again, has not a lot to talk about here because there’s not a lot you can do, I would call the problem of practice. I’ll give a very personal example here. I, recently in the last year and a half, have gotten very into espresso drinks. I went to Italy and had my first taste of a good shot of espresso. I have come back and tried to learn all I can about this world.
Jared: Wow, I’ve never been able to like espresso. To me it’s just burnt beans.
Stephen: That’s what I thought until I went to Italy. I had my conversion experience there. I was in Rome I have a text where I texted my wife and said, “This is nothing like the black bitter swill we’ve been drinking at home.” [laughs]
There’s a lot of head knowledge, problems of comparison, for example, that you can read about or try to learn and you can understand. But ultimately when it comes to pulling a really good shot or doing the latte and all that, it’s what I would call a problem of practice.
Really you’ve just got to sit down and practice if you want to get better at it. The understanding comes through the practice itself. Even in the past I’ve used the example of — was it chicken sexing? That would be a perfect example of this problem of practice.
Jared: Where identifying baby chicks before they’re physically identifiable is an important thing as to whether they’re male or female. There are people who just learn to do it. The way they learn to do it is they just practice and practice and practice next to someone who is experienced.
Every time they get it wrong the experienced person basically tells them they got it wrong. Over time they get it right. If they just put in enough practice, they can learn to do it.
Stephen: Yes. In fact, that example may make its way into my book, because it’s so perfect. It’s something you can’t be taught head knowledge. You just have to experience it, learn it, and develop the skill.
Jared: It sounds like a very Gladwellian 10,000 hours sort of thing.
Stephen: Yeah, exactly. Understanding, in this case, comes through practice, and you can’t really explain it, draw a diagram, or let someone play with a simulation. They just have to jump in and do it.
Jared: Surgeons practice their sutures. They just do that over and over and over again until the motor skills just happen. Baseball players learn to hit 95 mile per hour fastballs. It’s basically just getting in there and doing it over and over. We all do that. We just get better the more we do it. We’ve had these conversations. “How did you do that?” “I’ve just practiced a lot.”
Stephen: That’s exactly it, more perfect examples. The final type of understanding problem, I saved it for the last just because it’s the hardest to talk about. On the model where I put this all together it’s the one that’s the farthest to the right, the most complex. It’s what I’m calling the “problem of distributed cognition.”
A fancy word, but, in this case, basically what I’m saying is the knowledge you need for understanding is distributed across people, objects, environments, things like that. This obviously ties back into service design when we’re trying to connect all the pieces. Even if it’s not just service design, getting different devices to talk to each other, this one shows up in lots of different ways. I’ll give two examples.
One would be the original project to map the human genome. It was a project that took, I believe, nearly 15 years. It originally initiated in 1987. If you read about the history of mapping the human genome and this project, it was scientists working around the globe to try to map the 3.3 billion base pairs or something.
Definitely there was a lot of knowledge they had to get on a similar framework or similar language so they could share their data and work together. Definitely it wasn’t something that could be solved with one person using a better tool in the laboratory. The knowledge had to be distributed.
It’s quite an interesting story if you get a chance to read about it. There was some competition there with a private company that was trying to be the first to map the human genome so they could patent it. Then any advances that came out, we would have to pay for. [laughs] To keep it, in a sense, open source and freely available to the public there was a time frame they were racing against.
Jared: Right, not assign a huge price tag. This is the old argument of we’re investing a lot of money to figure this out. We should be able to recoup our investment through licensing fees and competitive advantage versus the “Hey, humanity needs this. We need to make this cheap and easy so that so many interesting things can be built on top of it.”
Stephen: Exactly. Don’t get me started on the ideology and political discussions there, because I definitely have strong opinions on that stuff.
Jared: [laughs] We’ll have another podcast.
Stephen: We’ll have another podcast on that. Another example, to bring it back down to Earth and a little more practical, let’s take a service like Uber.
Jared: Uber is the car summoning service that sort of competes with taxis in big cities.
Stephen: Exactly, I like car summoning. The experience is you open up the app and you can actually see cars nearby that are available and ready to pick you up. You can determine if they are close enough that it’s worth doing this. You push the button, and it requests the driver.
The driver comes. You get in the car and tell them where you’re going to go, and they drop you off. Even payment is handled passively, then, because you’ve given Uber your credit card information. When you get there, you just get out of the car.
All in all it’s a pretty good experience, especially compared to the traditional ways that we hail a taxicab. However, as good it is, it’s still broken at the seams. When I was using this at a recent conference with Karl Fast, who I mentioned it earlier, Karl Fast had never used it before.
Having that uninitiated perspective, he was actually able to articulate some of the seams where the experience was broken where cognition was in my head but not in the driver’s head, for example. There were at least two cases here where the understanding hadn’t been addressed.
One experience is, as the car is approaching, you don’t know which car is the car you’ve requested. They give you a picture of the driver. If the car is 30 feet away, you really can’t see the driver’s face. They give you the make and model of the car and the license plate number. There’s this stance that you have where you have the phone in your hand, which is kind of a signal to people that, hey, I’m looking for the specific car that is the Uber driver.
Jared: Yeah, and Uber drivers learn this signal. They can pick you off of the sidewalk because you’ve got the Uber stance.
Stephen: Yes, exactly, the Uber stance. You’re looking at license plates, so you’re craning your head looking. There’s a case where there’s distributed cognition. The person knows they’re the car. You don’t know they’re the car. You’re looking among these options. How could you fix that broken seam, if you will?
Jared: With Uber X, which is it’s not always a black sedan but it’s their personal car. They don’t even tell you what color. I wish I had a picture of the car, because it just says “Prius.” When you’re in San Francisco, that’s every third car.
Stephen: Oh gosh, yeah, every other car.
Jared: But it doesn’t tell you what color Prius, even.
Stephen: It reminds me of Lyft, which is a competitor of Uber. They’re known for the big pink mustaches that the drivers put on the front of their cars.
Jared: Right, and they just launched a line of white sedans and SUVs to compete against Uber’s black sedans. They don’t put the big pink mustache on front, but they have this mustache decal that goes on the grill that makes it very clear that this is a Lyft vehicle. But it’s more classy. It fits in with the design.
Stephen: The second area where the seam is broken in this experience is one that I’d actually love to see addressed and we have the technology to do so now. That is the moment you get into the car you have in your head an understanding of where you want to go. It may be an address. It may be just an area like, “Down by the harbor” or “downtown.”
You’ve got to get that knowledge into the head or the device of the person who’s going to drive you there. I can’t tell you how many experiences I’ve had where basically I handed my phone to the person or they gave me their phone and I typed in the address and handed it back.
When you’re talking about this great experience, this great service design, that’s an area where it’s actually breaking down. What I’m looking forward to are places where I can get into the car. I can have the address on my phone, for example, and I can just push it off of my glass and have their phone accept it, the information.
The technology to do that is out there. Scrabble is an example. If you look at Scrabble for iOS, you can put an iPad as your Scrabble board in the middle, and different people can use their iOS dividers to actually be their tile holders so you still have that privacy.
The technology is here. It just hasn’t been realized yet. There’s another project that Josh Clark just shared with me called Magic Grab or Grab Magic.
Jared: Right, I’ve seen this. It’s really cool.
Stephen: The guy just reaches out. He’s watching TV. He grabs. He makes a gesture with his hand, and he grabs. At that moment he’s grabbing whatever is on the screen at that second. Then he taps on his phone, and that image appears as a screenshot on his phone. You can grab whatever you see off the TV onto your phone.
Jared: It’s using a Kinect like device. I think it actually uses a Kinect to capture the gesture for the grab off the TV. Then it uses the motion sensors on the phone to detect and put this on my phone.
Stephen: Exactly. That was actually done in 2012. That was done as an evening hack-a-thon project. If that can be done with technology two years ago in an evening, what can some of these companies working on the whole TV, media, second screen experience actually do?
How can we get our devices to talk to each other in a way that’s seamless? This is a huge area where the knowledge can be passed not only between people and people and their device singular but between devices plural. It’s an exciting area to look forward to.
Jared: These seven things, these seven problems, as they were, that you can now come up with solutions for, having this taxonomy lets us understand how to prioritize our designs. I think this is brilliant.
A lot of what we struggle with in the design world is not having words to explain how one thing is different from another. Then we get all muddled and confused. I think you’re working on a comparison problem, but you think you’re working on a discovery problem. Because we don’t have words to explain this, our designs just get crazy, wacky, and we’re not working together.
I see this as a tool that can really help teams ask themselves, “In this release, what would we like to do? In our five year vision, what would we like to do” and actually break things down and have different answers for that but all be on the same page all the time.
Stephen: The better the language is for having this conversation, the most fruitful or constructive it’s going to be. It’s not just knowing the language and being able to identify it. Once you can do that, then you can decide what skills would be best to respond to this.
As a simple example, if it’s redesigning a medical form, you don’t have to get into a lot of heavy-handed interactions or coordination. It’s just a simple chart with five pieces of information, seven pieces of information. A simple visual makeover will do in most of those cases.
If you move into something like the problem of discovery, at a minimum you definitely have to think about the visual representation. You have to think about interactions. The data is so complex that you have to be able to manipulate and work with it. Then you have this whole language around interactions and what the types of interactions are and what they do cognitively.
In a case like the genome, for example, you probably have to think beyond just the tool and think about coordination across systems, across people, all of that. Now we can start to talk about representations as a response, interactions as a response.
Jared: Yeah, there’s a service design element in there.
Stephen: Yep, through the lens of saying, “This is the problem I’m trying to solve, the understanding problem I’m trying to solve,” we then are oriented to the skills that we need to respond to that problem. I’ve been describing it like this.
A lot of workshops and courses on things like visualization or information design tend to approach it like a recipe. We’re going to give you the recipe, and now you’ll know how to respond to this specific challenge. Of course you go back and the very next challenge you get doesn’t fit. It’s not the same pattern.
What I’m likening the approach I’m taking is, instead of giving you a recipe where now you can make this one particular thing, I’m trying to provide skills that can help you in the kitchen. We’re going to go over knife skills.
We’re going to go over how to pair ingredients. We’re going to go over the role of aromatics. To extend this cooking analogy, we’re going to give you the skills that then you can apply to make whatever recipes you want to make. You’ll understand the fundamentals.
Jared: This is moving past rule-based understanding to really being informed. Then you’re going to need some sort of improvisational layer on top of that to be able to say, “Hey, what if we tried this?” and everybody just knows what that means from their improvisational practice.
They do a “yes and” and they move it forward. You keep trying things until you get a solution that you’re really thrilled with.
Stephen: Yep, that’s the improvisation layer added to it, exactly. You know me, Jared. Everything I try to do is very much on the practical, like giving people skills they can walk away and use.
Jared: That’s one of the things I hate about you the most.
Stephen: [laughs] You hate?
Jared: It’s just like, “Oh, that Stephen Anderson, he’s so damn practical.”
Stephen: Yeah, there’s a degree of specificity that you can give someone that you’ve over specified and it’s really not a skill. Here’s a way to respond to this specific problem if you ever have it. I’m always looking for those skills that are timeless and will apply in most cases.
That’s been the hardest part in writing the book, really keeping my eye on the timeless skills that we need to learn around understanding and not get sucked into these surface or superficial things that are useful in their context but don’t scale across the whole variety of contexts that we have every day.
Jared: I’ve got to tell you, I’m really excited about this. I’m excited about the book, and I’m very excited about the workshop that you’re going to be doing at UI19. People are not only going to get a chance to learn this framework but also practice it and make those decisions and start down their path of mastering how to use this to make really substantially better designs than they’ve ever done before.
Stephen: That’s the plan hopefully. [laughs]
Jared: It’s great. It was highly rated last year at UI18. I heard nothing but amazing things from the people who saw you and Carl at the IA summit. I’m just so excited that you’re coming back to do this for us at UI19.
Stephen: It’s an honor to be invited back. Thank you.
Jared: It’s great. Stephen, thank you so much for sharing this framework with us today and talking about the seven ways to tackle this stuff. It’s really such fascinating stuff. We could do this for hours. You’re going to do it for hours at UI19, so that’s good. I can’t wait to be there.
Stephen: I can’t either. I tell you, every time this is reinvented I get rejuvenated and excited. I’m excited.
Jared: Excellent. You sound reinvigorated, so that’s good. We needed to get more vigor into you.
Stephen: [laughs] You don’t want a boring workshop leader.
Jared: Espresso isn’t the only way you become invigorated. [laughs] OK, Stephen, thank you for taking the time to talk to us today.
Stephen: Thank you, Jared.
Jared: I want to thank our audience for taking the time to listen to us today. I hope you’ll consider coming and seeing Stephen and the other great speakers we have at the User Interface conference. You can find out more about it at uiconf.com.
It will be happening in Boston October 27 through 29. That’s all I’ve got to talk to you about today. I’m glad Stephen came and glad you were listening. Thank you so much for encouraging our behavior. We’ll talk to you next time. Take care.