Meaning Maker


How to make sense of your UX research results

September 01, 2017

Making sense of UX research results

You have observed five or six users as those individuals conduct their work. You’ve asked each user some questions about their jobs and recorded details of their responses. You may have taken screen shots or photographs of the tools being used, etc.

Now you return to the design studio. In the privacy of your natural work place, you will attempt to make sense of what you have learned. This is hard but you are not alone: Researchers and designers continually describe this sensemaking process as a way of organizing complexity or finding clarity in chaos.

The Problem after UX Research

The user research sessions have produced pages of verbal transcript, hundreds of pictures, several hours of video footage, and dozens of artifact examples.

As a UX researcher, you will have thought: How on earth shall I make sense of this mess?

What do you do to find relationships or themes in the research data? How will you uncover hidden meaning in the behavior that you have observed and that is applicable to the design task at hand?

When you return from research and unpack your bag, the large number of things that you collected during research might seem like a mess. Abby Covert, the information architect, said:

The first step to taming any mess is to shine a light on it so you can outline its edges and depths. Once you brighten up your workspace, you can guide yourself through the complex journey of making sense of the mess.

Let’s see what we can do to turn on that light.

How to make sense of it all

Three steps to make sense of your #UXresearch resultsTweet this

Step 1: Collect observations

All research questions, methodologies, and conceptual frameworks are context-specific. But I think what every researcher brings home are observations.

An observation in the context of user research is the result of looking at what users do, where, when and how they do it. You look at someone, you see something and you can write down what you have seen—as an observation.

An observation is most often written down as a few sentences of text, backed by evidence in the form of data.

The data can be interview transcripts, participant observation field notes, journals, documents, literature, artifacts, photographs, video, websites, e-mail correspondence, and so on. The observation boils down the data to a pithy paragraph of text that can be quickly understood by a reader who knows the context.

So this is what you’ll do as a first step to shine some light on your research: Collect your data and write down observations that can be backed by that data.

Look at an example for such an observation from the registration process of a web-based system:

She started to fill out the user registration form and tried to submit it. The system responded: “Your password is not strong enough. It needs to be more than 8 characters long, with letters, digits and special characters.” She asked herself, frustrated: “How can I possibly remember such a password?”.

Step 2: Coding

No, this is not what you possibly think. “Coding” in this context does not mean “writing software”. It means assigning a code to a datum or an observation in order to bring some structure into the vast amount of them.

Johnny Saldaña, professor emeritus of theatre from Arizona State University, explains in his book The Coding Manual for Qualitative Researchers:

A code in qualitative inquiry is most often a word or short phrase that symbolically assigns a summative, salient, essence-capturing, and/or evocative attribute for a portion of language-based or visual data.

Here are some examples from the same book:

I notice that the grand majority of homes have chain link fences in front of them. There are many dogs (mostly German shepherds) with signs on fences that say “Beware of the Dog.” (code: SECURITY)

Or, from a different context where a mother describes her teenage son’s troubled school years:

My son, Barry, went through a really tough time about, probably started the end of fifth grade and went into sixth grade. (code: MIDDLE-SCHOOL HELL)

When he was growing up young in school he was a people-pleaser and his teachers loved him to death. (code: TEACHER'S PET)

Two boys in particular that he chose to try to emulate, wouldn’t, were not very good for him. (code: BAD INFLUENCES)

You’re getting the gist: The code summarizes the observation and captures the essence of it in a salient way.

Step 3: Find recurring themes

Once you have collected your observations and coded them, you will find that the observations tend to “cluster” around certain codes. As an example, let’s reconsider the above sample observation of a user signing up for a new system:

She started to fill out the user registration form and tried to submit it. The system responded: “Your password is not strong enough. It needs to be more than 8 characters long, with letters, digits and special characters.” She asked herself, frustrated: “How can I possibly remember such a password?”.

You can assign two codes here: registration and frustrated, one for the step in the process and the other one for the emotion that the user feels during that process step, respectively.

Chances are that you will have made many more observations about the registration process. They will all have registration as one code and possibly other codes like frustrated or satisfied.

If the combination of registration and frustrated occurs multiple times you realize that the software has a usability problem. In that case you have identified a recurring theme.

A theme can be described by a set of codes or a set of combinations of codes. Example: A theme like “Frustrated job seeker signing up for a job listing site” could also be expressed as the combination of the codes registration and frustrated and jobseeker.

Why would you use a combination of codes in addition to narrative prose in order to describe a recurring theme?

Well, computers can handle codes very well, and that’s why you code: to enable a machine to help you with sense-making by automatically finding such recurring combinations of codes.

A piece of cake, huh? Let me show you.

Let a machine help you find patterns and themes in your #UXresearch resultsTweet this

Let a machine help you make sense

Collect and tag observations

On the Just Ask Users platform, you will find the Meaning Maker product. This is the sense-making machine that I mentioned before.

With Meaning Maker, you can collect observations as text and write the codes with a simple syntax, just like a #hashtag on Twitter or Facebook:

screenshot of a tagged observation

The system will recognize each #hashtag and convert it into a tag that is attached to the observation. As you see, the list of attached tags is at the bottom of each observation.

On the right-hand side of the screen, you will see all tags that you used in the same project, regardless of the user study in which the observations were made. This gives you a nice overview of the topics emerging in that project, including those collected by your teammates.

Define your taxonomy early

It helps if you decide on a taxonomy of tags early in the process so that you tag your observations systematically. Here are some examples as an inspiration for you about what kinds of tags you can use:

Kind of tag Possible values
Media type #Audio, #Photo, #Video, #Document
Research method #UserTest, #Interview, #Survey
Experience Vector #Negative, #Neutral, #Positive
Magnitude #Weak, #Medium, #Strong
Frequency #Rarely, #Occasionally, #Frequently
Emotions #Embarrassment, #Amusement, #Annoyance
Phase in Customer Journey #Search, #Discover, #Buy

Feel free to create your own categories of tags.

Tags are nice. But: Now what?

Build themes

Once you’ve got the tags, you can find recurring patterns and derive themes from them. Themes will help you in discussions with your stakeholders and teammates to make sense of the possibly large number of observations. When a negative user experience occurs frequently enough, make it a theme and decide to change your design accordingly.

In Meaning Maker you will find the Theme Builder that comes in handy here. It shows you a list of your tags, together with the number of observations tagged by each one.

the list in Theme Builder helps you make sense

When you click on a green number, Theme Builder creates a filter that shows only those observations which are tagged by that tag. When you click an orange number, the tag is added to the same filter so that it will show only those which are tagged with both such tags.

Selecting an additional green tag will add a new filter. Now, observations found by both filters will be shown.

Selecting an additional orange tag will make a filter stronger, it will show fewer observations.

You can build any combination of tag filters. The theme on the left-hand side always shows the current tag filters and how they are combined with brackets and with “+” signs. Example:

[ Search Negative ] + [ Discovery Negative ]

This will find all observations that are tagged with both Search and Negative or with both Discovery and Negative.

Here are some examples how you could use this feature to find observations…

  • …where VIP users of your system had a negative experience, frequently
  • …about successfully completed registrations or successful password changes
  • …that have audio files made during interviews
  • …where the user felt embarrassed during search or discovery phases of her customer journey

The only limit for tag filters is your imagination.

Share themes and have insights

Now that you’ve seen how Just Ask Users can help you with analysis and synthesis, it’s your turn:

  • Start to build some themes and save them in Just Ask Users.
  • Send your teammates a link to a theme and ask them what they think about it.
  • Get actionable insights when the team sees the recurring theme and discusses how to help the user in the observed situation.

Get started with sense-making

You’ll be surprised at how easy this becomes with a little practice.

After a while, you will be able to do this live, directly during your next user study, for example, in the room next door to your user testing lab where the team is watching a running user test.

P.S. You can get started with Meaning Maker for free today. Sign up here to get your account:

Easy setup • Free trial with one multi-study project

About us

Our goal is to help companies improve their products.

We give you a tool that shortens the way from research to design decision: Collect evidence about your users’ behavior, make observations about it, mark the observations with tags and let the machine help your team discover recurring patterns. Grasp insights easily because you clearly see what bothers your users and what delights them.

We make it simple for product developers to get to testable new ideas, concepts, and products for their target market so that they can avoid dead ends and keep building products that their customers want.

Our tool helps to make UX findings transparent. The clarity, transparency, and focus allow teams to collaborate with less friction and produce great results.

© Copyright 2016-2018 Matthias Bohlen - All rights reserved.
Terms & Conditions – Privacy Policy

Get in touch

Phone: +49 170 772 8545

E-Mail: support@justaskusers.com

Mailing address:
Matthias Bohlen
Luise-Albertz-Str. 25
53340 Meckenheim