Why people don't read UX research reports
October 16, 2017
Are you a UX researcher, working in a product development organization with engineering, product, and marketing teams? Is it your job to design delightful experiences for the users of your product? Then, the product owner will often ask you for a complete research report, with comprehensive insights on the humans who use your product, together with method descriptions, detailed data and proof, right?
Wrong. Nobody really wants to see such a report more than once.
When you work in a UX design agency and your customers want you to create such a report, and it's the first time they see such a thing — then yes, O.K., go and write it.
However, if you work in an internal UX team for your company only, then the situation it is different:
Internal UX research is perceived differently
Your product owner and colleagues in the agile teams will want to use your research results to decide what to work on first, i.e. to prioritize the features in the backlog. They need a deep human understanding for your product's users, what makes them tick and what turns them away.
After some iterations, your colleagues will already know the methods that you are using in UX, so they will not need an explanation of those methods in every report.
In fact, internal teams are not interested in complete UX research reports at all. They want to know about the observations that you made, and about the trends and recurring themes that you have seen when making sense of the data.
What they don't want to see is a formal paper as described in the international standard, ISO/IEC 25062:2006 with an 80-item checklist for what should be included. Those of you who are brave and daring, click on the image to see the full 80-item checklist. Or maybe you want to see the original, scaring template?
Reports are a problem in themselves
It can really be frustrating: You write a nice report and nobody really reads it. And if they read it, they forget about it or ignore it. Why is that?
A report is hard to read
One reason for this is that a long, fluffy report not only contains a lot of data and insights but also a lot of metadata: Time, date and location of the research, the participants, a physical description of the test facility, the screen size and the input devices that were being used, the methods and statistical procedures by which the data collected were analyzed and scored, and so on.
Such a report will be a good candidate for an artifact to be exposed at the Smithsonian, not for being used in the daily practice of your product teams.
A report may conceal the interesting details
To be understandable, a report must suppress details to a certain extent. In small teams, UX research is often assigned to a single person on a team. That person becomes the spokesperson for user needs, the team's expert on users. This approach is a poor way to do research because it encourages the team to delegate all responsibility for understanding users to one person. When this person pours all her knowledge into a report, important things are lost. When the team tries to rebuild that knowledge by reading the report, something is lost, too.
Reports can be forgotten
Organizations have bad research memory. Before you embark on a new user study, you might send an email to the other teams. You may ask: "Did anyone do research on this, before?" And you'll get answers like: "Oh yes, we asked users about this already a year ago. The info is inside that old report – I don't exactly remember which one". And before you dig into the previous reports, you'll prefer to do the research again.
Reports are not distributed widely enough
The marketing people do research, too. Do you talk to them about it? And the other way round: Do the marketing people know about the research you did in UX?
As soon as a company exceeds 100 people in size, communication becomes difficult. Information silos tend to emerge. Each department has their own way to document research results, there is no central place to put them. Consequently, a product owner and her engineering team might miss important input for their product roadmap, only because the research results are buried inside a report made by the marketing department.
Reports are too big
As Tomer Sharon always says: "Reports are not atomic". They are big. Sometimes, a report might include a piece of information that has nothing to do with the reason for running the research study. And yet, it might serve the people in the future. Do you include such information or do you leave it out?
When a team inside your company seeks insights about their users, their work would be much more efficient if they find small pieces of insight instead of long reports.
Small pieces of research results (e.g. one observation at a time) can be more easily understood. After reading several such small observations, a product owner can see that a certain theme is forming around the users' problems. She can make up her mind easier and faster on how to add this insight as a new story for the team's backlog.
OK, so what should you do?
You heard me talk about your frustration: You write a nice report and nobody really reads it. And if they read it, they forget about it or ignore it.
And you heard me talk about why all this is like it is.
That's O.K. but now what?
I won't let you "stand in the rain". Of course, I've got some advice for you. When you follow it, your situation will improve. Take these steps to get out of the dilemma:
Make user research everyone's business
Why should you be the only one in your team who has direct contact with the users? Encourage everyone on the team to get their "exposure hours". Jared Spool's research shows that the most effective design teams spend at least two hours every six weeks observing users (for example, in field visits or usability tests). I would recommend even more: Have direct communication with users in every iteration or "sprint".
Deliver UX research results in small pieces
Stop writing long reports. Keep every observation "atomic", searchable by project, study, case, and the researcher who created it. Use a series of taxonomies, e.g. demographic, location, company size, experience (positive, neutral, negative), frequency (rarely, regularly, often), magnitude (weak, medium, strong). Make it easy for your colleagues to find observations by such tags.
Keep all observations in a central place
Why should every team or department have their own place to store research results, unknown to the others? Use one reliable central place, a system like meaningmaker.ai where every team can add and curate observations and themes. That way, you avoid information silos and accelerate the spreading of user insights in your company.
Link observations back to evidence
For every insight that you add to the system, add a link that points back to an original piece of evidence. That can be a photo, a video, a screenshot, a customer's email message, whatever you have seen. This way, other people will be able to understand your observations and empathize with the users whom you have seen.
Allow people to curate observations
Sometimes, several observations made by different people suddenly make sense. They "fall into place". Someone sees a theme or a pattern emerging from them, even if she did not make all those observations by herself. Such a theme can be extremely valuable for other people in your company.
Choose a system like meaningmaker.ai that allows you to create themes by curating observations. Don't put observations into long reports anymore. Make themes a first-class citizen.
Collect themes semi-automatically
Let a computer help you see the themes. Put your observations into meaningmaker.ai which allows you to tag observations quickly and cluster them by sets of chosen tags. That way, you can identify and collect themes fast and easily.
Make the pieces accessible to everyone
There is no reason why observations and themes should be kept private. Share them with all the interested teams in your company. A team looking for a feature that they can add to their product should always have a clear reason to do so. Each time they think about what to do next, they will find the answer easily: In the results of your research, of course.
Get this advice as a checklist
I have boiled this down to a simple checklist for you that you can download and use during every user study. It will remind you of the steps above so that you won't forget. Using it, you'll leave the frustration behind because now everyone will find your observations and insights. They will see how useful they are, in order to derive actionable product design decisions.
Download the checklist, use it and pat yourself on the back, watching more and more people use your research!