Getting started with a research repository
March 07, 2019
What is a research repository?
A research repository (sometimes called a research library, too), is both a place and a tool.
It’s a place where you and your team can store important research data like user interviews, user test results, findings from your competitor analysis, and more. The data that you put there can typically be text, images, audio or video. The repository (short form: “repo”) stores all this data for you in a central place and allows you to find it when you need it. That’s the place aspect of a repo. It is similar to what a library does.
A repo is also a tool that comes with features that support you in your efforts to collect, organise and analyse your data, to discover what all the findings mean. You want to understand what a participant said in an interview or what happened in a user test. Once understood, you and your team are able to decide what you can do next to improve your product or service.
Why would you want a research repo?
In the first place, you want a repo as a safe place to keep your research data.
Next, you will want to have access to your data whenever you need it. This includes search or a good structure that helps you find the information you need.
Then, you will want to make meaning of what is inside the repo. What is it that makes all that qualitative data (e.g. user interviews, customer feedback, etc.) relevant to the people in your organisation? What is it that you and your stakeholders should care really about?
Relevance and good communication of the results will then lead to action: Your teams will want to make decisions about what they should create or what they should change in your product or service, in order to make it better for your users.
If you compare the things that a repo does to the steps in the design thinking process (empathise, define, ideate, prototype, test), a repo fits nicely within the steps “empathise” and “define”. When you understand what all your data means, you can literally see and feel your users’ pains and define the exact problem that your product or service is going to solve for them. However, before you’re going to understand, you have to put the data somewhere. This is where everything begins.
The journey towards maturity
If you watch teams who become more experienced and mature with the use of a repo, almost all of them have touched one or more of these maturity levels:
- In the beginning, there was a mess of mixed research findings.
- Someone decided that there had to be some structure to find things. Different teams established different structures, so meaning making could only be done locally, within one team.
- Research, product, engineering and support teams decide that there must be a way to share insights and meaning across the whole customer journey and/or the entire company.
- At the highest level of maturity, whenever a team asks itself what to do next, they simply look into the research repo to find a customer pain that they want to solve. They go for it without asking for permission.
Let’s look at these levels, one by one.
Level 0: The messy beginning
A research repo solves one frequent problem that many teams have in the beginning: The research they made is “all over the place”.
- Videos have been captured and are still on a chip card inside a digital camera.
- Photos are stored somewhere in a folder on the company’s local network.
- Already transcribed videos of customer interviews are stored as text files, in a folder that belongs to the marketing or support team.
These things are not categorised nor systematically understood but the teams operate in a rather ad-hoc way. Research is made, collectively inspected and decisions are made within one team only, right on the spot.
Level 1: The searchable archive
With a research repo, things are already changing. The teams upload their findings to a cloud-based repo that has a structure, based on the team, the project and the study in which the findings were collected. This is already a huge step forward because everyone now at least knows where to look for the data.
Next, people want to search inside the repo to find things that relate to a certain topic like…
- a certain user pain like password strength at login time
- a certain step in the customer journey like search, discover, evaluate, buy
- a user’s emotion like happiness, curiosity or frustration
You can’t do this using folders on a network device. Searching like this is multi-dimensional. Folders on a disk won’t cut it because a folder structure has only one single hierarchy.
A good research repo will allow people to search for these things, not only using a text based search (looking for certain words) but also a search based on metadata that is attached to the original data.
At this level of maturity, people begin to tag certain portions of the data (usually text) with metadata. Example:
The user says: “I wanted to define my password but the system did not accept it. The system did not tell me what the rules for a good password were, so I gave up and will now look for a different product.”
One person might tag this with tags called
password strength, the other person might tag it with
lost prospect. These tags are typical examples of metadata.
Later, when an engineer wants to work on the login function of the system, she might search for all findings related to
login. And she would find the pain about password strength and unknown password rules.
A person from the customer success team might search for
frustration because she wants to report the latest customer pain points. She will find more, including the login problem.
All that works quite nicely, already.
However, with more success comes more mess. The mess might strike back.
Level 2: The journey towards meaning
After a while, the repo becomes larger if not huge. When searching, people notice that
- they have used different tags that might have the same meaning
- they have used the same tag but with a different meaning
The result will be that they don’t find stuff they were looking for, or they find stuff they were not looking for.
With increasing size of the repo, more agreed-upon structure and a shared understanding of the meaning is required so that the mess does not strike back.
This is the step where people put tags into groups and define rules on those groups or on the individual tags. (That is called a “folksonomy”, i.e. a kind of taxonomy that is created and maintained by “the folks” themselves).
Some teams go even further and define real taxonomies or even controlled vocabularies where a precise meaning is attached to each term and persons are defined who are responsible owners for the terms. (Taxonomies and controlled vocabularies are a subject for another blog post. Let’s continue with this journey, first).
When teams successfully complete this step and reach this level of maturity, they become able to successfully create and manage larger repositories with multiple teams, in multiple departments.
Level 3: Easy communication and decision-making
All the previous activities serve one purpose: People are trying to assign meaning as an outcome, to the output that came from research. Meaning is the answer to these questions (and possibly more):
- Why should we care now?
- Why is this relevant in the future?
- How can this inspire our teams with opportunities for design?
- How does this help us to decide what to do next?
Meaning that is shared and agreed upon by the teams and stakeholders involved, is an excellent precondition for the effective communication of research results. It makes it a lot easier to inform the stakeholders with what happened to the users, the pains they had and the stories from the original context that can be told.
Good research repos like Meaning Maker foster this kind of collaboration and communication.
OK, now you know why you possibly want to have such a research repo. Maybe you have been in one of the situations described above and want to take a step forward.
So, what could be the steps to get started with a repo? Read on.
How to work with a research repo
For the impatient readers, here is the short version of the steps to get started:
- Do it like Marie Kondo and collect a big pile of your existing research results.
- Create a structure in the repo that resembles the original context of your research.
- Upload each piece of your research into that structure.
- Decide about the structure of your meaning-making context or let it emerge from the next steps.
- Explore and interpret your data
- Tag your data with the pre-defined tags from above or with codes that you find during analysis.
- Build a theory of what you think has happened to your users, i.e. meaning and recurring patterns, and links to the “proof”, i.e. the data from the original context.
- Collect the insights and theory you discover and communicate them to the rest of the world.
- Together with the rest of your world, make the decisions about what to do next.
Did you notice that I mentioned two contexts instead of one? Curious what the difference is? Stay tuned and read on.
The steps that I describe now need not happen sequentially. You can work in iterations, you can skip or repeat a step. Consider these steps as a kind of shopping list to choose from.
Collect the data you have
First, find out what it is that you already have. Go treasure hunting:
- interviews with participants, recorded on audio or video
- transcriptions of those
- results from user tests (user tasks and their outcomes)
- photos from your participant’s context
- and more.
Collect all this data and find out where it came from:
- the team who created it
- the project and study in which it was made
- the real source, i.e. the participants
All this describes the first context: the source context, i.e. the one defined by your participants and their way to think and speak. (Later, you will also use your, the researcher’s way to think and speak, but be patient, this is not yet the time for that).
You will likely store your data as research notes or simply called “notes”. A note is a rich-text document that can contain formatted text, images, audio and video.
Import the data
Now import your data into the structure you created:
- The text goes directly into the notes. Copy/paste it there.
- There is an area for images, audios and videos; it is called “uploads”. Upload them and add them to the right places in the notes, later.
- Transcribe audio and video to text (or have them transcribed by a person. In a near release, Meaning Maker will do this for you, automatically!).
- Paste that transcription into a note and add the original, already uploaded audio or video above the text.
- Make sure to anonymise the data – your participants’ privacy is important. Don’t mention their real names or email addresses in the notes because later, you will want to keep the notes but throw away the personally identifiable information like names and email addresses. While interviewing, make sure that you don’t mention the name of your participant while talking to them.
Create the origin-oriented structure, first
In the research repo, create a structure that resembles the source context first (see further above). In Meaning Maker, for example, there are teams, projects, studies, and participants. These three serve as a basic structure in which you can store your data.
To structure your notes (because you will have plenty of them), create “note groups”. Label the groups with appropriate names that still come from the source context so that you don’t lose it. Drag and drop the notes within the note groups of one study so that you get a nice, ordered system inside that one study. The next study will have its own note groups.
Choose: Tags up-front, on the fly, or both?
The next step will be to highlight interesting portions in the text and tag them with tags that have meaning for you as a researcher.
For me, the way a user researcher codes makes a big difference. A user researcher needs to code in order to categorize phenomena. For example, when I code customer journey steps like search, evaluate, buy, pay, deliver, and when I code user emotions like delighted, neutral, frustrated, disappointed, etc. – then afterwards I can ask the research repo: “Show me the original data where a user was frustrated and tell me in which step of the customer journey that occurred”.
Before you do that, take a step back and think about this:
- Do we need to define tags and their meaning up-front, before we start tagging?
- Or is it sufficient to start and discover them and their order later, like you would do it with codes in the Grounded Theory Method?
This largely depends on the research methods that you use, on the number of researchers and departments involved, on the way you want to make sense of your data, and on the size of your repository.
Two extreme examples:
- Some researchers say: We must define the tags up-front, otherwise people will use different, non-uniform tags, and chaos will result. At least, searchability will suffer.
- Other researchers say: No, wait, pre-defined tags will push our thinking into a certain direction! It’s more important to start coding with an open, curious mind, with no pre-conceptions whatsoever. We can create a system of the codes, later.
My personal opinion: For a completely new domain, it’s good to start with option 2:
- Define the tags as descriptive codes for what you see in the data.
- Then, define additional, more analytical codes, i.e. for the story behind what you see in the data
- Last, define theoretical codes, for the larger theory that you have about how come that the story happened as it did.
In addition to these codes, you can later create story-independent tags that you can create for other data.
So, for more data from the same domain, you can use more and more of option 1.
But that’s only my opinion. Feel free to disagree here.
Explore and categorise your data
OK, now delve deep into your notes, highlight interesting portions of the text and associate them with codes or tags. In Meaning Maker, they are always called “tags”, even if you see rather consider them as codes. Never mind.
A note with such highlights and tags looks like this:
When you click on a tag, Meaning Maker will show you all the highlights where that tag has been used. When you click on such a highlight from the list, Meaning Maker will jump exactly to the place in a note where you or your colleagues used that tag.
This gives you a good opportunity to jump back and forth between overview level and detail level.
Write memos about your tags
While you “invent” the tags, give them a color and write a memo about that tag. Each tag has its own details page where you can add a description.
- What were the thoughts you had while you invented the tag?
- What does it mean, as a sort of definition?
Remember the Aristotelian definition of a definition?
Definition = superclass + differentiator
When you write a definition, you define the term by mentioning the superclass plus explaining the thing that differentiates it from its superclass.
Example for the definition of a
A human is a mammal who can utter something that makes sense.
O.K., so, how would describe a tag called
A politician is a human who…
Well, this looks like I digress…
From time to time, when you use the same tag in multiple places, it’s a good idea to come back to the tag’s details page to check the definition. Is your understanding still the same? Does the definition still hold? If not, talk to your colleagues about a possible change, or create a new tag that fits what you really mean.
This process of constant comparison (as the Grounded Theory Method people call it) is especially important if the tag has been created by someone else. If you use another researcher’s tag to tag your own data, make sure that you really agree with the definition that she or he made. Ask for a definition when there is none.
Memo-writing is a good thing. It helps you stay on track.
Discover meaning and patterns
After tagging (or coding) your notes, it is time to take a step back and bring a little order into the tags.
This is the second context that I mentioned above: the research context. It is defined by how you, the researcher, thinks and speaks (not the participant).
Click on “Analyze” and create tag groups to sort your tags.
Unlike the example above, you could also create a group called “user journey steps” for all the tags that sound like steps in the user’s process, like
deliver. Or, you could also create a group called “emotion” with tags like
Lastly, click on “Patterns” to see whether these tags are related in any way. Or export your data into a tabular format so that you can do all kinds of magic on it in MS Excel.
Collect and publish insights
The previous step has possibly made you aware of correlations between the tags you coded. Now, you can look at the original data again, i.e. your user interviews, tests or whatever. This fresh look should enable you to reason about the possible cause of the correlation.
Example: If Meaning Maker has shown you that a user’s frustration mostly happens during the checkout of the shopping cart, you will know that this is only a correlation. You can read, watch or listen again to what the original participants did or said, and you will possibly have a hypothesis about why they were frustrated. This hypothesis might be tested later.
Publish these insights to your team and to your stakeholders, e.g. the product team. They can change the checkout function of the system so that users won’t be frustrated any more.
Where to begin
After all this, you might wonder where to begin. Here are some recommended steps:
Collect your expectations
Get your team together and talk about what everybody expects from a research repository. This is not about the tool but about the outcomes:
- Which research questions do people have?
- Which answers do they expect?
- What kind of granularity do they want to work with? (User quotes, observations, or entire documents, e.g. a user interview transcript?)
- Who will contribute to the repo?
- Who will gain value by consuming it?
- At which steps in the research process will the repo be used?
Create a prototype
Create a prototype of your repo. With the prototype, do a first round of the typical repo activities that I described in the previous chapter:
- Collect the data you have
- Import the data into the repo
- Create an origin-oriented structure
- Explore and categorise your data with tags
- Write memos about your tags
- Discover meaning and patterns (after sorting the tags into groups)
- Collect and publish the insights
Inspect and adapt
After the first round, conduct a retrospective meeting to collect feedback from the researchers about what it was like to use this prototypical research repo.
- How well did data collection work?
- How well could the researchers find what they were looking for?
- How about the usability of the prototypical repo? Do people like to work with it?
- Could you build a folksonomy (or taxonomy) of tags, easily? Does it fit your needs?
- How about back-linking from the metadata to the original material, i.e. can you find the text position in the user interview easily where a particular quote came from?
Then, try Meaning Maker for your second prototype of the repo, and tell us what you think about the difference to the first prototype. Was it better? How does it feel now?
After this, you’ll have a good understanding of how you will want to continue.
P.S. You can get started with Meaning Maker for free today. Sign up here to get your account:
Easy setup • Free trial with one multi-study project