Using generative research to build the right thing

What is generative research?

Generative research is an exploratory technique used to understand users’ experiences, needs and behaviours and generate insights to support design and delivery. The goal is to identify problems to be solved first, rather than jumping straight into solutions and validating the idea with users afterwards, which risks teams building the wrong thing. By gaining an in-depth understanding of users’ motivations and challenges, teams can build empathy with their user base and explore solutions that meet user needs.

Generative discovery can take different forms:

Interviews: At the Department for Health and Social Care we conducted video interviews with overweight people with comorbid health conditions to understand barriers to accessing weight management services. This helped us to understand the needs of a wide range of users and make recommendations for improved signposting into digital services.

Contextual interviews: At the Ministry of Justice, we visited prisons to see how staff accessed and used data, to help understand how to replace a legacy reporting service. This allowed us to see user challenges first hand whilst using existing tools, and understand the environment. Is it noisy, what technology are people using, what workarounds do people use, and how could they be turned into opportunities for a new service?

Observations: At Pret a Manger, we visited coffee shops to observe the flow of customers, and to help understand how new digital services would intersect with in-store processes. If your prospective service has an offline touchpoint or will be used in a specific environment, immersing yourself in the environment and trying out services can help you to design a service that solves a whole problem for users, on and offline. In this example, seeing the flow of customers in store meant that we were able to make recommendations for how collection points could work for a new online ordering and digital messaging service.

Ethnographic research: At Jio we shadowed broadband installers in Mumbai to understand staff and customer experiences of broadband installation. This helped us to identify opportunities for new digital tools that track installation. We created customer journey maps to better understand the challenges faced by end users, and support the design of a new app for installers.

Surveys: One to one conversations are the preferred method for generative research so you can dig in with follow up questions and see context. However, surveys can be used to supplement this information and gain insight from a broader range of users. At the Department for Health and Social Care we also sent out a questionnaire with exploratory questions, which enabled us to gain insights from a broad range of users that were harder to reach through interviews.

Common questions

I’m replacing a legacy service, do I need to do this?

Yes, often working practices have evolved around systems rather than actually supporting users to do what they need to do. Investing time in some formative research helps to uncover unmet user needs and pain points, ensuring existing inefficiencies are not baked into a new system.

Can’t we just do prototype testing?

It’s tempting to jump ahead to prototyping solutions but there is a risk of making assumptions about the right problem to solve and partially deciding the solution. Once you start down this road it can feel much harder to go back from here once money has been spent and stakeholders are invested in an idea. It’s easy to come up with ideas; the trick is selecting the good ones that solve real problems and add value. Generative research will give you the evidence and confidence you need to choose the right solution.

Even if you have a couple of different ideas to solve an assumed problem, there are risks in showing users multiple prototypes and asking which they want most. Cognitive psychology has taught us that humans don’t have reliable insight into why they make the choices they make, so we can have little confidence in asking users what they want and why.

We have a few ideas already – can’t we just get started?

David Travis and Philip Hodgson share a useful anecdote illustrating this issue in a brilliant book called ‘Think like a UX researcher’. In a consumer study, people were presented with four pairs of identical ladies’ tights labelled A, B, C or D. All of the tights were identical but the majority of users chose option D. This was because of a known position effect, where people have a tendency to choose things from the right hand side. More interesting is the fact that people were able to give reasons for why they had chosen pair D, for example that they were better quality or they had more elasticity.

By exploring needs, rather than wants, through carefully curated open questions and observations we can start to uncover users’ challenges. This helps us to be confident that the solutions we’re proposing will solve real problems and delight users. In the example above, if tights are our ‘product’, we haven’t learned if people actually need to buy tights, because we forced a decision upon them in the research. We can conclude that if people want to buy tights, they want them to be good quality with good elasticity, but what if the solution people needed wasn’t actually tights? Perhaps the real problem is something else: keeping legs warm, dressing modestly or making a fashion statement. We won’t know unless we ask open questions. By giving people a solution before we fully understand the problem and asking them to choose A, B, C or D, we limit our opportunity to identify the right thing.

How to conduct generative research

Successful generative research should follow the steps below. You can read about these steps in more detail in this blog post.

1. Identify your goals – what are you trying to achieve?
2. Create an interview plan to meet these goals
3. Identify users from each subsection of your target audience
4. Run the research including interviews and observations
5. Analyse and synthesise the data into themes
6. Create actionable insights from the analysis

How to make sense of the data

You’re going to generate a lot of qualitative data through the research and this can feel overwhelming. Avoid analysis paralysis by remembering the goals you defined in step one. To make sense of data, you can run a ‘thematic’ analysis. You’re looking for:

Commonalities in the kinds of things users say – this can help you identify the biggest pain points or opportunities.

Differences in what different groups of users say – this can help to establish different personas for your target product or service. There are a number of practical ways to collate and sort this information, including:

• Capture insights in a spreadsheet, and turn them into Post It notes using a digital whiteboard tool such as Miro, then start to group them into themes
• Use a specialist digital tool such as Dovetail to tag the data and identify themes.
• Go old school and stick insights on Post It notes on a big wall

Whichever method you use to collate data, start by identifying big themes like ‘pain points’ and ‘plus points’. Perhaps the pain points theme can be broken down into further subgroups, like time management, technology challenges or quality of data.

The words that users choose are important here, so try to flag some of these in your analysis, as they can help you understand the mental model users have of the problem area. Natural language can be really helpful in the later design phase to produce content and name new products and services in a way that resonates for the target user.

Analysing findings can easily take double the time taken to run each interview, so it’s important to budget adequate time to explore data.

How to turn insights into something useful

Your insights can help you to identify the needs that must be met to solve users’ problems. You can communicate these as user needs or as jobs to be done. Be careful not to solutionise at this stage; it’s important to get to the roots of the problem before trying to solve it. Involving your team can be helpful in identifying appropriate actions from your insights.

A well written user need communicates the problem to be solved without solutionising. For example, as an events planner, I need to know how many people will be attending my event two weeks in advance, so I can plan how many staff I need to run the event.

When you have the problems formulated, you use techniques like ‘How Might We’ statements to start generating ideas. Let’s imagine we have found through research that users are reluctant to buy from smaller retailers online because they are worried about the returns process. Turning these problems into statements can highlight opportunities for a product or service that we are confident meet real user needs.

You might also want to map these user needs visually to bring the information to life for the team. This could be as personas which communicate user needs for different types of users and user journey maps which can show the experience at different touch points across a given task.

Insight: some customers are nervous to shop online with small retailers because they are worried about the time and costs associated with the returns process
How might we: How might we create a fast and seamless returns process?
How might we: How might we provide all the information users need to feel confident in placing an order knowing that they can return it if it’s not right?

Can we design something yet?

At the end of this activity you might find that people don’t really need to solve a problem in the area you were expecting. Don’t be disheartened, money has been saved on building something that wouldn’t add value and you can now use the insights to pivot to a different area with confidence

If you identify a problem to be solved you can start sketching ideas for solutions at this point. For every solution you and the team come up with to solve the problem, cross reference it against how well it meets the core user needs you have identified, as well as technical feasibility and complexity to deliver.

You can do this in a simple table, working as a whole team to formulate potential solutions and weigh them up against their feasibility and potential to help and delight end users.

The next step will be validating these ideas with your end users through the opposite of generative research- evaluative research. Methods like prototype testing and A/B testing can help you to validate with users that you have got the solution right. This can then be fine tuned through ongoing feedback loops with your end user.

This sounds long winded, do we have to do it?

Generative research can help reduce the risk of delivering something that users don’t want, or missing the opportunity to identify solutions that would add the most value. By spending time gathering insights and evidence about user behaviour you can build confidence in the solutions you are proposing. You may feel reluctant to commit to the cost of a discovery but this can be done with a fairly small team. It may be easier to justify when you consider the alternative costs of building the wrong solution, lost revenue, reputational damage, staff inefficiencies as well as development costs incurred by staffing engineers, QAs, maintenance and support for the wrong solution.