One of the reasons we enjoy working on Government Digital Service exemplar projects so much is the way they’ve approached the thorny issue of user research.

Why is this such a ‘thorny’ issue? After all, doing research with ‘real people’ is one of those things everybody agrees is generally a good thing to do, right?

But product owners and delivery teams often seem to struggle to find the time for regular research. And that’s because, even if you’re doing it on a shoestring it’s something that takes effort.

User research takes effort because it is one extra thing to think about. At the pace of an agile delivery project, sometimes it’s hard to synchronise the effort of research with the speed of delivery. Without a thoughtful research plan, and a library of user research, it might appear to be non-trivial to match up the right piece of insight with the right bit of design and development – and this means teams sometimes struggle to see the value of doing the research in the first place. (It’s not necessarily that hard, as we’ll see below, but it is one more thing to think about while you’re trying to build momentum).

5,000 miles away

And then there’s the cost. The team we were in at the Home Office spent time putting together a persuasive business case for a research project abroad while we were building the China Visit Visa product. Here’s what the Home Office Product Owner had to say about the user research project 5,000 miles away.

We invested our travel budget in two trips to China – to support transition testing and to conduct post-release user research. Both were money well spent, highlighting a variety of changes and improvements we could make. There is no doubt in my mind that one of our big lessons learnt is that there would have been value in sending our researchers out to China much earlier.

An interface in Chinese?

It’s not just the cost of the trip itself, but the sheer amount of preparation and planning required to pull it off. Especially, when you’re building a product with an interface in Chinese!

An obvious target for measuring how effective user research is seems to be ‘how much.’ How much research did we do? Or how many users did we speak to? You can argue that the type of usability testing (don’t call it user testing!) teams often do can be measured with ‘number of issues and fixes identified’. That makes sense if you’re testing and are focused on increasing quality. But it makes a lot less sense if you’re viewing user research as an investment.

You should be seeing user research as something which can save you money rather than some narrow use as a bug-fixing, snag-spotting tool. Done right, user research can save you a ton of effort and take you far beyond the bureaucratic nature of compliance efforts. You don’t often see Product Owners wishing they’d put less effort into user research or wishing they’d left it to the last minute.

Exposure Time

At GDS, they’re big on Exposure Time. Everybody in a delivery team should see people using the product you’re building so measuring this makes sense – it’s a great proxy metric for success and widely applicable to all delivery teams.

But I’ve got a couple of metrics I monitored very closely during my time with Equal Experts at the Home Office. Admittedly, they’re not as concrete as the minimum of ‘2 hours for everybody, every 6 weeks’. But they helped me understand when we were doing something right.

• How many developers and BAs turned up voluntarily and asked questions, laughed or grimaced at the ‘UX corner’ segment of the weekly showcase?

Every week we did around a 10-minute slot at showcase and showed ‘supercuts’ of issues (we edited user research footage together into themes and highlighted common difficulties). Or we talked through research summaries and asked the room to come up with ideas why users were having problems.

Closing the loop

I gave a special extended ‘UX Corner’ presentation to a packed house when we got back from China. This sparked discussions which carried on for days. The photos I took (which you can see in the blogs linked to above) were particularly useful. You could see developers ‘closing the loop’ as they gained, for the first time, a sense of the scale of the ‘real world’ operation they were feeding with the software we were building.

• How many times did developers make improvements or suggestions to the UX designs I produced for the product?

These suggestions are easily the most gratifying measure of user research carried out well. When developers make suggestions for how to improve the product you’re working on based on the clips of footage or summaries of research they’ve seen, you know you’re doing something very right.

A valuable investment

Done right, user research may cost you a significant amount of effort, but it is a valuable investment. When the whole team feels like they have the tools to contribute to the overall experience, you release less stuff which needs fixing. Quality, even if it involves sending researchers all the way to China, is usually cost effective.