March 6, 2015
Last time, I talked about how to start running usability testing on a shoestring. In this post, I’ll explain how you can scale up the shoestring version to something like the industry standard.
The good news – if you’ve done this with one person on a few occasions, then it’s a case of ‘more of the same,’ with some extra structure in place to help you be more objective and get actionable outcomes.
People love to debate the ‘right’ number of participants to test with. The best advice I can offer is this: Acknowledge that for every new person you test with, you will probably learn something interesting and useful — but that past a certain point, the law of diminishing returns applies. Where you draw the line depends on your resources, but many teams find that testing with 5 – 8 people strikes a good balance.
Work out what feature or functionality you want to test, and how long that might take you to work through. Asking someone about what they’re doing and why can slow things down considerably. Remember that novice users may take much longer to do something than regular users, so pick a task you think everyone should be able to accomplish in the available time. Keep sessions under an hour, or people’s attention will start to wane.
If you can, recruit people who are all similar in ways that you care about – experts, first-time users, older adults, people who use mobile devices but never PCs, etc. Remember that the distinction between expert and novice can apply to different devices and platforms as well as your product, so ask about this when looking for participants. Testing with a group of similar users on a similar platform generally allows you to draw stronger conclusions — though remember, those conclusions might only apply to that particular combination of demographic and platform.
Write a script to take to the session, to make sure you run through the same tasks and questions with everyone. That might sound unnecessary, but you can only say “5 out of 6 people said they did X” if you really did ask all 6 people — and you’d be surprised how easy it is to go off-piste from your script once the conversation starts. The act of preparing a script also encourages you to set clear goals for the session (for example, what will “this feature is usable” look like in terms of user behaviours?) and order the tasks sensibly.
Consider incentives. This depends partly on resources, but also on where you are recruiting. Within your own organisation, people may take part for free – providing tea and biscuits goes a surprisingly long way. If you are bringing people in from outside who like your product, just being heard may be reward enough, though giving away small corporate goodies costs little and may be appreciated. In fact, it’s not unusual for companies with a proper research budget to give incentives north of £50 for an hour of someone’s time, because they know the true value of conducting regular usability testing. But if you’re not there yet, starting small is fine.
Allow plenty of time at the start for explaining the task, and at the end for answering any questions. Scheduling people to arrive on the hour is easiest for you and them to remember. If you intend to run the sessions back-to-back, allow 15 minutes between one person leaving and the next arriving, so you can reset the software, stop the recording, and grab a drink or run to the bathroom. Remember that helpful volunteers might also be the kind of people who helpfully arrive 10 minutes early.
If you can, record the sessions. Software like Silverback, Camtasia or Morae lets you record the screen and where people click on it, and a small picture-in-picture of the participant. Having a digital recording of your session opens many doors. You can review it again later, play the best clips to colleagues at your next meeting, or share them electronically. If you do record, make sure you get explicit permission. And while it goes without saying that nobody wants to end up on YouTube, some participants do seem to want reassurance on this point.
Get someone else to take notes. Unless you are superhuman, it’s just not possible to keep track of what your participant is saying, be framing the next question, and take notes about what they say and do. This is the perfect opportunity to get your sceptical colleagues on board, and the quickest way to cut through their cynicism – it’s much harder to discount someone’s experiences when you’ve met and talked to them. Remember to explain to your participant why there’s another person in the room, and introduce them.
Pull out the best bits of the recordings to share with your team. Verbatim quotes are great; video clips of those quotes happening are even better. Some of the best clips don’t even need words – participants frowning or moving in close to the screen when they don’t understand is a classic. Everything that surprises you or which just flat-out didn’t work can help your team get closer to what you should be building. Share your findings, and encourage everyone to keep them in mind when designing.
Done that? Great — now pick a different feature, or a different demographic, and do it all again!
In my next post, I’ll talk about making the most of your user research findings, and some super-effective ways of analysing and sharing them.
All business models have their pros and cons. But, when you consider the type of problems we help our clients to solve at Equal Experts, it’s worth thinking about the level of experience and the best consultancy approach to solve them.
If you’d like to find out more about working with us – get in touch. We’d love to hear from you.