Anyone Can Usability Test, Part 1: On a Shoestring

When did you last sit down and talk with users of your product? Do you do this regularly? How often? Who talks to customers? Is it the same people who build your product or field support requests?

Spending time with the people who use your product is one of the most valuable investments your company can make. Put another way, every week you don’t do this is a week where your product might diverge from what your customers want or need. To misquote the old phrase, “no wireframe survives contact with the user” – why not schedule that feedback process early, to avert later pain? This is exactly how we work on our Central Government projects to help them deliver their Digital by Default vision. When you’re spending taxpayers’ money and have a user base of millions, it’s important to test your assumptions early, to save time and money later on.

But even if you don’t have government-sized resources, it’s possible to see immediate value from conducting usability testing. Below, I’ll outline how you can get started without any more resource than an hour or so of your time.

Sit down for half an hour with someone who uses your software (or someone who might want to). I encourage you not to call this “user testing.” In spending this time with you, people may feel vulnerable because you are the ‘expert’ and they are worried about doing something ‘wrong.’ We felt this very keenly when testing for services on our government engagements – people are intimidated by the state! We began sessions by explaining that it was the software we were testing, not the person. When you can reassure someone that there are no right or wrong answers and encourage them to be honest, they reward you with candid, useful information.

If your participant is a practiced user, ask them to walk you through something they often do, and think aloud to explain what they are doing. Are there bits they find confusing or slow? Regular and expert users often have opinions about the software they use, so you may not have to dig very deep to get something useful.

With someone who’s never used your software before, set them a basic task, like signing up for an account, or using a piece of entry-level functionality. Notice where they get stuck. We noticed on many of our engagements that novices sometimes lacked the confidence — or the vocabulary — to talk about how they felt, or the things they didn’t understand. And they had to work much harder just to understand what was happening, so sometimes we needed to remind them to think aloud. When we introduced the sessions, we would often say something like, “this isn’t finished yet, so if anything seems strange or doesn’t work, it’s not your fault.” Letting people know that your product isn’t finished seems to encourage them to be more honest.

Even if your product isn’t anywhere near finished, you can still get feedback on it. Give people sketches, mockups, wireframes, screenshots — anything. What do they think is happening on a given screen? Give them a task to complete: What will this button do? Where would you click next? We took flat wireframes to Starbucks on an iPad just to get quick feedback on the designs, and while we were there, got some new insights into how people feel about transacting with government, too.

Above all, be kind. The person sitting with you is doing you a huge favour. Check your ego at the door, especially if the product is your baby — if someone finds it difficult to use, that isn’t their fault (it isn’t necessarily anyone’s fault). We often get our developers involved in usability testing. The biggest challenge for them is to try not to help too much by asking leading questions or showing people how to use the product. Observe, ask simple, open questions, and listen. If participants ask you for help or for an answer, turn it around. “I’m interested in what you think.”

Ideally, record the session (get permission first). Camtasia, Silverback and Quicktime all let you make screen recordings. If you can’t record, block out half an hour afterwards to sit and make notes on everything that happened — walking through the task again will help refresh your memory. 

Now go and tell a colleague what you learned. Start with the thing that surprised you most, the big headline. Don’t be too surprised if they’re skeptical. It’s very difficult, as an expert, to put yourself in a non-expert’s shoes. But if you have real difficulty being heard, it’s also maybe a sign that people in your your organisation could use more exposure to real users. 

Exposure hours are a big deal. During our work on a Central Government service platform we’ve seen how vital it is for teams to be exposed to real people using the product. We shared real users’ experiences with the team and programme management every fortnight to help them understand how the service needed to change. Exposure builds empathy, and empathy helps you design with user needs in mind, which in turn saves you time and money. Everybody wins. 

This is why you need to do it again — and involve your colleagues this time!

In my next post, I’ll talk about how to scale up your testing when you’re ready.