June 10th, 2009 by David Poteet

Usability Testing Hack – “Digital” Paper Prototyping

About this post:

David and Melissa stumbled on this effective substitute for usability testing and paper prototyping.

Filed under Experience Design, Tips, Tricks & Hints

Sometimes the best ideas are born out of last minute necessity. We had an experience recently that led to a great new technique for conducting paper prototypes without the paper. It gives us many of the advantages of paper prototyping but it can even be done remotely.

We were working in January with Imperial College London Research Services to help them restructure their site so that it better supported the goals and needs of people seeking research funds or managing research projects.

We traveled to London for two and a half days of intensive on-site planning, during which we planned to

analyze the data we had collected through a carewords survey and baseline usability tests of the existing site
get input from a variety of stakeholders through KJ sessions
synthesize our findings to create new wireframes for the site
develop the wireframes into a paper prototype of the new site
conduct usability tests with the paper prototype

We wanted them to have a clear and validated direction for the new site by the time we left.

If you're not familiar with paper prototyping, it is a technique where you create a prototype of the site using paper and ordinary office supplies. You can draw screens by hand or create them on the computer and print them out. One person acts as the computer and manipulates the paper "screens." Another person facilitates the test and asks questions, while a third person participates as the user. When the user points to things he or she would click on, the person acting as the computer updates the screens.

Paper prototypes are quick, support rapid iteration, and save the expense of preparing a clickable version of the site.

Things were going smoothly until the last day when we were scheduled to do usability testing. We were creating our wireframes in OmniGraffle and we were working furiously to pull together all the pieces and sample content we would need to conduct a thorough test. It was getting down to the wire – only 15 minutes until the first users arrived. We realized we didn't have time to print everything for the paper prototype.

My colleague Melissa Beaver and I looked at each other and said "How can we do this test?"

"Why don't we project it on the wall and use that like a paper prototype?"

"Great idea, let's try it."

Here's how we ran the test.

We projected our wireframes on the wall from OmniGraffle.
I acted as the computer just as we would have done in a paper prototype test
We asked the test participants, instead of sitting in front of a computer or working with sheets of paper, to stand in front of the projector screen and point to the things they would click if they were using the site.

It worked amazingly well.

One of the things we found was that because there was a bit of a delay in the time it took me to change screens, Melissa was able to ask clarifying questions, and there was time for users to talk about what they were expecting to see on the next screen. It also allowed Melissa to take better notes through the process.

This hybrid digital / paper prototyping technique also gave us the advantage of not having to worry about whether everything that needed to be clickable was actually clickable in the prototype. We could just respond to what they thought was clickable, and I (as the computer) could jump to where I thought it should go. And we were able to adapt the prototype as needed as we went forward.

A month later we were scheduled to conduct usability tests for a client in the DC area. Eva Baird, one of our user experience architects based in Tucson, AZ, needed to be present for the tests but the timing and costs were not convenient for her to travel to DC. We were trying to work out some solution and it occurred to us that this same technique could be used remotely with a screen sharing program. We invited one of our partners, Daimon Caulk of Modal, to come and facilitate the tests for us in DC while Eva acted as the computer remotely from Tucson.

We relied on a screen-sharing program called Glance that we use for a lot of our virtual presentations. The process with Glance worked like this:

Eva shared her screen with OmniGraffle open and the wireframes on the screen, while the participant went through the usability test. She could pass control of the mouse back and forth to the test participant.
Daimon introduced each scenario and took notes as the users tried to find things or accomplish tasks.
Users moved the mouse to the thing they would click on and said "Click."
Eva would then take over control and switch to the appropriate screen, and give control back to the user.

We found that all of the test participants, as soon as they understood what was going on, basically forgot that there was another person on the other end controlling the computer. They had no trouble participating in this manner.

Taking this approach has saved the time we would normally spend creating a clickable prototype, especially when we want to be able to iterate quickly and test variations of the site. In the case I described with Eva and Daimon, we were able to run a test, make changes, and then run another test later in the same week and validate the changes.

The first two times we used this technique we had thumbnail views of the pages visible on the left side of the OmniGraffle screen. We did find that users were sometimes trying to click on the thumbnails, thinking they were part of the interface. So we now hide them and use the drop-down menu in OmniGraffle to switch screens – although that did make us wonder if some sort of page thumbnail view would make a good navigation model in some interfaces!

Daimon has now facilitated multiple tests with this digital paper prototyping technique, so I asked him to share his experiences:

Here are some tips from Daimon on facilitating a digital paper prototype test session:

The Right Setup- Make sure you have a good speakerphone and decent bandwidth. You'll need a good speakerphone so the second facilitator can hear the queues. Also, consistently good bandwidth is required because of the need to share the screen
Describe the Method- It's important to describe the test method to the participant; with one participant I forgot to mention the second facilitator was on the speakerphone so as soon as she spoke it scared him
Show and Tell- I made sure I demonstrated the screen behavior, how to interact with the screen, and how the participant needed to think aloud letting the second facilitator know their desired action
Close the Canvases- One thing I noticed was that if we had the canvases pane opened the participant tended to "look ahead" or use it as a navigation element, so make sure you hide your canvases
They Will Adapt- After a brief walkthrough of how everything was going to work the participants adapted surprisingly well to the interaction

I would definitely recommend this method especially if you are in between wireframing and designing as it gives you an opportunity to work with the Information Architect directly. It's quick and it is as good if not a bit better than using paper alone.

Let us know if you've done something like this, or have suggestions for improving the process!

Note:
Check out our earlier blog posts about the Imperial College Research Services project – Part 1 and Part 2