1. http://www.flickr.com/photos/dannymol/7520467800/
Handheld device usability testing techniques
Walt Buchan - July 2012
1

2. The goal for mobile device
testing has been to recreate
the desktop usability testing
and record the screen of the
device.
Typically this has meant
mounting the device on a sled
with a webcam, or t wo, to
video the screen and the
participants face.
http://www.flickr.com/photos/witflow/5936043207/ 2

3. That struck me as the wrong
approach. Like putting a
participant under a
microscope.
We know that people use
phones and tablets anytime,
anywhere, anyhow. I wanted
to get testing closer to that
ideal, real environment of
use.
http://www.flickr.com/photos/slumadridcampus/5750162705/ 3

4. Something where the
participant is as unimpeded by
recording equipment and
technology as possible.
But without the cat.
http://www.flickr.com/photos/earlysound/4490601295/ 4

5. The technique needed to
acknowledges the revolution in
usage that phones and tablets
have achieved.
No more do you have to sit at a
desk on an office chair in room
on your own. Now you can
watch TV at the same time too.
http://www.flickr.com/photos/smays/4489922240/ 5

6. Wait a minute, the guys that
are paying for the testing
still want to watch the test
and sleds allowed that
But using a sled creates an
opportunity to video the
screen and the participant’s
hands. Now we can see how
they physically use the phone
too.
http://www.flickr.com/photos/number10gov/5346045124/ 6

7. No sled here, just a camera
suspended over the area
where the participant will
use the phone.
Now they can pick the device
up, put it down, switch to
landscape and back. Noting
to impede their usage. And
the clients can still watch the
streamed video of the test.
7

8. How to record a mobile user test
8

9. cxpartners first mobile phone
test rig back in 2007 DV/HDMI
.
Cameras are great as they
have optical zoom and fixed
focus.
Webcams, unfortunately,
appear to autofocus on human
skin not shiny screens. That
makes videoing people using
phones very difficult.
9

10. Capture a contextual video. The
usability kit in development, this kit
can be used without the table in user
testing sessions.
Kit includes:
• DV cam
• Microphone stand
• EasyCap capture card (Windows and OSX
compatible)
• Use the EasyCap soft ware to record the
test session
• Mirror the computer screen via VGA to
an external monitor for clients
10

11. Other ideas to get that video quality
tip top.
• Use black sugar paper or ‘photographers
velvet’ to reduce white balance problems
• use phones with black cases, if possible
• anti-reflective sticky screens for the
phone to cut the glare
• indirect lighting
11

12. Alternatively use a
document camera.
The lower quality of the
camera means the
participant will lose
flexibility of use.
http://www.flickr.com/photos/teachingsagittarian/5528385861/ 12

13. Wish list -
The Note-Taker system using a
Point Grey Firefly camera and
the super clever system
developed by David Hayden.
The combination of a machine
vision cameras and multi-touch
www.haydenat.com pan and zoom control of the
picture is irresistible.
13

14. Sleds are available
14

15. www.mrtappy.com/
photos.htm
www.measuringusability.com/ www.noldus.com/human-
blog/mobile-usability-test.php behavior-research/accessories/
mobile-device-camera-mdc There are some great sleds
available, if that’s the
direction you choose.
15

16. My sled, made of K’Nex (my
kids haven’t forgiven me) and
a phone holder that would
normally be stuck to a car
windscreen.
It’s very cheap, very light,
fits any phone and spins to
allow landscape use of
phones.
16

17. Sleds are likely to make
people use t wo hands when
they normally use one hand
People can feel uncomfortable
holding them for long periods
of time (to be fair mobile
tests shouldn’t need people to
hold the phone for a long
time)
People may not pick the sled
up leaving them to jab at the
phone with a finger
References are included at
the end.
17

18. Alternatives that capture screen activity
18

19. Display Recorder
Only works on jailbroken iOS
phones. That’s not something
I’d recommend, particularly
in a test environment.
http://rpetri.ch/cydia/displayrecorder/
19

20. UX Recorder
Very promising soft ware to
record mobile websites and
shell apps. Currently waiting
for approval from Apple, I
assume ( in July 2012)
http://www.uxrecorder.com/
20

21. Test Studio
A kit to add whilst an app is
being developed. that will
record how Beta apps are
used.
http://www.telerik.com/automated-testing-tools/ios-testing.aspx
21

22. Summary
22

23. There’s been a debate that the
technique used to capture the
session depends on the objectives
of the test.
e.g.
Comprehension of the content -
maybe a sled is better
Ergonomics of the app - maybe a
contextual video is better
http://www.flickr.com/photos/petroleumjelliffe/694990145/ 23

24. I think the debate is over.
Always gather a contextual video.
With the right camera you’ll get
context and see, in detail, the screen
the participant sees and you’ll see
how the device is used.
If you can, add a screen recorder to
collect and stream the screen of the
device.
You don’t need a sled, their days are
numbered.
http://www.flickr.com/photos/ideath/6978494627/ 24

25. Reference list:
HCI Beyond the GUI. Design for Haptic, Speech, Olfactory, and Other Non-traditional
Interfaces. Edited by Philip Kortum - 2008, Morgan Kaufmann Publishers Inc.
Henry Been-Lirn Duh et al. Usability Evaluation for Mobile Device: A Comparison of
Laboratory and Field Tests - 2005
Rudy Schusteritsch et al. Towards the Perfect Infrastructure for Usability Testing on Mobile
Devices - CHI 2007
Apologies
I’m aware this presentation is skewed toward testing iOS devices. It’s a reflection of the
commercial testing I’ve been involved with.
I intend to revise the presentation to include ideas for user testing Android and Microsoft
and other OS devices.
25

26. Thank you
Walt Buchan
walt.buchan@cxpartners.co.uk
@silverfoxyboy
26