Cautionary Tales of Remote Usability Studies

1. Len Conte UPA October 2008

3. Conversation #8
 Developer (10 minutes before study): I’ve decided
to update the code with the latest feedback.
 Len (silently to self): This should be fun.
 Customer (10 minutes into study): What do all
those red X’s mean ?
 Len : Let’s ask the developer in the observation
room.

4. Conversation #7
 Len: Now that we’ve completed the study, do you
have any other comments about your experience?
 Remote customer: Well maybe you should ask
Justin (the sales rep)…he was actually doing most
of the driving.

5. Conversation #6
 Remote customer: I hear music…is that part of the
study?
 Len: Yeah…we want to see if you can listen to
smooth Jazz and drag & drop at the same time.
 Seems one of my remote observers has put us on
hold!!!

6. Conversation #5
 Len: What’s that sound I hear?
 Remote customer: Fire alarms…
 Len: Feel free to leave ...we can do this later!
 Remote customer: I’m good… I don’t smell any
smoke.

7. Conversation #4
 Len: I’m handing over the control of the mouse to
you, how much of my desktop can you see?
 Remote customer: not much …things look really
big…I have control now…hmmm every time I click
there’s an hourglass…should I REBOOT?
 Len: errr...I have this really nice paper prototype …

8. Conversation #3
(3-way conversation)
 Remote Observer: What can you tell me about Jim
(our customer)?
 Len: He’s got lots of issues…but let’s talk later...
Jim are you on the line?
 Jim (the Customer) : Yup, been here a while…
wanna talk about my issues now?
 Len: Hey Jim, guess you decided to dial-in a little
early…howse the weather in Detroit!

9. Conversation #2a
 Len: I’m sharing. Can you see the application?
 Remote customer: Actually, I can see your entire
desktop…is that your resume?
 Len: shhhh

10. Conversation #2b
 Len: Can you share your desktop?
 Remote customer: Sure…how does that look?
 Len: I didn’t know you guys are into Top Secret
stuff!
 Remote customer: hmm…I think we need a new NDA

11. Conversation #2c
 Len: Before I hand over the mouse to you I’m going
to start the recording on this end.
 Remote customer: OK…mind if record the session on
this end too…TargetLink will be in next week and we
always like to show them stuff you’re working on.

12. Conversation #1
 Len: Lets start with task number 1.
 Remote participant: [Screaming in PAIN] I think I
just blew out my knee!!!
 Len: Are you OK?
 Remote participant : NO. Mike scared me, he came in
without knocking … I jumped up, and jammed my knee
into the corner of the desk… it’s definitely starting to
swell!
 Len: Let’s move on to task 2.

13. Anyone with similar experiences?
Questions?