You know what your users experience in the lab setting, when they are in a controlled environment. But what happens when they actually take your product into the real world and try to use it? The technology explosion in the market research field has resulted in a wealth of new tools that allow UX designers and researchers to deploy users to test sites, apps, and products in the real world and report back on their experiences in ways that are actionable and meaningful. This session will arm attendees with the knowledge and technique they need to conduct mobile ethnography studies on their own. We will cover the end-to-end process of designing your research, identifying the right tool to conduct the research, and how to report back the results in engaging ways.

1. Out of the
Lab and
Into the Wild
TAKING USER EXPERIENCE RESEARCH TO THE USER
IN THEIR ENVIRONMENT JAY ZALTZMAN
BUREAU WEST
ABBY LEAFE
NEW LEAFE RESEARCH
Jay Zaltzman jay@bureauwest.comAbby Leafe abby@newleaferesearch.com

2. Part 1
WHY GET OUT OF THE LAB?

3. What’s in your
queue?

4. Cable customers tend to recall
this stuff…

5. …but not this stuff

6. Why?
Fear of judgment is only part of it
It’s hard to remember off the top of your head
Multi-viewer households

7. There’s lots that lab research is
good at
Usability
Nomenclature
Taxonomy
Task completion
Eye tracking
https://www.flickr.com/photos/dumfstar/8553474140

8. But there’s more to user
experience than usability

9. Decision-
making:
in the lab
vs real life
https://www.flickr.com/photos/vaxzine/2278300537

10. Case Study
Objective: A company was
developing a new navigation
app and wanted the best user
experience possible.
Approach: During an initial interview, participants discussed their
current navigation app and how they use it. Then they were shown the
new app. Then the (brave) researcher and a client got into the
participant’s car with the participant and drove to several locations using
the new app.

11. Case Study
Objective: Identify unmet needs
and potential products for dog
owners around the world.
Approach: Phase 1: week-long mobile diary during which participants
recorded their dogs’ activities, answered questions, and uploaded
photos and videos. Phase 2: online bulletin board discussion to uncover
unmet needs. Information from the diary served as stimulus and
catalyst for ideas.

12. Case Study
Objective: To understand how a
beta version of a new set-top box
operating system was functioning in
the real world.
Approach: One-on-one in-home interviews, using a task-based
approach to have participants walk through likely tasks (set up recording,
find On Demand programming, etc.). Respondents also kept a diary the
week prior to the interview to note any problems or pain points.

13. Case Study
Objective: Valpak had just launched
a responsive version of their website
and wanted to understand how its
mobile coupons performed in the
real world.
Approach: In part 1, respondents participated in a traditional one-on-
one usability interview in a lab setting. In part 2, they had 1 week to use
a Valpak mobile coupon in a store of their choice, and then report back
on the experience with photos and video via a mobile research app.

14. Part 2
QUESTIONS TO ASK YOURSELF

15. Is this study right for
ethnography?
 Look for experiences that you can’t recreate in
the lab
 Mobile apps are ripe for ethnography
because they are often designed for
use on-the-go

16. Should I attend?
There are plenty of good
reasons to be present…
or not

17. Pros & cons of in-person
ethnography
* Get to see things first hand
* Easier rapport
* See the whole environment
* Time consuming
* Can cost more
* Potential for observer bias

18. Pros & cons of remote
ethnography
* Privacy can create intimacy
* Can reach larger samples
* Less time and money
* See only what they show
* Don’t see body language
* Need to herd the kittens

19. What else should I be asking?
Do I go it alone?
(Consider partnering -- especially the first time)
What will the deliverable look like?
(Allocate resources)

20. Part 3
BLUEPRINT FOR CONDUCTING THE RESEARCH

21. Start with your target
Who do you really want?
May be similar to lab research
Consider screening for articulation,
openness, etc.

22. Make your expectations clear
In preparation for this study, please plan to clean
out your car so that two passengers beside
yourself can sit in it. This means you may have to
remove child car seats or other items from the
front and back seats of your car.
Also, please email us photos of your driver’s
license, car registration and proof of insurance.

23. Trust, but verify
Have respondents take a picture of
something that confirms their
qualification and send it to you (cable
bill, prescription medicine, etc.)
Consider pre-interviewing them on the
phone so you don’t waste time

24. Remote research options
Mobile screen capture
Diary (video, photos, voice, text)
Webcam
Many great providers to choose from
https://commons.wikimedia.org/wiki/File:FP_Satellite_icon.svg

25. In-home research
considerations
Safety and security -
for both research team
and respondents
Respect their space
Video

26. Make your expectations clear
Consumers:
 Don’t clean your house
 Don’t make snacks
 You will be recorded
 Expect 2-4 people
Observers:
 Cell phones off
 Dress professionally but no
company logos
 Don’t educate
 Poker face

27. Writing the guide
In-person:
◦ Allow time for set-up and getting comfortable
Remote:
◦ Be explicit
◦ Give examples if possible
◦ Give staggered timelines
(i.e. Complete X by Y)
All:
◦ Make it fun!

28. Keeping them engaged
Positive reinforcement – both privately and publicly
Question of the Day – so diaries don’t get repetitive
Bonus payments for certain activities

29. Other logistics
Have a dedicated videographer
Two hours and 3 observers is comfortable
What kind of release do you need?
Wear good socks

30. Don’t underestimate
the amount of data
you’ll get

31. Thoughts on reporting
Consider using a Livescribe pen
Think visually
Include photos and video when possible

33. Jay Zaltzman
jay@bureauwest.com
Abby Leafe
abby@newleaferesearch.com