1.
Gaining design insights
from your
research recruiting process
Dana Chisnell
@danachis

2.
Lesson learned

3.
How long?
How often?
How bad?
Medications?
Employed?
Insurance?

4.
TERMINATE (interview).

5.
Recruiting as user research:
bonus data

6.
Recruiting as user research:
conversational interviews

7.
How do I get started?
Where do I find people?
What do I do with old customer data?
Why is recruiting so time-consuming?
How can I stop no-shows?

8.
Steps to recruiting
happiness
•
•
•
•
Sourcing
Screening
Scheduling
Compensating

9.
Do it
yourself.
keep it in house
bond closely with consistent
resource

10.
Include the recruiter in the
study planning

11.
Objective of the study
Format of the sessions
Behavior to be observed

12.
First contact
This is part of the customer
experience

13.
Recap:
•
•
Recruiting is part of the research.
You should do it yourself.

14.
1. Sourcing
Snowball recruiting can be
more effective than your
database

15.
Sources
•
•
•
•
•
•
•
•
Snowball
Online social networks
Churches, school groups, social clubs
Professional associations
User groups, conventions, and conferences
Support groups
Intercepts
Craigslist

16.
Panels
•
Can take time to get a pull from a
company database
•
•
•
Customer data is out of date
People have to opt in
Some make a living responding to surveys
and doing studies

17.
Biases:
Every sample has a bias.
Mix them up.

18.
2. Screening
The art of the open-ended
interview.

19.
The art of the interview
•
•
This is not a test.
•
Frame the conversation based on the
study objectives
•
Get the respondent to volunteer
information
It’s a conversation to learn about the
person

20.
Be open
•
•
•
•
“tell me about...”
“when was the last time you...”
“say more about that...”
“what’s the thing you like most about x...?”

21.
Know who shouldn’t
be there
•
Create a question that will show a faker is
a faker
•
Be clear about who you don’t want
in your study

22.
Behavior versus
demographics
•
•
Demographics don’t predict behavior
Focus on actions you want to observe

23.
Behavior versus
demographics
•
•
Regulations may bound age ranges
Beware targeted segments

24.
plays first-person shooter games
downloads movies
applies for benefits
files claims
buys groceries
books hotels
shares photos

25.
Classifiers
•
Attributes that may indicate differences in
behavior

26.
Set expectations
•
•
•
•
•
•
•
Recording
NDAs
Homework
Possible personally identifying information
Using their device or yours
Alone or with other participants
Being observed

27.
Name
Books own
travel?
Business trips
in 12 mo
Pleasure trips
in 12 mo
Terry
Y
25
2
Pat
N
30
5
Tracy
Y
10
1
Keith
Y
9
2
Leslie
Y
15
1
Ari
Y
2
2
Kelly
N
50
0
Erin
Y
5
5

28.
More hints
•
•
•
Ask all the questions
You’ve just started a relationship
Respondents get invested

29.
So far
• Treat recruiting
like research
• Recruiting is first
contact
• Every sample has
bias
• Use networks to
find sources of
participants
• Focus on behaviors
• Screening
establishes a
relationship

30.
Coming up
• Scheduling
• Compensating
participants
• Case studies

31.
Break
Questions & Answers

32.
• Scheduling
• Compensating
participants
• Case studies

33.
3. Scheduling
It’s all about tradeoffs.

34.
Where = timing
•
Remote sessions offer the greatest
flexibility
•
Visiting participants can make them more
available
•
Going to you is actually the most timeconsuming for participants

35.
4. Compensation
Be as generous as possible.

36.
Compensation, not incentive

37.
Pay or gift?
•
•
•
•
•
Compensate as soon as possible
Cash is ideal
Gift cards can work
Licenses or subscriptions
Donations

38.
Remember to thank the people
who helped you find
participants, too.

39.
Send a thank-you note.

40.
Gaining design insights from the
research recruiting process

41.
Case 1: Usability test of a
health monitoring app
•
People with epilepsy who track symptoms
and meds

42.
Questions
What’s it like to have this
condition? Have you always had
it? How do you cope? Tell me
about meds.

43.
Surprise
A lot of the people we wanted to
meet were not patients.
They were caregivers.

44.
Case 2: Interviews about
travel experiences
•
People who travel between two major
cities
•
A mix of business travelers, pleasure
travelers, and weekenders

45.
Questions
Tell me about the last time you made
the trip.
[If needed] Why did you make the
trip?
[If needed] Who were you with?
[If needed] How long did it take?
[If needed] How often do you make
this trip?

46.
Surprise
It’s not just the purpose of the
trip, it’s the people you’re with.

47.
Case 3: Usability test of a
hotel booking site
•
People who stay in hotels
when they travel

48.
Questions
Tell me about your last trip.
Where’d you stay?
How did you decide where to
stay?
How did you make a
reservation?

49.
Surprise
We actually wanted people who
book their own hotel rooms.

50.
Summary
•
Do it yourself or include the recruiter when
you plan your study
•
Continuous, snowball recruiting prequalifies & expands the sample
•
•
Biased samples aren’t all bad
Open-ended, voice-to-voice interviews are
key to show rates

51.
Summary
•
Open-ended screening reveals nuances
that can bust your assumptions
•
You may lose real users if you aren’t
•
•
•
open
flexible
attentive

52.
Bonus tip!
•
•
•
Confirm the appointment by email
Remind the participant by email
Remind the participant by phone

53.
usabilitytesting.wordpress.com