Recruiting Better User Research Participants: UPA 2011

23 June 2011 1 Recruiting Better User Research Participants Recruiting Better User Research Participants Principal of Research & Strategy Electronic Ink UPA 2011 23 June, 2011

Why talk about recruiting? 23 June 2011 Recruiting Better User Research Participants 2 User research is only as good as the participants you recruit. It’s difficult to recruit participants who … ,[object Object]

Can express themselves and provide useful information

Show up,[object Object],[object Object],[object Object],[object Object],[object Object]

What we’ll talk about 23 June 2011 Recruiting Better User Research Participants 8 Recruiting for moderated research ,[object Object]

Recruiting backup participantsRecruiting for unmoderated, remote research ,[object Object]

Using online intercepts,[object Object]

Recruiting for Moderated Research. 23 June 2011 Recruiting Better User Research Participants 10 Who should do the recruiting?

Who should do the recruiting? 23 June 2011 Recruiting Better User Research Participants 11 Recruiting options You Your client A recruiting company

Recruiting for Moderated Research. 23 June 2011 Recruiting Better User Research Participants 12 Doing the recruiting yourself

Doing the recruiting yourself 23 June 2011 Recruiting Better User Research Participants 13 Advantages You know what you’re looking for. You have more control over who is recruited. You sound more human and less like a telemarketer. You save money.

Doing the recruiting yourself 23 June 2011 Recruiting Better User Research Participants 14 Disadvantages It’s difficult if you don’t already have a list to work from. It’s time consuming to find, contact, screen, and schedule. It may require a lot of cold calling.

Doing the recruiting yourself 23 June 2011 Recruiting Better User Research Participants 15 Start with lists of potential participants - customer lists, membership lists, employee directories, etc.

Or gather participants from your website with a recruiting page or an online intercept. Doing the recruiting yourself 23 June 2011 Recruiting Better User Research Participants 16

Or advertise or connect through social media and other sources. Doing the recruiting yourself 23 June 2011 Recruiting Better User Research Participants 17

Recruiting for Moderated Research. 23 June 2011 Recruiting Better User Research Participants 18 Having your client do the recruiting

Having your client do the recruiting 23 June 2011 Recruiting Better User Research Participants 19 Advantages Clients have access to employee, customer, and member lists. Employees, customers, or members are more likely to volunteer. It saves you money and time.

Having your client do the recruiting 23 June 2011 Recruiting Better User Research Participants 20 Disadvantages Clients may not be very good at recruiting. Clients may not possess the time and organizational skills required. Clients often pass on vague or incorrect information about the research.

Having your client do the recruiting 23 June 2011 Recruiting Better User Research Participants 21 Provide your client with detailed instructions. Profile of who to recruit And who not to recruit

Having your client do the recruiting 23 June 2011 Recruiting Better User Research Participants 22 Provide your client with detailed instructions. Schedule of sessions

Having your client do the recruiting 23 June 2011 Recruiting Better User Research Participants 23 Provide your client with detailed instructions. Description of the study to give to participants

Having your client do the recruiting 23 June 2011 Recruiting Better User Research Participants 24 Ensure they do it correctly. Delegate to someone with time and organizational skills. Monitor the recruited participants. Email the scheduled participants to introduce yourself and provide more details.

Recruiting for Moderated Research. 23 June 2011 Recruiting Better User Research Participants 25 Using a recruiting company

Using a recruiting company 23 June 2011 Recruiting Better User Research Participants 26 Advantages It saves you a lot of time and effort. It’s a good option when you have no existing lists or other resources. With the right information, a good recruiting company can get the best participants.

Using a recruiting company 23 June 2011 Recruiting Better User Research Participants 27 Disadvantages It’s the most expensive recruiting option. You might get “professional” participants who are only in it for the money. With poor direction or a bad recruiting company, you can get the worst participants.

Using a recruiting company 23 June 2011 Recruiting Better User Research Participants 28 Selecting a recruiting company Get referrals from other researchers. Get quotes from several recruiting companies - cost and confidence. Look for good communication. Cultivate several recruiting companies you can use regularly.

Using a recruiting company 23 June 2011 Recruiting Better User Research Participants 29 A strong and clearly written screener is essential.

Using a recruiting company 23 June 2011 Recruiting Better User Research Participants 30 Review the list of recruited participants. Ensure they fit what you’re looking for. Have your client review and approve the list also - CYA!

Recruiting for Moderated Research. 23 June 2011 Recruiting Better User Research Participants 31 Deciding who will do the recruiting

Deciding who will do the recruiting 23 June 2011 Recruiting Better User Research Participants 32 Do your own recruiting when: You have more time than money. You don’t mind or are effective at cold calling. You can hire someone else to do the recruiting – intern, staff recruiter. You can use existing lists, recruit from your website, or advertise. You already have a connection with potential participants.

Deciding who will do the recruiting 23 June 2011 Recruiting Better User Research Participants 33 Have your client do the recruiting when: Your client has detailed contact lists of employees, customers, or members. Your client already has a connection with potential participants. Your client has the time and coordination to do this effectively. Your client has successfully done recruiting in the past.

Deciding who will do the recruiting 23 June 2011 Recruiting Better User Research Participants 34 Hire a recruiting company when: You have more money than time. You don’t want to “cold call” and screen participants. You don’t have an easy way to find potential participants.

Recruiting for Moderated Research. 23 June 2011 Recruiting Better User Research Participants 35 Writing a better screener

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 36 Keep people from hanging up on you Quickly get to the point. You’re not selling anything. You’re recruiting for a paid study.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 37 Ask likely elimination questions first Only the most promising will go through all the questions.

Eliminate the usual suspects Eliminate recent participants. How would you answer this question if you wanted to get into the study? Writing a better screener 23 June 2011 Recruiting Better User Research Participants 38

Eliminate the usual suspects “Professional” participants might lie. Recruiters should check their records before calling people. “A few years ago.” Writing a better screener 23 June 2011 Recruiting Better User Research Participants 39

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 40 Eliminate conflicts of interest Existing relationship with your client - employee, employee of a competitor, etc. Too much inside knowledge - web designer, developer, marketer, etc.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 41 Avoid reading multiple choice answers The “correct” answer is often obvious How would you answer this question if you wanted to get into the study?

Avoid reading multiple choice answers The “correct” answer is often obvious “None of these” Writing a better screener 23 June 2011 Recruiting Better User Research Participants 42

Ask open-ended questions instead of reading the answers How would you answer this question if you wanted to get into the study? Writing a better screener 23 June 2011 Recruiting Better User Research Participants 43

Ask open-ended questions instead of reading the answers Requires the recruiter to know which professions should terminate the screening Writing a better screener 23 June 2011 Recruiting Better User Research Participants 44

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 45 Don’t make the elimination answers obvious How would you answer this question if you wanted to get into the study?

Don’t make the elimination answers obvious “19 – 24” seems like a safe answer. Writing a better screener 23 June 2011 Recruiting Better User Research Participants 46

Include multiple elimination answers. How would you answer this question if you wanted to get into the study? Writing a better screener 23 June 2011 Recruiting Better User Research Participants 47

Include multiple elimination answers It’s not obvious which answers will get you eliminated. Writing a better screener 23 June 2011 Recruiting Better User Research Participants 48

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 49 Avoid giving away what you’re looking for How would you answer this question if you wanted to get into the study?

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 50 Avoid giving away what you’re looking for “Yes”

Avoid giving away what you’re looking for Providing many choices obscures what you’re looking for How would you answer this question if you wanted to get into the study? Writing a better screener 23 June 2011 Recruiting Better User Research Participants 51

Avoid giving away what you’re looking for Providing many choices obscures what you’re looking for Writing a better screener 23 June 2011 Recruiting Better User Research Participants 52

Avoid giving away what you’re looking for Beware of those who choose everything to try to get into the study. Writing a better screener 23 June 2011 Recruiting Better User Research Participants 53

Avoid giving away what you’re looking for Set a reasonable limit to eliminate fakers. Writing a better screener 23 June 2011 Recruiting Better User Research Participants 54

Avoid giving away what you’re looking for Use open-ended questions to completely hide what you’re looking for. Writing a better screener 23 June 2011 Recruiting Better User Research Participants 55

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 56 Ask questions that only qualified participants can answer

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 57 Ask questions that only qualified participants can answer

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 58 Require proof Require participants to provide proof of qualification. Provide a graceful exit, so that people don’t have to be caught in a lie.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 59 Recruit based on behavior and attitudes Most demographics aren’t very important for small-scale research.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 60 Recruit based on behavior and attitudes Screen for desired behaviors.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 61 Recruit based on behavior and attitudes Negative attitudes towards your client or subject matter may disqualify people.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 62 Recruit based on behavior and attitudes Screen for desired attitudes.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 63 Recruit based on behavior and attitudes Interpret attitudes and behaviors based on answers. What do you think we’re looking for here?

Recruit based on behavior and attitudes Interpret attitudes and behaviors based on answers. People interested in Japanese cars Writing a better screener 23 June 2011 Recruiting Better User Research Participants 64

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 65 Don’t ask questions that are difficult to answer Can you answer this question easily and accurately?

Don’t ask questions that are difficult to answer Estimating the average number of hours per day is easier. Is this question easier to answer? Writing a better screener 23 June 2011 Recruiting Better User Research Participants 66

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 67 Don’t use the screener to gather information Only include questions that screen participants. Save information gathering for a pre-study questionnaire.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 68 Screen for computer, Web, and technology experience You may want to eliminate those with very low and very high experience.

Screen for computer, Web, and technology experience Self-assessments allow those who are truly uncomfortable with technology to excuse themselves from the study. But don’t rely only on self-assessments. Writing a better screener 23 June 2011 Recruiting Better User Research Participants 69

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 70 Screen for computer, Web, and technology experience Make your own assessment based on activities and behaviors.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 71 Screen for eloquence Ask an open-ended question related to your topic to judge expressiveness.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 72 Screen for eloquence But be careful not to eliminate qualified, quiet participants. “Normal” people may not be as entertaining as “professional” participants. Recruit representative participants, not only those who speak in perfect sound bites.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 73 Ensure that people are physically able to participate

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 74 Let people know what they’ll be doing Eliminate surprises and potential discomfort. Allow those who would be uncomfortable to back out.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 75 Test your screener with a colleague Ask a colleague to try to fake his/her way through the screener. Recruiting company should point out problems in logic.

Writing a better screener 23 June 2011 Recruiting Better User Research Participants 76 Get your client to approve the screener – CYA! Your client becomes responsible for the types of people recruited. It’s difficult for them to complain later about the quality of the participants.

Recruiting for Moderated Research. 23 June 2011 Recruiting Better User Research Participants 77 Minimize No-Shows

Minimize No-Shows 23 June 2011 Recruiting Better User Research Participants 78 Locate your research conveniently. Where are your participants usually located during your session times? How easy is it to get to and find your location? Traffic? Public transportation? Parking? Is it more convenient to go to your participants or conduct remote sessions?

Minimize No-Shows 23 June 2011 Recruiting Better User Research Participants 79 Schedule your research conveniently What are your ideal participants usually doing during your session times? Schedule a variety of session times. Who’s most available for weekday research?

Minimize No-Shows 23 June 2011 Recruiting Better User Research Participants 80 Avoid days and conditions that result in more no-shows Fridays Days before holidays Bad weather – snowstorms, rainstorms, etc.

Minimize No-Shows 23 June 2011 Recruiting Better User Research Participants 81 Provide appropriate incentives. Make it worth their while to show up. Make it appropriate to the person and to the activity.

Minimize No-Shows 23 June 2011 Recruiting Better User Research Participants 82 Contact participants to confirm Remind them to attend. Give directionsand parking / transportation information. A good way to receive early warning of no-shows and get a head start on recruiting replacements

Recruiting for Moderated Research. 23 June 2011 Recruiting Better User Research Participants 83 Recruit Backup Participants

Recruit Backup Participants 23 June 2011 Recruiting Better User Research Participants 84 Two approaches to backup participants Floaters Over-recruiting Floaters Over-recruiting

Recruit Backup Participants 23 June 2011 Recruiting Better User Research Participants 85 Floaters Wait on standby to fill in for no-shows or inappropriate participants

Recruit Backup Participants 23 June 2011 Recruiting Better User Research Participants 88 Over-recruiting Recruit several extra participants to use for no-shows Dismiss and pay those you don’t need

Recruit Backup Participants 23 June 2011 Recruiting Better User Research Participants 91 Use floaters when You want to be sure your observers always have a participant to observe You don’t have time for an extra day of backup sessions You don’t have money to pay for an extra day of sessions at a research facility

Recruit Backup Participants 23 June 2011 Recruiting Better User Research Participants 92 Use over-recruiting when It’s not important that your observers have a participant to watch for each session You have time and money for an extra day of backup sessions You want to save money on incentives Your participants don’t want to wait around as floaters

Recruiting for Unmoderated, Remote Research. 23 June 2011 Recruiting Better User Research Participants 93

Recruiting for Unmoderated, Remote Research 23 June 2011 Recruiting Better User Research Participants 94 What is different about unmoderated, remote research? Large numbers of participants from anywhere in the world can participate Participants visit a website to complete the study on their own time Unmoderated tools automatically capture and present the data

Recruiting for Unmoderated, Remote Research 23 June 2011 Recruiting Better User Research Participants 95 With a large number of potential participants, different methods are required to: Find Screen Contact And convince people to participate

Recruiting for Unmoderated, Remote Research 23 June 2011 Recruiting Better User Research Participants 96 Three recruiting methods for unmoderated studies Emailing participants Online panel Online intercept

Recruiting for Unmoderated, Remote Research. 23 June 2011 Recruiting Better User Research Participants 97 Emailing Potential Participants

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 98

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 99 Gather a list of potential participants Use existing lists of customers, members, employees, etc. Recruit from your website Advertise

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 100 Narrow the list of participants using existing information.

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 101 Avoid adding screening questions to your study Each question adds to the length of the study, increasing abandonment.

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 102 Determine how many people to email Determine the minimum number of participants for acceptable results. Tullis and Wood showed 30 - 50 is needed for online card sorting. [1] Assume a low response rate of 5% and email an appropriately larger population. [1] Tullis, Tom, and Larry Wood. “How Many Users Are Enough for a Card-Sorting Study?” Proceedings UPA 2004, Minneapolis, MN, June 7–11, 2004. Retrieved March 7, 2011: http://home.comcast.net/~tomtullis/publications/UPA2004CardSorting.pdf

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 103 Determine how many people to email Determine the maximum number of participants to accept. Avoid paying too much in incentives.

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 104 Keep it short, briefly describe the study, and provide a link to the study

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 105 Don’t provide the study instructions in the email

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 106 Highlight the incentives, both monetary and altruistic

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 107 Provide appropriate incentives: individual incentives are small but add up.

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 108 Provide appropriate incentives: entry into a drawing is often easier.

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 109 Provide appropriate incentives: with equal appeal to everyone.

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 110 Establish credibility with individual emails in a friendly, professional tone.

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 111 Establish credibility by providing contact information.

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 112 Establish credibility by sending the emails from a familiar person.

Emailing Potential Participants 23 June 2011 Recruiting Better User Research Participants 113 Monitor participation and send reminders Monitor the number and type of people who have completed the study. If necessary, send reminder emails.

Recruiting for Unmoderated, Remote Research. 23 June 2011 Recruiting Better User Research Participants 114 Using an Online Panel

Using an Online Panel 23 June 2011 Recruiting Better User Research Participants 115 A panel is a database of people who have volunteered for surveys.

Using an Online Panel 23 June 2011 Recruiting Better User Research Participants 116 Panels are provided by many recruiting and market research companies.

Using an Online Panel 23 June 2011 Recruiting Better User Research Participants 117 Panel members are already pre-screened You provide the details of the people you’re looking for instead of a screener. The recruiter filters the database to find people matching those characteristics.

Using an Online Panel 23 June 2011 Recruiting Better User Research Participants 118 Screening You can also add additional screening questions to the beginning of your study. Panel participants are motivated to take surveys, so you don’t have to worry as much about extra questions leading to abandonment.

Using an Online Panel 23 June 2011 Recruiting Better User Research Participants 119 You pay the panel company based on the number of “completes.” For each person who completes the study, you pay a recruiting fee and small incentive. Limit your costs by ending the study after a maximum number of participants.

Using an Online Panel 23 June 2011 Recruiting Better User Research Participants 120 Quality of panel participants They want to participate in studies, and they want to be invited back. This motivates them to take the study seriously and provide thorough feedback.

Using an Online Panel 23 June 2011 Recruiting Better User Research Participants 121 Quality of panel participants They are more likely to be “professional” participants, primarily in it for the money. They may not represent your typical user.

Recruiting for Unmoderated, Remote Research. 23 June 2011 Recruiting Better User Research Participants 122 Using Online Intercepts

Using Online Intercepts 23 June 2011 Recruiting Better User Research Participants 123 Online intercepts pop-up on websites to invite people to participate.

Using Online Intercepts 23 June 2011 Recruiting Better User Research Participants 124 Those who accept are taken to a brief screening questionnaire.

Using Online Intercepts 23 June 2011 Recruiting Better User Research Participants 125 Those who pass screening can either be … ,[object Object]

Contacted immediately for a remote, moderated study

Taken directly into an unmoderated study,[object Object]

Otherwise you may need a developer to create the intercept and tie it into the site.,[object Object]

Automatic screening and recruiting saves time.

They can remain on a website indefinitely, continually recruiting for long-term studies. ,[object Object]

Intercepted participants have less motivation to complete the study.

Require a lot of site traffic or waiting a long time to get enough participants.,[object Object]

Using prize drawings may be an easier, less expensive incentive.** Be careful of the legal issues surrounding contests and sweepstakes. Consult a lawyer if needed.

Recruiting for Unmoderated, Remote Research. 23 June 2011 Recruiting Better User Research Participants 130 Choosing Between These Methods

Choosing Between These Methods 23 June 2011 Recruiting Better User Research Participants 131 Email the participants if … ,[object Object]

You don’t need to do much additional screening.

Recruiting Better User Research Participants: UPA 2011

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (17)

Recently uploaded

Recently uploaded (20)

Recruiting Better User Research Participants: UPA 2011

Editor's Notes