In this webinar with NewMR sponsor Ayda and Ray Poynter, the challenges facing research in 2022 are addressed in two presentations. In the second of two presentations for this event, Ray Poynter of NewMR highlights seven steps you can take to simultaneously improve the participant experience and collect better data. Ray draws on more than 40 years of research and collecting better data. Focusing on online surveys, Ray shows seven straightforward approaches you can use to take your data collection to the next level. To listen to the recordings from this event, visit NewMR.org/Play-Again.

1. 7 Tips for Asking Better Questions
Ray Poynter
14 April 2022

2. Sponsors
Communication

3. Agenda
📌 HX and the research instrument
1. Create a narrative flow
2. The 4 basic rules of questions
3. Avoid ‘Norman Questions’
4. An answer for everybody
5. Mobile first
6. The last question
7. Testing and monitoring
8. Other thoughts
9. Q & A

4. HX (Human
Experience)
& the
Research
Instrument
People have to be willing to complete the
questionnaire
The questionnaire has to hold people’s
attention
The questionnaire should leave people happy
to do further projects
HX
The questionnaire should collect information
that is consistent when it should be
consistent. Consistent between people and
consistent within individuals.
The questionnaire should identify differences
where relevant differences exist. Between
people and within individuals
Research
Instrument

5. Create a Narrative Flow

6. The 4 basic rules of questions
A. People should understand what the question is asking
• Would you describe your condition as Acute or Chronic?
B. People should know what the answer is
• How fast is your internet connection at home, in Megabits per second?
C. People should be willing to answer the question
• Do you overclaim your expenses at work?
D. You should be able to interpret the answer
• Was the train clean and punctual?

7. Avoid ‘Norman’ Questions
Copyright Alan Levine, https://www.flickr.com/photos/cogdog/16110218680/, CC 2.0
If price was not an issue and if all of
these options were equally available
to you, and you were going to switch
to a new energy provider next year,
which would you pick?
1. Apple Energy
2. Amazon Feed
3. Microsoft Fuel
4. Tesla Tent
5. None of these
TLDR – long text usually means bad question

8. An answer for everybody
Which is your favorite?
1. Chicken
2. Beef
3. Pork
Which State do live in?
1. Alabama
2. Alaska
49. Wisconsin
50. Wyoming
Options
• None
• Other
• Other – please specify
• Don’t know
• Not applicable

9. Mobile First
Are the questions
and answers visible?
Make sure it renders
OK on computers
and tablets
Can a typical user
manipulate the screen to
answer the questions?
Which is your favorite? Please
drag the options into order,
most favorite 1st, least 7th
1. Coke
2. Diet Coke
3. Pepsi
4. Pepsi Max
5. Fanta
6. 7 Up
7. Dr Pepper

10. The last question
Is there anything else you’d like to tell us
about this topic or this survey?
1. ________________
________________
________________
2. No

11. Testing and
Monitoring
You and the client should always take the survey
on your mobile phones – does look and feel
right?
With a printout of the questionnaire – check that
every question can be reached/displayed
Use random a response generator to test the
questionnaire
Soft launch
Check the answers to the last question. Are
there too many DKs, NA, ‘strange’ responses.
How long did people take to do the survey?
Monitoring during the
fieldwork
Similar to the soft launch

12. Other Thoughts?
Don’t create ‘new’
scales and
questions (mostly)
Quality measures –
e.g traps
System 1 and 2 Cognitive load
Games and
Gamification
Adaptive scripting
Chat and
conversational
approaches
Voice
Showing and
uploading video
Virtual reality

13. Questions?

14. Sponsors
Communication