Effects of splitting long surveys into two
•Transferir como PPTX, PDF•
0 gostou•280 visualizações
The document summarizes research into the effects of splitting long surveys into two shorter surveys. It finds that data quality is maintained or improved with shorter surveys, as respondents make fewer errors and provide more detailed open-ended responses. Results from different question sets are largely equivalent between survey lengths. Respondent satisfaction may also be higher for shorter surveys, as indicated by more positive survey comments. While not a definitive solution, splitting long surveys appears to address issues like respondent fatigue without harming data quality or equivalence.
Recomendados
Recomendados
Mais conteúdo relacionado
Destaque
Destaque (15)
Semelhante a Effects of splitting long surveys into two
Semelhante a Effects of splitting long surveys into two (20)
Mais de Annie Pettit, Research Methodologist
Mais de Annie Pettit, Research Methodologist (13)
Último
Último (20)
Effects of splitting long surveys into two
- 1. Effects of Splitting Long Surveys Into Two Annie Pettit Chief Research Officer Tweet! #AAPOR @LoveStats
- 2. 90 77 79 73 72 62 43 40 34 31 21 14 36 28 25 21 15 9 http://www.people-press.org/2012/05/15/assessing-the-representativeness-of-public-opinion-surveys/ Tweet! #AAPOR @LoveStats
- 3. 50% of surveys are >=15 minutes 5 10 15 20 25 30 35 40 45 50 55 60+ Tweet! #AAPOR @LoveStats Source: DIY Sampling Panel
- 4. Tweet! #AAPOR @LoveStats 12 Minute Survey N=300 Split incentive Method 12 Minute Survey N=300 Split incentive 20 Minute Survey N=600 Full incentive
- 5. Problem: Nothing is directly Comparable • People choose whether to participate • Each version is different – Different length – Different topics – Different leading/priming questions Tweet! #AAPOR @LoveStats
- 6. Tweet! #AAPOR @LoveStats Agenda 1. Data quality 2. Data equivalence 3. Responder satisfaction 11/6/2014 6
- 7. Data Quality 11/6/2014 7 Tweet! #AAPOR @LoveStats
- 8. Fewer Red Herrings Tweet! #AAPOR @LoveStats 2 fake hotels 2 fake airlines 2 fake loyalty programs
- 9. Straightlining P<.05 Tweet! #AAPOR @LoveStats 50% 40% 30% 20% 10% 0% Fail Straight 1 Fail Straight 6 Fail Straight 10 Fail Straight 64 Fail Straight 66 Fail Straight 85 Fail Straight 91 Fail Straight 93 Fail Straight 104 Long Short A Short B
- 10. Follow Instructions Better Did not follow instruction to “choose only one” Tweet! #AAPOR @LoveStats
- 11. Better Open Ends Tweet! #AAPOR @LoveStats Optional
- 12. Speeding differences not discernable Tweet! #AAPOR @LoveStats Long Short A Short B Standardized Number of Seconds m=0, sd=1, ceiling outliers removed
- 13. Drop the worst 5% from each dataset 6.1% of Long 6.3% of Short A 5.2% of Short B Tweet! #AAPOR @LoveStats
- 14. Data Equivalence Tweet! #AAPOR @LoveStats
- 15. Survey Middle: Trips Taken Tweet! #AAPOR @LoveStats 80% 60% 40% 20% 0% 1 to 3 trips 4 or more trips Long Short A Short B
- 16. Survey Middle: Airline Aware P<.05 P<.05 Tweet! #AAPOR @LoveStats P<.05 P<.05 100% 75% 50% 25% 0% Long Short A Short B
- 17. Survey End: Like Advertising Tweet! #AAPOR @LoveStats 60% 50% 40% 30% 20% 10% 0% Youtube Facebook Flyers Billboards Long Short A Short B P<.05
- 18. Survey End: Privacy Concerns Tweet! #AAPOR @LoveStats 60% 40% 20% 0% Top2Q91privacy Top2Q91security Top2Q91family Top2Q91home Top2Q91online Long ShortB
- 19. Survey End: Personality Tweet! #AAPOR @LoveStats
- 20. Respondent Engagement Tweet! #AAPOR @LoveStats
- 21. Same Completion Trend Optional watch a commercial Optional watch a commercial Tweet! #AAPOR @LoveStats 100% 75% 50% 25% Long Short A Short B Grid
- 22. More Words in Verbatim Tweet! #AAPOR @LoveStats 60 40 20 0 Verbatim 1 Verbatim 2 Verbatim 3 Verbatim 4 Verbatim 5 Verbatim 6 Long ShortA ShortB Optional P<.05
- 23. Possibly Higher Satisfaction P<.05 P<.05 Tweet! #AAPOR @LoveStats 100% 75% 50% 25% 0% Easy Boring Long Reco Long Short A Short B P<.05
- 24. More Positive Survey Comments Tweet! #AAPOR @LoveStats 75% 86% 93% Long Short A Short B Percentage of optional survey comments that were positive
- 25. Conclusions • Not a slam-dunk win • Data quality is as good or better • Results are equivalent • Survey satisfaction is as good or better Tweet! #AAPOR @LoveStats
- 26. Final Thoughts • Responders self-select for a 10 or 20 minute Tweet! #AAPOR @LoveStats experience • Differences are in line with sampling error • Large portions of most surveys do not need to be answered by everyone. Plan ahead.
- 27. Thank you! Annie Pettit Chief Research Officer Peanut Labs annie@peanutlabs.com @LoveStats on Twitter www.peanutlabs.com Tweet! #AAPOR @LoveStats : )