1. BUSINESS SENSITIVE
Item Non-Response: Does Mode
(Web versus Paper) Make a
Difference?
Glenna Redmond Wolf
Jerome Kramer
Betsy Payn

2. BUSINESS SENSITIVE
Surveys
• We will look at two surveys where respondents
were given the option of completing the survey on
paper or the web
– National Survey on Heart Health Matters for Duchenne
Becker Muscular Dystrophy (DBMD) Carriers
– CDC National Practices in Infant Nutrition and Care
(mPINC)

3. BUSINESS SENSITIVE
Protocol
• CDC National Practices in Infant Nutrition and Care (mPINC)
– Paper survey mailed with instructions for how to complete the web
version of the survey
– Multiple follow-ups (postcard, second survey mailing, and two
telephone reminder calls)
– No cash incentives
– Facility survey (low sensitivity)
• National Survey on Heart Health Matters for Duchenne
Becker Muscular Dystrophy (DBMD) Carriers
– Paper survey mailed with instructions for how to complete the web
version of the survey
– Multiple follow-ups (two reminder/thank you letters, one additional
survey mailing)
– $5 incentive mailed with survey
– Survey available in English and Spanish

4. BUSINESS SENSITIVE
Respondent Characteristics
• CDC National Practices in Infant Nutrition and Care (mPINC)
– Mostly women
– Professional
– Easy access to internet
– High interest in subject
• National Survey on Heart Health Matters for Duchenne
Becker Muscular Dystrophy (DBMD) Carriers
– Women only
– Age 19 or older, given birth to a son with DBMD, and/or believed to be
a carrier of DBMD
– Mixed SES
– High interest in subject

5. BUSINESS SENSITIVE
Design
DBMD mPINC
Paper sent with instructions
for web
YES YES
Web survey allows skips only
after a reminder
YES YES
Progress indicator on web YES
(by section
only)
YES

6. BUSINESS SENSITIVE
Overall Response
Web Paper
DBMD N = 247
17.0%
N = 1206
83.0%
mPINC N = 655
24.8%
N =1986
75.2%

7. BUSINESS SENSITIVE
Formatting/Skip Instructions
• Sub questions set off to the right with an arrow on
the mPINC paper version (topic: prenatal classes
offered by facility) had higher non-response on
paper version:
– Q01a: 66 or 2.7% non-response
– Q01b: 109 or 4.4% non-response
• Only 1 person on the web missed one of these
questions.

8. BUSINESS SENSITIVE
Items with “Not Sure” Option
• Also in mPINC, there is very little non-response
when there is a “not sure” option.
– In four consecutive questions with “not sure” as an option,
there were 10539 valid responses and 25 no responses
(0.2%)
– There were 56 “not sure” responses in these four
questions (0.5%)
• This did not vary between web and paper
completers.
• In DBMD “don’t know” option was given for most
questions where appropriate. This did not
significantly decrease non-response as compared to
questions without the “don’t know” option.

9. BUSINESS SENSITIVE
Adding to 100 percent
• On questions where respondents were asked to
enter information for several things that should total
to 100%, there is more error on paper.
– For one question on mPINC, nearly 92% of respondents
answers’ added to 100%
– Of those that were done incorrectly, 90% were paper
surveys
– Of those who skipped the question entirely (1.4%), more
were done on paper, but the difference was not significant
• Very similar results are found for other similar
questions in the survey.

10. BUSINESS SENSITIVE
Grid-type items
• There were some significant differences in terms of
item non-response on grid-type items
– On one grid in mPINC with 9 items, 3 of the questions had
significantly more non-response for surveys completed on
paper (these were probably the most sensitive questions).
– On another grid in mPINC with 12 items, there were no
significant differences in non-response between web and
paper completers.

11. BUSINESS SENSITIVE
Grid type items continued
– DBMD grid questions on the web were more likely to be
answered with only 0-2% missing data for those items.
The same paper version grid questions were more likely to
be skipped having 1-6% missing data for these items.
– DBMD sensitive health behavior questions in a grid in
paper format had an even higher missing rate (3-10%),
web versions of these same questions maintained a lower
missing rate at 0.5% - 1%.
– Web responders got warnings that they had missed items
if they skipped Q’s, paper respondents got no such
warnings – this may account for the difference in non-
response

12. BUSINESS SENSITIVE
Sensitive non-grid questions
– In DBMD, height, weight, age, smoking and drinking
behavior questions that were not in grid format showed no
significant difference in missing data between web and
paper completes.
– In addition, on a scale item (1-5) regarding feelings about
impacts of DBMD on their lives, the missing rate went up
for both paper and web, but the paper version had a much
higher missing rate (9-22%), than the web version (0.5%)
– On a question regarding family members with DBMD the
missing rate for paper responders was much higher (14-
22%) than for web responders

13. BUSINESS SENSITIVE
Other specify option on grid Q’s
• Paper surveys can leave this blank which results in
missing data.
• Web survey had options of Yes, No or DK on “other
specify” options.
• This results in less missing data – if they select No
or DK we know that they saw the question and there
was no “other” to “specify”.
• No warnings were given on web survey if the other
specify option was skipped.
• Still more paper other specify options were left blank
overall than those administered on the web survey.

14. BUSINESS SENSITIVE
Other specify on grid Q’s continued
Study
Q #
Web N % Web
NR
Paper N % Paper
NR
% diff
between
modes
DBMD
S1q02K 23 11.97 330 27.38 15.4
S1q03L 58 30.20 316 26.22 3.98
S1q14L 38 19.79 323 26.80 7.01
mPINC
A30 84 13.33 336 12.59 0.74
B12 78 12.38 429 21.04 8.66

15. BUSINESS SENSITIVE
Started web and completed on paper
N % of total web
accessed switched
to paper to
complete
DBMD 8 3.1%
mPINC 26 3.8%

16. BUSINESS SENSITIVE
Conclusions
• Overall most people still find that paper is more
convenient than web.
• Thus the higher overall response rate via paper in
mixed paper/web surveys.
• This chart shows that even though some may start
the survey on the web, they may break off and finish
the survey, or start over, using the paper survey
instead.
• Also, if a respondent needs to look up data (as was
the case with the mPINC survey) paper may be
more convenient since they can take the Q’s with
them to look at files etc. and fill in the info directly
onto the paper survey.

17. BUSINESS SENSITIVE
Conclusions, cont.
• Formatting your questions and skip instructions matters for
paper surveys, no need to worry about this with web surveys
since they are automatic
• Grid questions are less likely to be skipped on the web –
reminders that they skipped questions probably account for
this
• You may get a higher response to sensitive questions if you
administer them in web mode with reminders if they skip
them at first.
• The format of web encourages response to Other Specify Q’s
which results in more confidence that the R saw the question
than on paper surveys where non-response to these Q’s is
higher.
• Including a “not sure” or “don’t know” as a response option
may decrease your non-response in both modes