This document provides an outline for a survey design workshop. The workshop objectives are to understand the importance of a rigorous survey design process, consider common survey administration methods, examine nuts and bolts of questionnaire design such as question style and response formats, consider implementation issues like sampling methods, and critically review example surveys. The workshop will cover research methods, questionnaire design, levels of measurement, sampling, and survey resources. [END SUMMARY]

1. Survey Design
Workshop
Inter-University Research
Workshop Program
Dr. James Neill
Centre for Applied Psychology
University of Canberra
1 February, 2011

2. Outline
• Objectives
• Introductions
• Logins & Resources
• Research methods
• Questionnaire design
• Levels of measurement
• Sampling
• Evaluation 2

3. Objective 1
Understand the importance of a
rigorous, step-by-step process
in planning, developing &
implementing research
questionnaires
3

4. Objective 2
Consider the pros and cons of
common methods for
survey administration
2. Face-to-face interview
3. Telephone survey
4. Mail survey
5. Internet/mobile survey
4

5. Objective 3
Examine nuts & bolts of
questionnaire design e.g.,
2 Question style,
3 Response formats,
4 Layout, and
5 Pre- and pilot testing
5

6. Objective 4
Consider implementation
issues
2 Sampling methods
3 Sample size
6

7. Objective 5
Critically review example
surveys.
2 Existing examples
3 Student in-progress examples
with a view towards planning,
drafting, and/or revising of an
initial draft (pilot) survey.
7

8. Resources
• Survey Design Workshop Notes
(Wikiversity)
• Readings
• Books about surveys design and
survey research (check library)
8

9. Introductions
Introductions
9

10. Types of Research
(Research Methods)
There are 3 main research methods:
2.Experimental
3.Quasi-experimental
4.Non-experimental
10

11. Types of Research -
Experimental
Characterised by:
• Random assignment
• Control over extraneous variables
11

12. Types of Research -
Quasi-experimental
Characterised by:
•Non-random assignment
•Control over some extraneous
variables
•Groups are “naturally occuring”
12

13. Types of Research -
Non-experimental
Characterised by:
•No “groups” or “conditions” are
created or used (i.e., no full
experimental or quasi-experimental
groups)
•Minimal control over extraneous
variables
13

14. Survey Research Characteristics
•Surveys are widely used in
non-experimental social science
research.
•Often use interviews or
questionnaires.
•Involve real-world samples.
•Often quantitative, but can be
qualitative. 14

15. History of Survey Research
•Survey research methodology
was initially developed in the
1940's – 1960's.
•Since the 1980's, theories and
principles evolved to create a
unified perspective on the design,
conduct, and evaluation of
surveys.
15

16. 8 Survey Research Characteristics
Backstrom & Hursh-César (1981, pp. 3-4)
1Systematic: follows a specific set of
rules, a formal and orderly logic of
operations
2Impartial: selects units of the
population without prejudice or preference
3Representative: includes units that
together are representative of the problem
under study and the population affected
by it
16

17. 8 Survey Research Characteristics
Backstrom & Hursh-César (1981, pp. 3-4)
4Theory-based: operations are
guided by relevant principles of human
behaviour and by mathematical laws of
probability (chance).
5Quantitative: assigns numerical
values to nonnumerical character
6Self-monitoring: procedures can be
designed in ways that reveal any
unplanned and unwanted distortions
(biases) that may occur 17

18. 8 Survey Research Characteristics
Backstrom & Hursh-César (1981, pp. 3-4)
7Contemporary: it is current, more
than historical, fact-finding
8Replicable: other people using the
same methods in the same ways can get
essentially the same results
18

19. Advantages of
Survey-Based Research
• Ecological validity
• Access to wide range of
participants
• Potentially large amounts of data
• May be more ethical
(than experiments)
19

20. Disadvantages of
Survey-Based Research
• Lack of control
→ less internal validity
• Data may be 'superficial'
• Can be costly to obtain
representative data
• Self-report data only
• Potentially low compliance rates
20

21. Example Surveys
Unit Satisfaction Survey
Community Library Use
21

22. The research process
1. Establish
need for info/
research
2. Problem
6. Reporting definition/
Hypotheses
5. Data 3. Research
analysis design
4. Sampling/
Data collection
22

23. The research process
23

24. Survey construction:
Overview
1What is a survey?
2Types of questionnaires
3Questionnaire development
4Writing questions
5Types of questions
6Response formats R LOM
7Survey formatting 24

25. What is a survey?
•A standardised stimulus
•A measuring instrument
•A way of converting fuzzy
psychological stuff
into hard data
for analysis
25

26. Types of surveys
Types of
surveys
Self - Interview -
administered administered
Postal Delivered and TelephoneFace to face
collected structured
interview
Web-based 26

27. Questionnaire development
1. Formulate 2. Expand
generic the
questionnair questionnair
Turneinto Question e Draft qs &
separate order & response
sections based funnel qs formats
on
study objectives.
4. Finalise 3. Pre-test,
questionnair pilot test,
e & redraft
& implement 27

28. Formulate Generic Questionnaire
• Turn objectives into sections of
the survey
• Ensure all questions relate to
research objectives
• For explanatory objectives or
hypotheses ensure both
dependent and independent
variables exist
28

29. Cover Letter & Ethics Statement
Introduction or cover letter:
• Outline details of research project and
allow for ethical informed consent.
• Few will read it without good prompting
and being easy-to-read
29

30. Instructions
• Provides consistency - helps to
ensure standard conditions across
different administrations
• Explain how to do the survey in a
user-friendly manner
• Example:
Life Effectiveness Questionnaire
30

31. Instructions: Example
31

32. Cover letter / ethics statement
checklist
Outline details of research project e.g.,:
• Who are you? Are you bona fide?
• Purpose of survey?
• What's involved?
• Explain any risks/costs/rewards
• How will results be used?
• Human ethics approval #
• How is consent given / not given?
• Voluntary - can choose not to continue anytime
• More info: Complaints, how to obtain results,
32

33. Layout
•Font (type, size)
•No. of pages
•Margins
•Double vs. single-siding
•Colour, etc.
33

34. Layout
Demographics
• single section, usually at
beginning or end of
questionnaire
• only include relevant questions
34

35. Layout
•Space for comments?
•Indicate the end
•Say thanks!
35

36. Flow/Structure
• Logical order of questions
(use sections)
• Ask screening questions first, rather than
later. Does the participant qualify for the
survey? (esp. for internet surveys)
• Use funneling/branching questions to move
respondents through survey
• Start off with easy to answer and engaging
questions
• More controversial questions in middle36

37. Expanding the
Survey
37

38. Personal Information
• Generally, researchers put
personal information at beginning
of survey (if required). However,
this may put off respondents, so
also consider uncluding at towards
end.
• Consider response format e.g,
Income in categories or ranges
38

39. Personal Information
• More likely to respond to personal
questions for anonymous or mail
surveys as opposed to face to
face or telephone
• Show cards for responses may
help for face to face interviews
39

40. Pre- & pilot-testing
•Pre-test items on convenient others -
ask for feedback
•Revise items e.g.,
–Which don’t apply to everybody
–Are redundant
–Are misunderstood
–Are non-completed
• Reconsider ordering & layout
•Pilot test on a small sample from the
40

41. Types of Questions
Be able to justify and defend your
choices... 41

42. Writing questions - Dos
1 Define target constructs
2 Check related research &
questionnaires
3 Draft items
(for important, fuzzy constructs aim to have
multiple indicators)
4 Pre-test & revise
42

43. Writing questions - Dos
• Focus directly on topic/issue
• Be clear
• Be brief
• Avoid big words
• Use simple and correct grammar
43

44. Writing questions – Don'ts
Inapplicable –
must apply to all respondents
Over-demanding –
e.g., recall of detail or time-consuming,
unnecessary questioning
Ambiguous –
meaning must be clear to all respondents
Double negative –
e.g., Do you not disapprove of tax reforms?
44

45. Writing questions – Don'ts
Double-barrelled -
e.g., “Do you think speed limits should be
lowered for cars & trucks?”
Leading -
e.g., “don’t you see some danger in the
new policy?”
Loaded –
e.g., “Do you advocate a lower speed limit
to save human lives?” vs “Does traffic
safety require a lower speed limit?” 45

46. Response biases
• Social desirability
• Acquiescence
– yea- and nay-saying
• Self-serving bias
• Order effects
46

47. Demand characteristics
Interview
• High demand characteristics
• Can elicit richer information
Questionnaire
• Lower demand characteristics
• Information may be less rich
47

48. 48

49. Accuracy of recall
decreases over time
49

50. Objective questions
• A verifiably true answer (i.e., factual
information) exists for each unit.
• The question could be accurately
answered by an observer.
e.g.,
How times during 2009 did you visit a
G.P.? ______
50

51. Subjective questions
•Asks about fuzzy personal perceptions.
•There is no “true”, factual answer.
•Many possible answers per unit.
•Can't be accurately answered by an
observer. e.g.,
Think about the visits you made to a G.P.
during 2010. How well did you
understand the medical advice you
received?
perfectly very well reasonably poorly not at all
51

52. Objective vs. subjective
questions
• Both types of questions may be
appropriate; depends on the
purpose of the study.
• One criticism of this distinction:
There is no such thing as “objective”
and that all responses are
subjective.
52

53. Open-ended Questions
• Rich information can be gathered
• Useful for descriptive, exploratory
work
• Difficult and subjective to analyse
• Time consuming
53

54. Open-ended questions
• Rich information can be gathered
• Useful for descriptive,
exploratory work
• Difficult and subjective to
analyse
• Time consuming
54

55. Open-ended questions:
Examples
What are the main issues you are
currently facing in your life?
How many hours did you spend
doing university study last
week? _________
55

56. Closed-ended questions
• Important information may be
lost forever
• Useful for hypothesis testing
• Easy and objective to analyse
• Time efficient
56

57. Levels of
Measurement
=
Type of Data
Stevens (1946)
57

58. Levels of measurement
•Nominal / Categorical
•Ordinal
•Interval
•Ratio
58

59. Discrete vs. continuous
Discrete
----------
Continuous
___________
59

60. Each level has the properties of the preceding
levels, plus something more! 60

61. Categorical / nominal
•Conveys a category label
•(Arbitrary) assignment of #s to
categories
e.g. Gender
•No useful information, except as
labels 61

62. Categorical / nomimal example:
Phrenological labels
62

63. Ordinal / ranked scale
•Conveys order, but not distance
e.g. in a race, 1st, 2nd, 3rd, etc. or
ranking of favourites or preferences
63

64. Ordinal / ranked example:
Ranked importance
Rank the following aspects of the
university according to what is most
important to you (1 = most important
through to 5 = least important)
t Quality of the teaching and education
Q Quality of the social life
Q Quality of the campus
Q Quality of the administration
Q Quality of the university's reputation64

65. Interval scale
•Conveys order & distance
•0 is arbitrary
e.g., temperature (degrees C)
•Usually treat as continuous for > 5
intervals
65

66. Interval example:
8 point Likert scale
66

67. Ratio scale
•Conveys order & distance
•Continuous, with a meaningful 0
point
e.g. height, age, weight, time, number
of times an event has occurred
•Ratio statements can be made
e.g. X is twice as old (or high or heavy)
as Y 67

68. Ratio scale:
Time
68

69. Why do levels of
measurement matter?
Different analytical procedures
are used for different
levels of data.
More powerful statistics can be
applied to higher levels
69

70. Levels of measurement:
Revision question
Fill in all cells
Level Prop-erties Examples
Descriptive Statistics
Graphs
Nominal / Categorical
Ordinal / Rank
Interval
Ratio
70

71. Closed-ended rating scales
1.Dichotomous
2.Multichotomous
3.Verbal frquency scale
4.The list (multiple response)
5.Ranking
6.Likert scale
7.Graphical rating scale
8.Semantic differential
71

72. Dichotomous
2 response options e.g.,
Excluding this trip, have you
visited Canberra in the previous
five years?
__ Yes __ No
Provides the simplest type of
quantification 72

73. Multichotomous
How many hours did you spend
doing university study last week?
__ less than 5 hours
__ > 5 to 10 hours
__ > 10 to 20 hours
__ more than 20 hours
73

74. Multichotomous
More than two possible answers e.g.,
What type of attractions in your
current trip to Canberra most
appeal to you?
__ historic buildings
__ museum/art galleries
__ parks and gardens
74

75. Verbal frequency scale
Over the past month, how often have
you argued with your intimate
partner?
1. All the time
2. Fairly often
3. Occasionally
4. Never
5. Doesn’t apply to me at the moment
75

76. The list (multiple response)
Provides a list of answers for
respondents to choose from e.g.,
Tick any words or phrases that
describe your perception of
Canberra as a travel destination:
__ Exciting __ Important
__ Boring __ Enjoyable
__ Interesting __ Historical
76

77. The list (multiple-response)
What are the main issues that you are
currently facing in your life? (tick all that
apply)
__ financial
__ physical / health
__ academic
__ employment / unemployment
__ relationships
__ other (please specify)
77

78. Ranking
Helps to measure the relative
importance of several items
Rank the importance of these
reasons for visiting Canberra (from
1 (most) to 4 (least)):
__ to visit friends and relatives
__ for business
__ for educational purposes
78

79. Likert scale
Measures strength of feeling or
perception.
Indicate your degree of agreement
with this statement:
“I am an adventurous person.”
(circle the best response for you)
1 2 3 4 5
strongly disagree neutral agree strongly
disagree agree
1 2 3 4 5
strongly agree neutral disagree strongly
agree disagree
79

80. Graphical rating scale
How would you rate your enjoyment
of the movie you just saw?
Mark with a cross (X)
not enjoyable very enjoyable
80

81. Semantic differential
What is your view of smoking?
Tick to show your opinion.
Bad ___:___:___:___:___:___:___ Good
Strong ___:___:___:___:___:___:___ Weak
Masculine ___:___:___:___:___:___:___ Feminine
Unattractive ___:___:___:___:___:___:___ Attractive
Passive ___:___:___:___:___:___:___ Active
81

82. Non-verbal scale
Point to the face that shows how
you feel about what happened
to the toy.
Also called an idiographic scale.
82

83. Verbal frequency scale
Over the past month, how often
have you argued with your
intimate partner?
1. All the time
2. Fairly often
3. Occasionally
4. Never
5. Doesn’t apply to me at the moment
83

84. Sensitivity & reliability
•Scale should be sensitive yet
reliable.
•Watch out for too few or too
many options.
84

85. Number of response options?
How many response options?
•Minimum = 2
•Average = 3 to 9
•Maximum = 10?
Basic guide: 7 +/- 2
85

86. Number of response options?
Likert scale example
AGREEMENT ABOUT SOMETHING
2-Categories
DISAGREE AGREE
3-Categories
DISAGREE NEUTRAL AGREE
4-Categories
STRONGLY MILDLY MILDLY STRONGLY
DISAGREE DISAGREE AGREE AGREE
5-Categories
STRONGLY MILDLY MILDLY STRONGLY
DISAGREE DISAGREE NEUTRAL AGREE AGREE
86

87. Watch out for too many or too
few response options
“Capital punishment should be
reintroduced for serious crimes”
1 = Agree 2 = Disagree
1 = Very, Very Strongly Agree 7 = Slightly Disagree
2 = Very Strongly Agree 8 = Disagree
3 = Strongly Agree 9 = Strongly Disagree
4 = Agree 10 = V. Strongly Disagree
5 = Slightly Agree 11 = V, V Strongly Disagree
6 = Neutral
87

88. Example: How could
this question be improved?
How old are you?
___ 18-20
___ 20-22
___ 22-30
___ 30 and over
88

89. Example: How could
this question be improved?
Are you satisfied with your marriage
and your job?
__________________________
89

90. Example: How could
this question be improved?
You didn’t think the food was very
good, did you?
_____ Yes _____ No
90

91. Example: How could
this question be improved?
Environmental issues have become
increasingly important in choosing
hotels. Are environmental
considerations an important factor
when deciding on your choice of
hotel accommodation?
____ Yes ____ No
91

92. Example: How could
this question be improved?
What information sources did you
use to locate your restaurant for
today’s meal?
(please tick appropriate spaces)
____ yellow pages
____ Internet
____ word of mouth
92

93. Comparison of
data collection methods
Personal Telephone Mail
Data collection costs High Medium Low
Data collection time required Medium Low High
Sample size for a given budget Small Medium Large
Data quantity per subject High Medium Low
Reaches widely dispersed sample No Maybe Yes
Reaches special locations Yes Maybe No
Interaction with respondents Yes Yes No
Degree of interviewer bias High Medium None
Severity of non-response Low Low High
Presentation of visual stimuli Yes No Maybe
Fieldworker training required Yes Yes No
Alreck and Settle (1995:32) 93
Alreck & Settle (1995; 32)

94. Finalise Questionnaire Draft
• Questions need to be exhaustive
and mutually exclusive
– Include ‘other (please specify)’
– Ensure categories do not overlap
94

95. Finalise Questionnaire Draft
• Length
– Try to keep them as short as
possible
– Only ask questions that relate to
objectives
– Tricks? Font size/double sided
photocopying/numbering sections
95

96. Maximising Response Rate
• Layout and design is key
• Respondent’s level of interest
• Colour of paper
• Accompanying letter / introduction
• Mail surveys - self-addressed
stamped return envelope
• Rewards
• Reminders or follow up calls96

97. Pre-testing and Pilot testing
• Pre-test – try out on convenient
others & revise
• Pilot test – try out on a small
sample from the target
population & revise
• Be assertive and interactive
about seeking feedback – ask
questions & observe
97

98. Pre-test & Revise
•Pre-test items and ask for feedback
•Revise:
–items which don’t apply to everybody
–redundancy
–skewed response items
–misinterpreted items
–non-completed items
•Reconsider ordering & layout

99. LUNCH BREAK
99

100. Survey Design Critique
In pairs, look through the example
questionnaires, and highlight
aspects which:
• could be improved
• are particularly good
• you would like to ask about
•
100

101. Examples
• Examine questionnaire
examples
• Examine structure, design
issues and question styles
• Note cover page and details
provided
101

102. Sampling
102

103. Sampling:
Overview
Sampling terminology
What is sampling?
Why sample?
Sampling methods
Example: Shere Hite’s survey
103

104. Sampling terminology
• Target population
– To whom you wish to generalise
• Sampling frame
– Those who have a chance to be selected
• Sample
– Those who were selected and responsed
• Representativeness
– The extent to which the sample is a good
104

105. What is sampling?
“Sampling is the process of
selecting units (e.g., people,
organizations) from a
population of interest so that
by studying the sample we
may fairly generalize our
Picture 2
results back to the population
from which they were chosen.”
105

106. Why sample?
• Reduces cost, time, sample
size etc.
• If the sample is representative,
the sample data allows
inferences to be drawn about
the total population.
106

107. Representativeness of a
sample depends on:
• Adequacy of sampling frame
• Sampling method
• Adequacy of sample size
• Response rate – both the % &
representativeness of people in sample
who actually complete survey
It is better to have a small,
representative sample than a large,
107

108. Sampling methods
Probability sampling
• Random
• Systematic
• Cluster
– Multi-Stage Cluster
Non-probability sampling
–Quota
–Convenience
–Snowball 108

109. Random/probability sampling
• Each unit has an equal chance of
selection
• Selection occurs entirely by
random chance
• Also called representative
sampling
109

110. Simple random sampling
• Everyone in the target population
has an equal chance of selection
• Useful if clear study area or
population is identified
• Similar to a lottery:
– List of names are assigned #s and
randomly select #s of respondents
– Randomly select # through table of
random #s or by computer 110

111. Systematic random sampling
• Selecting without first numbering
• Respondents (units) selected from
a list/file.
• Useful when survey population is
similar e.g. List of students
• Select sample at regular intervals
from the population e.g., every 5th
person on a list, starting at a random
number between 1 and 5 111

112. Stratified random sampling
• Sub-divide population into strata
(e.g., by gender, age, or location)
• Then random selection from
within each stratum
• Improves representativeness
• e.g., Telephone interviews using
post-code strata
112

113. Non-random / non-probability
• Also called purposive or
judgemental sampling
• Useful for exploratory research
and case study research
• Able to get large sample size
quickly
• Limitations include potential bias
and non-representativeness
113

114. Convenience sampling
• Sampling is by convenience rather
than randomly
• Due to time/financial constraints
• e.g. surveying all those at a tourist
attraction over one weekend
114

115. Purposive sampling
Respondents selected for a
particular purpose e.g., because
they may be “typical” respondents
• e.g., select sample of tourists aged 40-60
as this is the typical age group of visitors to
Canberra
• e.g., Frequent flyers to contact regarding
service quality in an airline setting
115

116. Snowball sampling
• Useful for difficult to access
populations e.g., illegal
immigratnts, drug users
• Respondents recommend other
respondents
• e.g., in studying ecstasy users, gain trust of
a few potential respondents and ask them
to recommend the researcher to other
potential respondents
116

117. Sampling process
1Identify target population and
sampling frame
2Select sampling method
3Calculate sample size for desired
power.
4Maximise return rate
117

118. Summary of sampling
strategy
•Identify target population and
sampling frame
•Selection sampling method
•Calculate required sample size
•Maximise return rate

119. Sampling Example:
Shere Hite
‘American Sexology’
119

120. Hite's survey of American
male-female relations (early 1980's)
•Shere Hite ‘doyenne of sex polls’
•Media furors & worldwide attention
•127-item questionnaire about
marriage & relations between sexes
•Sample: 4500 USA women, 14 to 85
years
•Conclusion: Society and men need to
change to improve lives of women
120

121. Some of Hite’s findings
about American women....
•Only 13% married for 2+years were still in love
•70% married for 5+ years were having affairs...
– usually more for 'emotional closeness’ than sex
– 76% of these women did not feel guilty
•87% had a closer female friend than husband
•98% wanted “basic changes” to love
relationships
•84% were emotionally unsatisfied
•95% reported emotional & psychological
harassment from their men 121

122. Some of the critical comments....
The survey often
seems merely to
She goes in with provide an
prejudice & occasion
comes for the author’s
out with a own male-bashing
statistic. diatribes.
Hite uses statistics to bolster her
opinion that American women are
justifiably fed up with American
men. 122

123. Hite's response rate &
selection bias
•100,000 questionnaires were sent
to a variety of women’s groups
(feminist organisations, church groups,
garden clubs etc.)
•4,500 replied
(4.5% return rate)
123

124. Hite's response rate &
selection bias
“We get pretty nervous if respondents
in our survey go under 70%.
Respondents to surveys differ from
nonrespondents in one important
way: they go to the trouble of filling
out what in this case was a very
long, complicated, and personal
questionnaire.”
124

125. Lessons from Hite's male-
female relations survey
1Sample size – it's not how big, it's
how representative
2Objectivity – watch out for
manipulating the survey questions
and results interpretation to suit your
personal conjectures
125

126. Measurement
Error
126

127. Measurement error
•Observed score =
true score + measurement error
•Measurement error =
systematic error + random error
•Any deviation from the true
value caused by the
measurement procedure.
127

128. Sources of measurement error
Picture 5
Non-sampling Sampling
(e.g., unreliable (e.g., non-rep. samp
or invalid
tests)
Personal bias
(e.g., researcher f
Paradigm
(e.g., Western focus on individualism)
128

129. To minimise measurement error
Use well designed measures:
• Multiple indicators
• Sensitive to target constructs
• Clear wording on
questions/instructions
129

130. To minimise measurement error
Reduce demand effects:
• Train interviewers
• Use standard protocol
130

131. To minimise measurement error
Maximise response rate:
• Pre-survey contact
• Minimise length / time / hassle
• Offer rewards / incentives
• Coloured paper
• Call backs / reminders
131

132. To minimise measurement error
Ensure administrative accuracy
• Set up efficient coding, with well-
labelled variables
• Check data
132

133. Summary of sampling
strategy
•Identify target population and
sampling frame
•Selection sampling method
•Calculate required sample size
•Maximise return rate

134. Sampling Task
A research project's aim is –
“To identify the behaviour and
attitudes of UC students with
regard to its computing services”.
• What is the research population?
• How might you get hold of a
sample frame?
• What sampling technique would
134

135. Confidence Levels
/ Margins of Error
• Relates to representativeness of a
sample of a target population - to
what extent can we be confident
about the results?
• Gives the estimated range of
values into which we expect other
samples to fall say 95% of the time
• Social sciences = at least 90% or
above, preferably 95% 135

136. Example 95%
Confidence Level Graphs
136

137. Example 95%
Confidence Level Table
137

138. Confidence Level Example
Survey of visitors to Canberra between
June 1 - August 31, 2004 = 10,000
visitors
Want to work with 95% confidence
level and 3% margin of error
How many to survey?
Answer: Sample size = 964 people
Q: What are your favourite attractions?
138

139. Following Results
+ / - 3% error @ 95% confidence level
139

140. Confidence Intervals /
Margins of Error
• One time out of 20, we expect the
answers may be greater than +/- 3%
– Establish the confidence level, margin of
error you want to work with
– Identify the number of surveys to be done
– Once you have completed that number, do
not do any additional surveys
HOWEVER….
140

141. Confidence Intervals /
Margins of Error
• Sometimes we do surveys and we do
not know how many will be returned
until later, as with postal surveys
• Thus you have to calculate the
margin of error afterwards….
– Count up # of returned surveys
– Identify target population and
confidence level
141

142. Confidence Intervals /
Margins of Error
• All this assumes you know your target
population and can get a sample
frame
– If not, a non-random sampling
technique is best
• Also consider whether you want to
report results for sub-groups – the
margins of error will be wider
142

143. Summary - 1
1 Survey research has developed into
a popular research method since
the 1940's.
2 A survey is a standardised stimulus
designed to convert fuzzy
psychological phenomenon into
hard data.
143

144. Summary - 2
3 Survey development - types of
questions and response formats.
4 Sampling - probability & non-prob.
5 Levels of measurement &
parametric / non-parametric stats
6 Ethical considerations
7 Sources of measurement error
144

145. Evaluation
• Please complete the workshop
evaluations
145

146. References
Alreck, P. & Settle, R. (1995). The survey
research handbook (2nd ed.). New York:
Irwin.
Stevens, S.S. (1946). On the theory of
scales of measurement. Science, 103,
677-680.
Trochim, W. M. K. (2006). Sampling. In
Research Methods Knowledge Base.
Wikipedia (2009).
Shere Hite - Methodology.
• 146

147. Open Office Impress
●
This presentation was made using
Open Office Impress.
●
Free and open source software.
http://www.openoffice.org/product/impress.html
●