Designing Questionnaire 
and Survey Research. A series of six presentation, introduce scientific research in the areas of cross-cultural, using quantitative approach.

1. Hangzhou, January 2011
Prof. Dr. Hora Tjitra, Zhejiang University
Designing
Questionnaire
and
Survey
Research
Quantitative Research Methodologies (4/6)

2. Designing
Ques*onnaire
and
Survey
Research
2
Designing Questionnaire and Survey Research
Survey
research:
a
specific
type
of
field
study
that
involves
the
collec3on
of
data
from
a
sample
of
elements
drawn
from
a
well-­‐defined
popula3on
through
the
use
of
a
ques3onnaire.
The reasons to conduct survey research…
• Study
a
representa-ve
sample
through
field
research
is
rela-vely
easy
and
prac-cal.
• Research
can
confidently
generalize
the
finding
from
selected
sample
to
en-re
popula-on.
• It
provides
ideal
condi-ons
for
explora-on
of
process
×
individual
difference
interac-on.

3. Designing
Ques*onnaire
and
Survey
Research
3
14
years
in
Germany
7
years
in
China
Born
and
grew
up
in
Indonesia
Prof.Dr.Hora Tjitra - Cross-cultural and Business Psychology
Dipl.-Psych.,Technical University of Braunschweig
Organizational Psychology and Human Resource Management
Dr.Phil.,University of Regensburg
Intercultural Psychology and Strategic Management
Executive Education,INSEAD
HR Management in Asia

4. Designing
Ques*onnaire
and
Survey
Research
Outline
1 Doing Survey Research 4
2 Designing Questionnaire 13
3 Sampling Design in the Survey Research 19
4

5. Doing
Survey
Research

6. Designing
Ques*onnaire
and
Survey
Research
6
Types of Surveys
…the
broad
area
of
survey
research
encompasses
any
measurement
procedures
that
involve
asking
ques3ons
of
respondents.
Ques%onnaires:
Ques-onnaires
are
usually
paper-­‐and-­‐pencil
instruments
that
the
respondent
completes.
Interviews
(constructed):
Interviews
are
completed
by
the
interviewer
based
on
the
respondent
says.

7. Designing
Ques*onnaire
and
Survey
Research
7
The Issues in Survey Research
Question Issues
✓ What types of questions can be asked?
✓ How complex will the questions be?
✓ Will screening questions be needed?
✓ Can question sequence be controlled?
✓ Will lengthy questions be asked?
✓ Will long response scales be used?
Population Issues
✓ Can the population be enumerated?
✓ Is the population literate?
✓ Are there language issues?
✓ Will the population cooperate?
✓ What are the geographic restrictions?
Bias Issues
✓ Can social desirability be avoided?
✓ Can interviewer distortion and subversion be
controlled?
✓ Can false respondents be avoided?
Sampling Issues
✓ What data is available?
✓ Can respondents be found?
✓ Who is the respondent?
✓ Can all members of population be sampled?
✓ Are response rates likely to be a problem?
Content Issues
✓ Can the respondents be expected to know
about the issue?
✓ Will respondent need to consult records?
Administrative Issues
✓ Costs & Facilities
✓ Time & Personnel

8. Designing
Ques*onnaire
and
Survey
Research
8
Three Types of Measurement
Scaling
is
the
assignment
of
objects
to
numbers
according
to
a
rule.
Likert or "Summative" scaling there are a variety possible response scales
(1-to-7, 1-to-9, 0-to-4).
Guttman or “Cumulative” scaling the answering item is “Yes” or “No”
Thurstone or Equal-
appearing interval scaling
rate each statement on a 1-to-11 scale in
terms, then choose “yes” or “No”

9. Designing
Ques*onnaire
and
Survey
Research
9
The Basic Steps in Developing a Likert or "Summative" Scale.
Defining the Focus: define
what
it
is
you
are
trying
to
measure
Administering the Scale: reversal
items
1=
strongly
disagree
2=
disagree
3=
undecided
4=
agree
5=
strongly
agree
Generating the Items: create
the
set
of
poten-al
scale
items.
These
should
be
items
that
can
be
rated
on
a
1-­‐to-­‐5
or
1-­‐to-­‐7
Disagree-­‐Agree
response
scale.
Rating the Items:
have
a
group
of
judges
rate
the
items.
Usually
you
would
use
a
1-­‐to-­‐5
ra-ng
scale
where:1=
strongly
unfavorable
to
the
concept
;
5=
strongly
favorable
to
the
concept
Selecting the Items.
compute
the
intercorrela-ons
between
all
pairs
of
items,
based
on
the
ra-ngs
of
the
judges.
•Throw
out
any
items
that
have
a
low
correla<on
with
the
total
(summed)
score
across
all
items
•
For
each
item,
get
the
average
ra<ng
for
the
top
quarter
of
judges
and
the
boFom
quarter.
Then,
do
a
t-­‐test
of
the
differences
between
the
mean
value
for
the
item
for
the
top
and
boFom
quarter
judges.

10. Designing
Ques*onnaire
and
Survey
Research
10
Example: The Employment Self Esteem Scale
INSTRUCTIONS: Please rate how strongly you agree or disagree with each of the following statements
by placing a check mark in the appropriate box.
Strongly Disagree
Somewhat
Disagree
Somewhat Agree Strongly Agree 1. I feel good about my work on the job.
Strongly Disagree
Somewhat
Disagree
Somewhat Agree Strongly Agree 2. On the whole, I get along well with others at work.
Strongly Disagree
Somewhat
Disagree
Somewhat Agree Strongly Agree 3. I am proud of my ability to cope with difficulties at work.
Strongly Disagree
Somewhat
Disagree
Somewhat Agree Strongly Agree 4. When I feel uncomfortable at work, I know how to handle it.
Strongly Disagree
Somewhat
Disagree
Somewhat Agree Strongly Agree 5. I can tell that other people at work are glad to have me there.
Strongly Disagree
Somewhat
Disagree
Somewhat Agree Strongly Agree 6. I know I'll be able to cope with work for as long as I want.
Strongly Disagree
Somewhat
Disagree
Somewhat Agree Strongly Agree 7. I am proud of my relationship with my supervisor at work.
Strongly Disagree
Somewhat
Disagree
Somewhat Agree Strongly Agree
8. I am confident that I can handle my job without constant
assistance.
Strongly Disagree
Somewhat
Disagree
Somewhat Agree Strongly Agree 9. I feel like I make a useful contribution at work.
Strongly Disagree
Somewhat
Disagree
Somewhat Agree Strongly Agree 10. I can tell that my coworkers respect me.

11. Designing
Ques*onnaire
and
Survey
Research
11
Survey-Research Design
Cross-Sectional Design
the collection of data at a single point in time from a
sample drawn from a specified population
Successive Independent Samples Design
a series of cross-sectional survey are conducted over
time
Longitudinal Design
the same sample of respondents is surveyed more than
once

12. Designing
Ques*onnaire
and
Survey
Research
12
Total survey error
The ultimate goal of survey research is to accurately measure particular
constructs within a sample of people who represent the population of
interest
• Coverage Error: the bias that can result when the pool of potential survey
participants from which a sample is selected does not include some portions of the
interest
• Sampling Error: the random differences that invariably exist between any
sample and the population from which it was selected
• Nonresponse Error: the bias that can result when data are not collected from
all of the members of a sample
• Measurement Error: all distortions in the assessment of the construct of
interest, including systematic biases and random variance that can be brought by
respondents’ own behavior, interviewer behavior, and the questionnaire

13. Designing
Questionnaire

14. Designing
Ques*onnaire
and
Survey
Research
14
Questionnaire Design:
The six steps for preparing the questionnaire
1.
Decide
what
informa0on
should
be
sought
2.
Decide
what
type
of
ques0onnaire
should
be
used
3.
Write
a
first
dra;
of
the
ques0onnaire
4.
Reexamine
and
revise
the
ques0onnaire
5.
Pretest
the
ques0onnaire
6.
Edit
the
ques0onnaire
and
specify
the
procedure

15. Designing
Ques*onnaire
and
Survey
Research
15
Question Content
• Is the Question Necessary/
Useful?
♪ Do you need the age of each child or just the
number of children under 16?
♪ Do you need to ask income or can you
estimate?
• Are Several Questions
Needed?
♪ double-barreled question (eg. Teacher & parents)
♪ does not cover all possibilities
♪ does not give you enough context
♪ does not determine the intensity
• Do Respondents Have the
Needed Information?
♪ You should ask a filter question first (e.g., Have you
ever watched the show ER?) before asking them
their opinions about it.
• Does the Question Need to be
More Specific?
♪ How well did you like the book?
♪ Did you recommend the book to others?
• Is Question Sufficiently General?
• Is Question Biased or Loaded?
♪e.g. What do you see as the benefits of a tax cut?
♪e.g. What do you see as the disadvantages of eliminating
welfare?
• Will Respondent Answer
Truthfully?

16. Designing
Ques*onnaire
and
Survey
Research
16
Question Words
♪
Can the Question be Misunderstood? e.g. What
kind
of
headache
remedy
do
you
use?
One
of
the
major
difficulty
in
wri-ng
good
survey
ques-ons
is
geRng
the
wording
right
♪
Is the time frame specified? Whenever
using
the
words
"will",
"could",
"might",
or
"may"
in
a
ques<on,
you
might
suspect
that
the
ques<on
asks
a
<me-­‐related
ques<on.
♪
What Assumptions Does the Question Make?
♪
Is the wording too direct? E.g. How did you feel about being in the war?
How well did the equipment hold up in the field?
How
well
were
new
recruits
trained?
♪
Does the question contain difficult or unclear terminology?…..

17. Designing
Ques*onnaire
and
Survey
Research
17
Question Placement
Considering following questions:
♪
Is
the
answer
influenced
by
prior
ques<ons?
♪
Does
ques<on
come
too
early
or
too
late
to
arouse
interest?
♪
Does
the
ques<on
receive
sufficient
aFen<on?
A Checklist of Considerations
✓ start
with
easy,
nonthreatening
ques<ons
✓ put
more
difficult,
threatening
ques<ons
near
end
✓ never
start
a
mail
survey
with
an
open-­‐ended
ques<on
✓ for
historical
demographics,
follow
chronological
order
✓ ask
about
one
topic
at
a
<me
✓ when
switching
topics,
use
a
transi<on
✓ reduce
response
set
(the
tendency
of
respondent
to
just
keep
checking
the
same
response)
✓ for
filter
or
con<ngency
ques<ons,
make
a
flowchart
The
Golden
Rule
Do
unto
your
respondents
as
you
would
have
them
do
unto
you!

18. Designing
Ques*onnaire
and
Survey
Research
18
Other Important Issues
Question Order
✓ Grouping Questions by topic may be useful
✓ It is easier for the respondents to think in the same topic
❖ For bipolar scales ( e.g., running from positive to
negative with neutral in the middle), reliability and
validity are highest for about 7 point.( Matell & Jacoby,
1971)
❖ The reliability and validity of unipolar scales (e.g.,
running from no importance to very high importance)
seem to be optimized for a bit shorter scales,
approximately 5 points lon (Wikman & Waarneryd,
1990)
❖ A good number of studies suggest that data quality is
better when all scales points are labeled with words
than only some are. ( Krosnick & Berent, 1993)
Ra#ng
Scale
Formats

19. Sampling
Design
in
the
Survey
Research

20. Designing
Ques*onnaire
and
Survey
Research
20
Sampling in Survey Research

21. Designing
Ques*onnaire
and
Survey
Research
21
Statistical Sampling Terms
Variable:
is a specific measurement value
that a sampling unit supplies
Statistic:
the
responses
that
we
get
for
our
en<re
sample
Parameter:
the measurement the entire population

22. Designing
Ques*onnaire
and
Survey
Research
22
Different Approaches to Sampling Methods
Probability Sampling Non-Probability Sampling
Any method of sampling that utilizes
some form of random selection
➡ Simple Random Sampling
➡ Stratified Random Sampling
➡ Systematic Random Sampling
➡ Cluster (Area) Random Sampling
Methods do not involve random selection
➡ Convenience Sampling
➡ Purposive Sampling

23. Designing
Ques*onnaire
and
Survey
Research
23
Probability Sampling
any method of sampling that utilizes some form of random selection
Some Definitions
✓ N = the number of cases in the sampling frame
✓ n = the number of cases in the sample
✓ NCn = the number of combinations (subsets)
of n from N
✓ f = n/N = the sampling fraction

24. Designing
Ques*onnaire
and
Survey
Research
24
Simple Random Sampling
Objective:
To
select
n
units
out
of
N
such
that
each
NCn
has
an
equal
chance
of
being
selected.
Procedure: Use a table of
random numbers, a computer
random number generator, or a
mechanical device to select the
sample.

25. Designing
Ques*onnaire
and
Survey
Research
25
Stratified Random Sampling
Objective:
Divide the population into non-
overlapping groups (i.e., strata) N1,
N2, N3, ... Ni, such that N1 + N2 +
N3 + ... + Ni = N. Then do a simple
random sample of f = n/N in each
strata.
Procedure:
Divide your population into
homogeneous subgroups and then
taking a simple random sample in
each subgroup.

26. Designing
Ques*onnaire
and
Survey
Research
26
Systematic Random Sampling
• number the units in the
population from 1 to N
• decide on the n (sample size)
that you want or need
• k = N/n = the interval size
• randomly select an integer
between 1 to k
• then take every kth unit

27. Designing
Ques*onnaire
and
Survey
Research
27
Cluster (Area) Random Sampling
• Divide population into clusters
(usually along geographic
boundaries)
• Randomly sample clusters
• Measure all units within
sampled clusters
Beijing
Jinan
Nanjing
Shanghai
Hangzhou
Changsha
Shenzhen
Chongqing
Tianjin


ShijiaZhuang
(Hebei)
Zhengzhou
(Henan)
 Qingdao

Hefei
(Anhui)
 Suzhou
Chengdu

 Shantou
 Xiamen
Wuhan
Xi’an
(Shaanxi)
 Zhanjiang
Guiyang


Kunming
Eg. Measuring Chinese Personality

28. Designing
Ques*onnaire
and
Survey
Research
28
Non-Probability Sampling
does
not
involve
random
selec7on
• Modal
Instance
Sampling
• Expert
Sampling
• Quota
Sampling
• Snowball
Sampling
Purposive Sampling:
sample
with
a
purpose
in
mind.
or called Accidental,
Haphazard sampling
they are not representative of the
populations
Convenience Sampling:
tradi0onal
"man
on
the
street"

29. Designing
Ques*onnaire
and
Survey
Research
29
Advantage and Disadvantage of Sampling Approach
Probability Sampling
Non-Probability Sampling
Advantage
 Research can be confident that a
selected is representative
 It permits researchers to precisely
estimate the amount of variance
present in a given data set that is
due to sampling error
Disadvantage
✓ Calculation
Advantage
 Convenient
 Flexible
Disadvantage
✓ We may or may not represent
the population well,
✓ It will often be hard for us to
know how well we've done so

30. Thank
You
Contact us via …
Mail: hora_t@mac.com
Follow: twitter@htjitra
Website: http://horatjitra.com
Zhejiang
University,
Hangzhou
(China)