The document describes various research methods being used by different companies and organizations. Some key points:
- A food manufacturer is conducting research to understand the demographics of organic food purchasers.
- British Airways is testing in-flight internet services on one of its flights from New York to Tokyo, charging different prices ($30 one week, $15 the next) to test consumer response.
- Cosmopolitan magazine is sending different magazine covers (one with a female model, one with female and male models) to different reader groups to test differences in purchase response.
4. The goal of this research is to discover the real nature of the problem & to suggest new
possible solutions or new ideas.
A food manufacturer wants to know the demographics of people who purchase organic foods.
A firm is considering hiring American celebrity Paris Hilton to endorse its products.
British Airways would like to test in-flight Internet services on one of its regular flights from New
York to Tokyo. The company charges $30 one week and $15 the next week.
This type of study attempts to discover answers to the following questions: who, what, when,
where, or how much.
A manufacturer investigates whether consumers will buy a new pill that replaces eating a meal.
Cosmopolitan magazine sends out a cover in selected markets featuring a female model to
half of its readers and a cover with a female and male model to the other half of its readers to
test differences in purchase response between the two groups.
A hair-care manufacturer interviews wholesalers, retailers, and customers to determine the
potential for a new shampoo package.
This type of research attempts to capture a population’s characteristics by making inference
from a sample’s characteristics and testing hypotheses.
Descriptive
On the CBS television show Undercover Boss, top executives disguised as middle level or
lower
7. 11-7
Learning Objectives
Understand . . .
• The distinction between measuring objects, properties, and
indicants of properties.
• The similarities and differences between the four scale types used in
measurement and when each is used.
• The four major sources of measurement error.
• The criteria for evaluating good measurement.
8. 11-8
Measurements Will Vary Over Time
“The only man who behaved sensibly was my tailor; he took my
measurement anew every time he saw me, while all the rest
went on with their old measurements and expected them to fit me.”
George Bernard Shaw
playwright and essayist
10. Measurement in research consists of:
• assigning numbers to empirical events, objects or properties, or
activities in compliance w/ a set of rules.
• Textbook uses an example of auto show attendance.
• A mapping rule is a scheme for assigning numbers to aspects of an
empirical event.
13. Levels of Measurement
Ordinal
interval
Ratio
Nominal Classification
Order ( > or < )
Classification
• Order means that the numbers are ordered. One number is
greater than, less than, or equal to another number.
E.g., Pizza Hut is better than Papa Johns, ranking
15. 11-15
Ordinal Scales
• Ordinal data require conformity to a
logical postulate, which states:
If a is greater than b, and
b is greater than c, then
a is greater than c.
• The appropriate measure of central
tendency is the median. The median is
the midpoint of a distribution. A
percentile or quartile reveals the
dispersion.
18. Ratio Scales
11-18
Examples
Weight
Height
Number of children
• Ratio data : actual amounts of a variable.
• E.g., monetary values, population counts, distances, return rates, and
amounts of time.
• Central tendency and coefficients of variation may also be calculated.
• Higher levels of measurement generally yield more information and are
appropriate for more powerful statistical procedures.
19. 11-19
Sources of Error
1. Respondents may also suffer from temporary factors like fatigue and
boredom.
2. Any condition that places a strain on the interview
3. The interviewer can distort responses by rewording, paraphrasing, or
reordering questions.
• Stereotypes in appearance and action also introduce bias.
• Careless mechanical processing will distort findings and can also
introduce problems in the data analysis stage through incorrect
coding, careless tabulation, and faulty statistical calculation.
4. A defective instrument
• confusing and ambiguous.
• not explore all the potentially important issues.
20. 11-20
Evaluating Measurement Tools
Criteria
Validity
Practicality Reliability
• Validity is the extent to which a test measures what we actually wish to
measure.
• Reliability refers to the accuracy and precision of a measurement
procedure.
• Practicality is concerned with a wide range of factors of economy,
convenience, and interpretability.
33. 12-33
Learning Objectives
Understand…
• The nature of attitudes and their relationship to behavior.
• The critical decisions involved in selecting an appropriate
measurement scale.
• The characteristics and use of rating, ranking, sorting,
and other preference scales.
34. 12-34
Measurements are Relative
“Any measurement must take into account the
position of the observer. There is no such thing
as measurement absolute, there is only
measurement relative.”
Jeanette Winterson
journalist and author
37. 12-37
Nature of Attitudes
Cognitive
I think oatmeal is healthier
than corn flakes for breakfast.
Affective
Behavioral
I hate corn flakes.
I intend to eat more oatmeal
for breakfast.
“All survey questions must be actionable if you want
results.”
Frank Schmidt, senior scientist
The Gallup Organization
39. Improving Predictability of Attitudes
Reference
groups
Multiple
measures
Factors
Strong
Specific
Basis
Direct
Factors the applicability of attitudinal research for business.
1. Specific attitudes are better predictors of behavior
2. Strong attitudes are better predictors of behavior composed of little intensity or
topic interest.
3. Direct experiences with the attitude object produce behavior more reliably.
4. Cognitive-based attitudes influence behaviors better than affective-based
attitudes.
41. Selecting a Measurement Scale
Research objectives Response types
Data properties
Number of
dimensions
Forced or unforced
choices
Balanced or
unbalanced
Rater errors
Number of
scale points
Attitude scaling: process of assessing an attitudinal disposition using a number
that represents a person’s score on an attitudinal continuum ranging from an
extremely favorable disposition to an extremely unfavorable one.
44. 12-44
Balanced or Unbalanced
Very bad
Bad
Neither good nor bad
Good
Very good
Poor
Fair
Good
Very good
Excellent
How good an actress is Angelina Jolie?
45. 12-45
Forced or Unforced Choices
Very bad
Bad
Neither good nor bad
Good
Very good
Very bad
Bad
Neither good nor bad
Good
Very good
No opinion
Don’t know
How good an actress is Angelina Jolie?
46. Number of Scale Points
Very bad
Bad
Neither good nor bad
Good
Very good
Very bad
Somewhat bad
A little bad
Neither good nor bad
A little good
Somewhat good
Very good
How good an actress is Angelina Jolie?
47. Rater Errors
Error of
central tendency
Error of leniency
•Adjust strength of
descriptive adjectives
•Space intermediate
descriptive phrases
farther apart
•Provide smaller
differences
in meaning between
terms near the
ends of the scale
•Use more scale points
49. Rater Errors
Halo Effect
• Rate one trait
at a time
• Reveal one trait
per page
• Reverse anchors
periodically
• The halo effect is the systematic bias that the rater introduces
by carrying over a generalized impression of the subject from
one rating to another.
e.g., a teacher may expect that a student who did well on the
first exam to do well on the second.
52. 12-52
Multiple-Choice, Multiple-Response
Scale
What sources did you use when designing your new
home? Please check all that apply.
Online planning services
Magazines
Independent contractor/builder
Designer
Architect
Other (specify:_____________)
53. 12-53
Likert Scale
The Internet is superior to traditional libraries for
comprehensive searches.
Strongly disagree
Disagree
Neither agree nor disagree
Agree
Strongly agree
54. Semantic Differential
• studies of brand and institutional image, employee morale, safety,
financial soundness, trust, etc.
• usually with 7 points, by which one or more participants rate one or
more concepts on each scale item.
• Proposition: an object can have several dimensional meaning located
in multidimensional property space, called semantic space.
55. Adapting SD Scales
Convenience of Reaching the Store from Your Location
Nearby ___: ___: ___: ___: ___: ___: ___: Distant
Short time required to reach store ___: ___: ___: ___: ___: ___: ___: Long time required to reach store
Difficult drive ___: ___: ___: ___: ___: ___: ___: Easy Drive
Difficult to find parking place ___: ___: ___: ___: ___: ___: ___: Easy to find parking place
Convenient to other stores I shop ___: ___: ___: ___: ___: ___: ___: Inconvenient to other stores I shop
Products offered
Wide selection of different
kinds of products ___: ___: ___: ___: ___: ___: ___:
Limited selection of different
kinds of products
Fully stocked ___: ___: ___: ___: ___: ___: ___: Understocked
Undependable products ___: ___: ___: ___: ___: ___: ___: Dependable products
High quality ___: ___: ___: ___: ___: ___: ___: Low quality
Numerous brands ___: ___: ___: ___: ___: ___: ___: Few brands
Unknown brands ___: ___: ___: ___: ___: ___: ___: Well-known brands
58. Numerical Scale
• Numerical scales have equal intervals that separate their numeric
scale points. The verbal anchors serve as the labels for the extreme
points.
• Numerical scales are often 5-point scales but may have 7 or 10
points.
• The participants write a number from the scale next to each item.
• It produces either ordinal or interval data.
59. Multiple Rating List
Scales
“Please indicate how important or unimportant each service characteristic is:”
IMPORTANT UNIMPORTANT
Fast, reliable repair 7 6 5 4 3 2 1
Service at my location 7 6 5 4 3 2 1
Maintenance by manufacturer 7 6 5 4 3 2 1
Knowledgeable technicians 7 6 5 4 3 2 1
Notification of upgrades 7 6 5 4 3 2 1
Service contract after warranty 7 6 5 4 3 2 1
Exhibit 12-3: A multiple rating scale is similar to the numerical
scale but differs in 2 ways:
1) it accepts a circled response from the rater, and
2) the layout facilitates visualization of the results.
• This scale produces interval data.
60. • Used as an alternative to the semantic differential, especially when it
is difficult to find bipolar adjectives that match the investigative
question.
• interval data.
Stapel Scales: 3 attributes of corporate image.
61. Constant-Sum
Scales
• The participant allocates points to more than one attribute or property indicant,
such that they total a constant sum, usually 100 or 10.
• Participant precision and patience suffer when too many stimuli are
proportioned and summed.
• A participant’s ability to add may also be taxed.
• Its advantage is its compatibility with percent and the fact that alternatives that
are perceived to be equal can be so scored.
• This scale produces interval data.
65. Forced Ranking
Scale
• This method is faster than paired comparisons and is usually easier and
more motivating to the participant.
• A drawback of this scale is the limited number of stimuli (usually no
more than 7) that can be handed by the participant.
• This scale produces ordinal data.
68. 12-68
MindWriter Scaling
Likert Scale
The problem that prompted service/repair was resolved
Strongly
Disagree Disagree
Neither Agree
Nor Disagree Agree
Strongly
Agree
1 2 3 4 5
Numerical Scale (MindWriter’s Favorite)
To what extent are you satisfied that the problem that prompted service/repair was
resolved?
Very
Dissatisfied
Very
Satisfied
1 2 3 4 5
Hybrid Expectation Scale
Resolution of the problem that prompted service/repair.
Met Few
Expectations
Met Some
Expectations
Met Most
Expectations
Met All
Expectations
Exceeded
Expectations
1 2 3 4 5
69. 12-69
Ideal Scalogram Pattern (social distance, organizational
hierarchies, and evolutionary product stages)
Item
Participant
Score
2 4 1 3
X X X X 4
__ X X X 3
__ __ X X 2
__ __ __ X 1
__ __ __ __ 0
* X = agree; __ = disagree.
73. 13-73
Learning Objectives
Understand...
• The link forged between the management dilemma and the
communication instrument by the management-research question
hierarchy.
• The influence of the communication method on instrument design.
• The three general classes of information and what each contributes
to the instrument.
74. 13-74
Learning Objectives
Understand . . .
• The influence of question content, question wording,
response strategy, and preliminary analysis planning on
question construction.
• Each of the numerous question design issues influencing
instrument quality, reliability, and validity.
• The sources for measurement questions
• The importance of pretesting questions and instruments.
75. 13-75
Measurement Skepticism
“Research that asks consumers what they did and why is
incredibly helpful. Research that asks consumers what
they are going to do can often be taken with a grain of
salt.”
Al Ries
author, co-founder, and chairman
Ries & Ries.
79. Strategic Concerns in Instrument Design
What type of scale is needed?
What communication approach will be used?
Should the questions be structured?
Should the questioning be disguised?
82. 13-82
Dummy Table for American Eating Habits
Age
Use of Convenience Foods
Always
Use
Use
Frequently
Use
Sometimes Rarely Use Never Use
18-24
25-34
35-44
55-64
65+
84. Question Categories and Structure
Administrative Target Classification
3 categories of measurement questions.
1. Administrative questions identify the participant, interviewer, interviewer
location, and conditions. These questions are rarely asked of the participant
but are necessary for studying patterns within the data and identify possible
error sources.
2. Classification questions usually cover sociological-demographic variables that
allow participants’ answers to be grouped so that patterns are revealed and
can be studied. These questions usually appear at the end of a survey.
3. Target questions address the investigative questions of a specific study.
These are grouped by topic in the survey. Target questions may be structured
or unstructured.
85. 13-85
Engagement = Convenience
“Participants are becoming more and more
aware of the value of their time. The key to
maintaining a quality dialog with them is to
make it really convenient for them to
engage, whenever and wherever they want.”
Tom Anderson
managing partner
Anderson Analytics
86. 13-86
Question Content
Should this question be asked?
Is the question of proper scope and coverage?
Can the participant adequately
answer this question as asked?
Will the participant willingly
answer this question as asked?
87. Criteria of Question Wording
Criteria
Shared
vocabulary Single
meaning
Misleading
assumptions
Adequate
alternatives
Personalized
Biased
1. Is the question stated in terms of a shared vocabulary?
2. Does the question contain vocabulary with a single meaning?
3. Does the question contain unsupported or misleading
assumptions?
4. Does the question contain biased wording?
5. Is the question correctly personalized?
6. Are adequate alternatives presented within the question?
88. Response Strategy
Factors
Objectives
of the study
Participant’s level
of information
Degree to which participants
have thought through topic
Ease and clarity with which
participant communicates
Participant’s
motivation to
share
In choosing response options in questions, researchers must consider
these factors.
89. 13-89
Free-Response Strategy - open-ended questions
What factors influenced your enrollment in Metro U?
____________________________________________
____________________________________________
90. Dichotomous Response Strategy
Did you attend the “A Day at College”
program at Metro U?
Yes
No
Which one of the following factors was
most influential
in your decision to attend Metro U?
Good academic standing
Specific program of study desired
Enjoyable campus life
Many friends from home
High quality of faculty
Multiple Choice Response Strategy
91. Checklist Response Strategy
Which of the following factors influenced your decision to enroll in Metro U?
(Check all that apply.)
Tuition cost
Specific program of study desired
Parents’ preferences
Opinion of brother or sister
Many friends from home attend
High quality of faculty
Strongly influential Somewhat Not at all
Good academic reputation
Enjoyable campus life
Many friends
High quality faculty
Semester calendar
92. Ranking
Please rank-order your top three factors from the following list based on their
influence in encouraging you to apply to Metro U. Use 1 to indicate the most
encouraging factor, 2 the next most encouraging factor, etc.
_____ Opportunity to play collegiate sports
_____ Closeness to home
_____ Enjoyable campus life
_____ Good academic reputation
_____ High quality of faculty
93. 13-93
Summary of Scale Types
Type Restrictions Scale
Items
Data Type
Rating Scales
Simple Category
Scale
• Needs mutually exclusive choices One or
more
Nominal
Multiple Choice
Single-Response
Scale
• Needs mutually exclusive choices
• May use exhaustive list or ‘other’
Many Nominal
Multiple Choice
Multiple-Response
Scale
(checklist)
• Needs mutually exclusive choices
• Needs exhaustive list or ‘other’
Many Nominal
Likert Scale • Needs definitive positive or
negative statements with which to
agree/disagree
One or
more
Ordinal
Likert-type Scale •Needs definitive positive or
negative statements with which to
agree/disagree
One or
more
Ordinal
94. 13-94
Summary of Scale Types
Type Restrictions Scale Items Data Type
Rating Scales
Numerical
Scale
Needs concepts with standardized
meanings;
Needs number anchors of the scale or end-
points
Score is a measurement of graphical space
One or many Ordinal or
Interval
Multiple
Rating List
Scale
Needs words that are opposites to anchor
the end-points on the verbal scale
Up to 10 Ordinal
Fixed Sum
Scale
Participant needs ability to calculate total
to some fixed number, often 100.
Two or more Interval or
Ratio
95. Summary of Scale Types
Type Restrictions Scale Items Data Type
Rating Scales
Stapel Scale Needs verbal labels that are
operationally defined or standard.
One or more Ordinal or
Interval
Graphic
Rating Scale
Needs visual images that can be
interpreted as positive or negative
anchors
Score is a measurement of graphical
space from one anchor.
One or more Ordinal
(Interval, or
Ratio)
Ranking Scales
Paired
Comparison
Scale
• Number is controlled by
participant’s stamina and interest.
Up to 10 Ordinal
Forced
Ranking Scale
• Needs mutually exclusive choices. Up to 10 Ordinal or
Interval
Comparative
Scale
• Can use verbal or graphical scale. Up to 10 Ordinal
99. Sources of Questions
• Handbook of Marketing Scales
• The Gallup Poll Cumulative
Index
• Measures of Personality and
Social-Psychological Attitudes
• Measures of Political Attitudes
• Index to International Public
Opinion
• Sourcebook of Harris National
Surveys
• Marketing Scales Handbook
• American Social Attitudes Data
Sourcebook
101. 13-101
Guidelines for Question Sequencing
Interesting topics early
Simple topics early
Sensitive questions later
Classification questions later
Transition between topics
Reference changes limited
102. 13-102
Illustrating the Funnel Approach
1. How do you think this country is getting along in its relations with
other countries?
2. How do you think we are doing in our relations with Iran?
3. Do you think we ought to be dealing with Iran differently than we
are now? (If yes) What should we be doing differently?
4. Some people say we should get tougher with Iran and others think
we are too tough as it is; how do you feel about it?
109. 14-109
Small Samples Can Enlighten
“The proof of the pudding is in the eating.
By a small sample we may judge of the
whole piece.”
Miguel de Cervantes Saavedra
author
114. 14-114
What Is a Sufficiently Large Sample?
“In recent Gallup ‘Poll on polls,’ . . . When asked about the
scientific sampling foundation on which polls are based . .
. most said that a survey of 1,500 – 2,000 respondents—a
larger than average sample size for national polls—cannot
represent the views of all Americans.”
Frank Newport
The Gallup Poll editor in chief
The Gallup Organization
118. 14-118
Types of Sampling Designs
Element Selection Probability Nonprobability
Unrestricted Simple random Convenience
Restricted Complex random Purposive
Systematic Judgment
Cluster Quota
Stratified Snowball
Double
119. 14-119
Steps in Sampling Design
What is the target population?
What are the parameters of interest?
What is the sampling frame?
What is the appropriate sampling
method?
What size sample is needed?
120. 14-120
When to Use Larger Sample?
Desired
precision
Number of
subgroups
Confidence
level
Population
variance
Small error
range
121. Simple Random
Advantages
• Easy to implement with
random dialing
Disadvantages
• Requires list of population
elements
• Time consuming
• Larger sample needed
• Produces larger errors
• High cost
Systematic
Advantages
• Simple to design
• Easier than simple random
• Easy to determine sampling
distribution of mean or proportion
Disadvantages
• Periodicity within population
may skew sample and results
• Trends in list may bias results
• Moderate cost
122. 14-122
Stratified
Advantages
• Control of sample size in strata
• Increased statistical efficiency
• Provides data to represent and
analyze subgroups
• Enables use of different
methods in strata
Disadvantages
• Increased error if subgroups
are selected at different rates
• Especially expensive if strata
on population must be created
• High cost
123. 14-123
Cluster
Advantages
• Provides an unbiased estimate
of population parameters if
properly done
• Economically more efficient
than simple random
• Lowest cost per sample
• Easy to do without list
Disadvantages
• Often lower statistical efficiency
due to subgroups being
homogeneous rather than
heterogeneous
• Moderate cost
124. 14-124
Stratified and Cluster Sampling
Stratified
• Population divided into few
subgroups
• Homogeneity within subgroups
• Heterogeneity between
subgroups
• Choice of elements from within
each subgroup
Cluster
• Population divided into many
subgroups
• Heterogeneity within subgroups
• Homogeneity between
subgroups
• Random choice of subgroups
126. 14-126
Double Sampling
Advantages
• May reduce costs if first stage
results in enough data to
stratify or cluster the population
Disadvantages
• Increased costs if
discriminately used
136. 14-136
Estimates of Dining Visits
Confidence Z score % of Area Interval Range
(visits per month)
68% 1.00 68.27 9.48-10.52
90% 1.65 90.10 9.14-10.86
95% 1.96 95.00 8.98-11.02
99% 3.00 99.73 8.44-11.56
137. 14-137
Calculating Sample Size for Questions involving
Means
Precision
Confidence level
Size of interval estimate
Population Dispersion
Need for FPA
138. 14-138
Metro U Sample Size for Means
Steps Information
Desired confidence level 95% (z = 1.96)
Size of the interval estimate .5 meals per month
Expected range in population 0 to 30 meals
Sample mean 10
Standard deviation 4.1
Need for finite population
adjustment
No
Standard error of the mean .5/1.96 = .255
Sample size (4.1)2/ (.255)2 = 259
139. 14-139
Proxies of the Population Dispersion
• Previous research on the topic
• Pilot test or pretest
• Rule-of-thumb calculation
– 1/6 of the range
140. 14-140
Metro U Sample Size for Proportions
Steps Information
Desired confidence level 95% (z = 1.96)
Size of the interval estimate .10 (10%)
Expected range in population 0 to 100%
Sample proportion with given attribute 30%
Sample dispersion Pq = .30(1-.30) = .21
Finite population adjustment No
Standard error of the proportion .10/1.96 = .051
Sample size .21/ (.051)2 = 81