Detailed description of vastly improved Brand Imagery measurement technique

1. MACROConsulting, Inc.

2. Brand Imagery
MEASUREMENT
A New Approach
Paul Richard McCullough
Sawtooth Software Conference 2013
MACROConsulting, Inc.

3. THANK
YOU
www.macroinc.com
 Ricky Odello, Survey Sampling International
 Tom Eagle, Eagle Analytics of California
 KirillZaitsev, MACRO Consulting, Inc.
 Keith Chrzan, Sawtooth Software, Inc.
 Christine Lafontaine, MACRO Consulting, Inc.
MACROConsulting, Inc.
3

4. AGENDA
Brand Imagery Measurement
Introduction
• Current Approach
• Issues
www.macroinc.com
New Approach
Case Study
• Research Objectives
• Research Methodology
• Summary of Findings
MACROConsulting, Inc.
MACROConsulting, Inc.
4

5. AGENDA
Brand Imagery Measurement
Introduction
• Current Approach
• Issues
www.macroinc.com
MACROConsulting, Inc.
MACROConsulting, Inc.
5

6. Introduction
Current Approach
• Rating Scales
• Each brand is rated
independently on
each statement in
an image battery,
eg, 10 point rating
scale.
www.macroinc.com
MACROConsulting, Inc.
6

7. Introduction
Current Approach Issues
1
2
3
4
5
•
•
•
•
Flat Responses Across Statements
Flat Responses Across Brands
Scale Usage Bias
Brand Halo
Ratings Scales
6
7
8
Resulting data are typically
non-discriminating and
highly correlated.
9
10
www.macroinc.com
MACROConsulting, Inc.
7

8. AGENDA
Brand Imagery Measurement
New Approach
www.macroinc.com
MACROConsulting, Inc.
MACROConsulting, Inc.
8

9. New Approach
Brand-Anchored Max/Diff
Brand-anchored Max/Diff removes brand halo,
scale-usage bias andis more discriminating than rating scales.
www.macroinc.com
MACROConsulting, Inc.
9

10. New Approach
Brand-Anchored Max/Diff with Dual Response
Dual Response Max/Diff allows for a zero point in Max/Diff utilities,
making comparisons across studies (and brands) feasible.
www.macroinc.com
MACROConsulting, Inc.
10

11. New Approach
Modified Brand-Anchored Max/Diff
Max/Diff takes longer than ratings scales. Modified brand-anchored Max/Diff
hopes to decrease the interview time of the Max/Diff Tasks.
www.macroinc.com
MACROConsulting, Inc.
11

12. New Approach
Animated Modified Brand-Anchored Max/Diff
Animated Modified Brand-Anchored Max/Diff hopes
to hold the respondent’s attention longer than traditional Max/Diff.
www.macroinc.com
MACROConsulting, Inc.
12

13. New Approach
Direct Binary Response- Positive DBR
Dual Response Max/Diff ALLOWS FOR A
ZERO POINTin Max/Diff utilities, MAKING
COMPARISONS ACROSS STUDIES
feasible.
Direct Binary Response is a MORE TIME-EFFICIENT way
to collect dual-response data.
However, Dual Response Max/Diff has been shown to
RE-INTRODUCE SOME SCALE USAGE BIAS.
www.macroinc.com
MACROConsulting, Inc.
13

14. New Approach
Dual Direct Binary Response- Negative DBR
By ADDING A SECOND, NEGATIVE DIRECT
BINARY RESPONSE QUESTION, we hope to
REMOVE or MINIMIZE scale usage bias.
As a FURTHER ATTEMPT to minimize scale use bias,
half of respondents will be required to SELECT AS MANY
NEGATIVE ATTRIBUTES AS POSITIVE.
www.macroinc.com
MACROConsulting, Inc.
14

15. Summary of New Approach
AA nm a ae e dM oo dfiife e dBB a a n d a a n c h o e e dM a a xDDfiff fSS c ailnn g w i t h
ni im t t d M di i d
r r nd- - nchor r d M x/ / i
cal i g
wP t h iP i v e t av e a ne g a te v e t Dv ee D t r B cn aB yn R r y p o n s e n A e B A M B R )
i os tosi i nd N d N iga i ir c i e i t ri a es Respo ( s M
MACROConsulting, Inc.
15

16. New Approach
Analytics-Derived Parsimony
2
Latent Class Choice Models
•
•
With Large Sample
With Covariates
Hierarchical Bayes
•
•
Covariates in upper model
Adjusted priors
The goal of the above analytic approaches is to minimize the number of Max/Diff tasks each
respondent must complete and still estimate disaggregate utilities with acceptable accuracy.
www.macroinc.com
MACROConsulting, Inc.
16

17. AGENDA
Brand Imagery Measurement
Case Study
• Research Objectives
• Research Methodology
• Summary of Findings
www.macroinc.com
MACROConsulting, Inc.
MACROConsulting, Inc.
17

18. Research Objectives
•
Compare two approaches to brand
imagery measurement, ratings scales
and max/diff, in terms of:
-
•
Explore alternative methods of
estimating max/diff utilities most
accurately and most efficiently:
-
www.macroinc.com
Inter-brand discrimination
Inter-item discrimination
Predictive validity
Standard HB
HB with positive Direct Binary Response
HB with positive DBR and unconstrained
negative DBR
HB with positive DBR and constrained
negative DBR
Latent Class Choice
Use of covariates
Tuned priors
MACROConsulting, Inc.
MACRO Consulting, Inc.
18

19. Research Methodology
Online Survey:
• Two cells
- Rating Scales (n=436)
- Max/Diff (n=2,605)
• Three brands
• 12 items
• Questionnaire:
- Brand image measurement
- Three dependent variables
Item top 3 rank-order
Brand purchase likelihood
Brand forced-choice preference
- Demographics
www.macroinc.com
MACROConsulting, Inc.
19

20. Summary of Findings
In general, AMBAMBR is superior to ratings scales:
-
Better inter-item discrimination
Better predictive validity
Fewer unacceptable respondents
Elimination of both brand halo and scale usage bias
Of the AMBAMBR methods tested, the two methods which
included negative DBR were superior:
- Positive DBR reinserts brand halo into the data
- Positive DBR has slightly weaker inter-item discrimination than either
Negative DBR
AMBAMBR takes longer to administer and has higher
incompletion rates
Task set reduction could not be fully explored with these data
www.macroinc.com
MACROConsulting, Inc.
MACRO Consulting, Inc.
20

21. Positive DBR Appears to Show Greater Inter-Item Discrimination Than
Rating Scales
8.50
8.00
7.50
Rating Scales
7.00
6.50
6.00
5.50
Brand#1
NewBrand
Brand#2
5.00
1.00
0.50
0.00
Positive DBR
-0.50
Brand#1
-1.00
NewBrand
-1.50
Brand#2
-2.00
-2.50
-3.00
www.macroinc.com
21
MACROConsulting, Inc.

22. Negative DBR Approaches Yield Similar Results
2.00
1.50
Constrained
Negative DBR
1.00
0.50
0.00
Brand#1
NewBrand
Brand#2
-0.50
-1.00
2.50
2.00
1.50
Unconstrained
Negative DBR
1.00
Brand#1
NewBrand
0.50
Brand#2
0.00
-0.50
-1.00
www.macroinc.com
22
MACROConsulting, Inc.

23. Negative DBR Approaches Bring New Brand Closer
Constrained
Negative DBR
Rating Scales
8.50
2.00
8.00
1.50
7.50
7.00
Brand#1
5.50
5.00
Brand#1
0.50
NewBrand
Brand#2
6.00
1.00
NewBrand
6.50
0.00
Brand#2
-0.50
-1.00
1.00
2.50
0.50
2.00
0.00
1.50
-0.50
Brand#1
-1.00
NewBrand
Brand#1
1.00
NewBrand
0.50
-1.50
Brand#2
-2.00
Brand#2
0.00
-2.50
-0.50
-3.00
-1.00
Positive DBR
www.macroinc.com
Unconstrained
Negative DBR
MACROConsulting, Inc.
23

24. Inter-Item Discrimination Greatest for Negative DBR
Average number of statistically significant differences across 12 items, within brand*
Ratings
No DBR
Positive DBR
Unconstrained
Negative DBR
Constrained
Negative DBR
1.75
4.46
3.90
4.30
4.68
NEW BRAND
0
4.28
3.16
4.25
4.50
BRAND#2
1
4.69
3.78
4.48
4.70
BRAND#1
* 10 random draws of n=436 were pulled for all data sets except Ratings
www.macroinc.com
MACROConsulting, Inc.
24

25. Predictive Validity of AMBAMBR Superior to Rating Scales
Hit Rates for Top 3 Items Ranking
Random
Numbers
Ratings
No DBR
Positive DBR
1 OF 1
8%
14%
27%
28%
27%
26%
(1 OR 2) OF 2
32%
30%
62%
64%
62%
65%
(1, 2 OR 3) OF 3
61%
51%
86%
87%
86%
88%
www.macroinc.com
Unconstrained Constrained
Negative DBR Negative DBR
MACROConsulting, Inc.
25

26. AMBAMBR Yielded More Valid Completes
Invalid Completes
Max/Diff
Ratings
www.macroinc.com
4%
32%
32%
MACROConsulting, Inc.
26

27. Brand Halo Was Measured Using Confirmatory Factor Analysis
If brand halo exists,
halo latent will positively
influence scores on all items
www.macroinc.com
MACROConsulting, Inc.
27

28. Ratings and Positive DBR Reflect Strong Brand Halos
Brand Halo
Latent
Ratings
Std Beta
Prob
No DBR
Std Beta
Prob
Positive DBR
Std Beta
Prob
Unconstrained
Negative DBR
Std Beta
Prob
Constrained
Negative DBR
Std Beta
Prob
ITEM 1
0.85
***
-0.14
***
0.90
***
0.44
***
0.27
***
ITEM 2
ITEM 3
ITEM 4
ITEM 5
ITEM 6
ITEM 7
ITEM 8
ITEM 9
ITEM 10
ITEM 11
ITEM 12
0.84
0.90
0.86
0.77
0.85
0.83
0.82
0.88
0.87
0.77
0.88
***
***
***
***
***
***
***
***
***
***
na
-0.38
-0.20
0.10
-0.68
-0.82
0.69
0.24
0.58
0.42
-0.05
0.26
***
***
***
***
***
***
***
***
***
0.015
na
0.78
0.95
0.90
0.88
0.87
0.83
0.75
0.90
0.94
0.85
0.91
***
***
***
***
***
***
***
***
***
***
na
-0.56
0.42
0.30
0.03
-0.21
0.42
0.01
0.77
0.86
0.07
0.69
***
***
***
0.25
***
***
0.87
***
***
0.02
na
-0.72
0.32
0.16
0.01
-0.24
0.20
-0.23
0.62
0.90
-0.12
0.53
***
***
***
0.78
***
***
***
***
***
***
na
www.macroinc.com
MACROConsulting, Inc.
28

29. Scale Usage Bias Was Measured Using Confirmatory Factor Analysis
Brand halo drives scores
within brand. Scale
usage bias drives scores
independent of brand.
If scale usage bias
exists, the scale usage
latent should load
positively on all items
www.macroinc.com
MACROConsulting, Inc.
29

30. Only Ratings Reflect Strong Scale Usage Bias
Scale Usage
Latent
Ratings
No DBR
Positive DBR
Unconstrained
Negative DBR
Constrained
Negative DBR
NUMBER OF
NEGATIVE LOADINGS
0
14
5
10
15
NUMBER OF
STATISTICALLY
SIGNIFICANT
LOADINGS
35
30
28
32
29
www.macroinc.com
MACROConsulting, Inc.
30

31. AMBAMBR Superior But Slower
AMBAMBR Has Higher Dropout Rates
RATINGS
AMBAMBR
TOTAL INTERVIEW LENGTH
9.7 MINUTES
15.8 MINUTES
BRAND IMAGE MEASUREMENT
1.7 MINUTES
6 MINUTES
RATINGS
AMBAMBR
9%
31%
INCOMPLETION RATE
Can We Reduce the Number of
Max/Diff Tasks to Shorten Interview
Length and Decrease Dropout Rates?
www.macroinc.com
MACROConsulting, Inc.
31

32. HB Models May Perform Slightly Better Than LC With Full Task Sets
But All Perform Well
U N C O N S T R A I N E D Negative DBR
C O N S T R A I N E D Negative DBR
8 Tasks
HB
LC
4 Tasks
HB
LC
2 Tasks
HB
LC
8 Tasks
HB
LC
4 Tasks
HB
LC
2 Tasks
HB
LC
1 OF 1
27%
19%
21%
20%
20%
19%
26%
21%
24%
21%
22%
22%
(1 OR 2) OF 2
62%
54%
59%
57%
58%
56%
65%
61%
61%
59%
59%
56%
(1, 2 OR 3) OF 3 86%
81%
85%
82%
82%
83%
88%
84%
86%
84%
85%
82%
www.macroinc.com
MACROConsulting, Inc.
32

33. All HB and LC Models Perform Very Well - Perhaps Too Well
• Hit rates seem relatively unaffected by:
−
−
−
−
−
www.macroinc.com
Number of tasks
Number of latent classes
Tuned priors
Covariates
Sample size
What’s
Going On?
MACROConsulting, Inc.
33

34. Perhaps These Data Have Little Heterogeneity
• Category is not emotionally engaging
• Brands are not differentiated
− Commodity-like category
− No polarizing brands, eg, Microsoft, Apple or Donald
Trump
− Brands with new technologies not yet established
www.macroinc.com
MACROConsulting, Inc.
34

35. Aggregate Model Works as Well as Disaggregate
U N C O N S T R A I N E D Negative DBR
Random
HB
8 Tasks
N=1,324
HB
2 Tasks
N=105
HB
2 Tasks
Constantutils
N=105
1 OF 1
8%
27%
22%
25%
(1 OR 2) OF 2
32%
62%
59%
61%
(1, 2 OR 3) OF 3
61%
86%
82%
82%
www.macroinc.com
MACROConsulting, Inc.
35

36. Summary and Implications
•
The forms of Max/Diff referred to here as AMBAMBR are superior to rating
scales for measuring brand imagery:
-
Better inter-item discrimination
Better predictive validity
Elimination of brand halo
Elimination of scale usage bias
Fewer invalid completes
•
Positive DBR alone reintroduces brand halo
•
Positive DBR must be combined with some form of negative DBR
•
For comparability across brands and time:
-
•
Raw utils must be used rather than rescaled utils
Some form of dual response must be used
AMBAMBR takes longer to administer and has higher incompletion rates
www.macroinc.com
MACROConsulting, Inc.
36

37. Further Research
• Can we reduce the number of tasks when brands are
heterogeneously perceived?
• How many brands and statements can realistically by
accommodated?
• Is constrained or unconstrained negative DBR superior?
• How would traditional dual response format affect these results?
• Is there a better way to evaluate utility performance?
www.macroinc.com
MACROConsulting, Inc.
37

38. And The BIG Question:
• Is there a shorter name for this technique
than:
A n i m a t e d Mo d i f i e d
B r a n d - a n c h o r e d Ma x / D i ff S c a l i n g
with Positive and Negative
Direct Binary Response?
www.macroinc.com
MACROConsulting, Inc.
38

39. 39