Webinar Mobile Research: Are the results valid?”

Gerard Loosschilder
CIO & Partner
Robin de Rooij
Senior Manager
Abigail Joffre
Today’s webinar host
#SKIMwebinar
Share your thoughts online:
Mobile Choice-based Conjoint
A valid alternative

Mobile surveys: it’s time to act
Increasingly people take surveys on a
mobile platform. Filtering them out
may skew the sample.
2
Source: Burke, July 2013
89%
4%
7%
2013
Smartphone
Tablet
PC
Screen
In
Out
Biased sample
Standard
survey design
Mobile survey
design
% of people taking
the test survey by
platform
SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”

So we better format right…
3 SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”

Introducing 3*3 mCBC
Today’s presentation is about the
validity of mobile choice-based conjoint

Adapting to the shorter attention span of the mobile user
By reducing the number of choice tasks and number of choices per task
5
Three choice tasks Completed inThree options each

At the same valid results as a traditional CBC exercise
The good news
The MAE of share predictions is
<4% at the correct sample size
Overall results, conclusions and
recommendations are the same
The bad news
So far, it is impossible to reduce
the number of choice tasks and
options per task to fewer than the
3*3 platform

3*3 mCBC requires a larger sample size
As a rule of thumb, sample size is
tied to number of parameters to be
estimated, determined by the
number of attributes and levels
Traditionally, we recommend a minimal
sample size of n= 200 for any conjoint
study. Now, the sample size goes up
with the number of parameters to be
estimated.
If # of
parameters is
Sample size
n=
12 (3 atts*4 levels) 400
16 (4 atts*4 levels) 800
20 (5 atts*4 levels) 1,200
25 (5 atts*5 levels) 1,600

3*3 mCBC requires a well-balanced research design
In a 3*3 mCBC research design,
the number of data points
collected is much smaller than the
number of parameters to be
estimated.
So, the key success factor is a
high-quality research design
Frequency balanced
Every combination of two
attributes and their levels is
represented well by all research
designs
Market knowledge included
We take current market shares
and knowledge about consumer
preferences into account in design
generation

How do we know if mCBC
using a 3*3 design is valid?
Empirical evidence

Because we have validated it …
We determined the relation between
sample size and convergent validity
It shows that at increasing sample sizes,
mobile CBC is a good approximation of
common CBC on traditional platforms

… in a study into pain killers and batteries
11
What is the validity of mobile CBC at various sample sizes
if compared with the same conjoint study in a traditional way at n= 200?
Attributes:
 Brand
 Price
 Size
3-5 levels each;
12 parameters
Attributes:
 Brand
 Admin method
 Dosage
 Package
 Price
3-5 levels each;
16 parameters

Attribute importance values show great similarities
at all sample sizes for mobile CBC
12
The values for
batteries are
equally similar
Variations in
attribute
importance values
are insignificant
0%
10%
20%
30%
40%
Attributeimportance
Platform and sample size
Pain killers
Brand
Admin method
Dosage
Package
Price

However, at larger sample sizes, preference shares get
closer to benchmark leg, demonstrating greater validity
13
Mean Absolute Error
(MAE) is a metric
expressing how far
apart the preferences
shares are as
measured by mCBC at
increasing sample
sizes compared with
the benchmark CBC
at n= 2000.0%
2.5%
5.0%
7.5%
10.0%
200 300 400 500 600 700 800 900
MAEvaluein%
Sample size in the mobile leg
Mean Absolute Error values (MAE, in %)
Batteries
Pain killers
The break point is at n= 400; twice the
sample size of the benchmark
Especially at pain killers at 16
parameters (instead of 12 for batteries)

Conditions to make 3*3 mCBC work – research design
Every display of product profiles in a choice task is valuable. It
should not be wasted on prohibitions or alternative specific designs.
• We can not tolerate any reduction in D-efficiency values
So for now, we can apply mCBC in some standard situations.
We do not apply 3*3 mCBC in situations in which there are
prohibitions, utility balanced designs and alternative-specific
designs

Conditions to make 3*3 mCBC work – real estate
15
There is no space for large
amounts of information
Max. 5 attributes
Text is succinct, keywords
only
No complex descriptions
Pictures are
icons or schematic
Be careful with package or
ad testing

16
Interaction design
http://tinyurl.com/skimmcbc
Powered by

3*3 mCBC in Price Sensitivity Analyses – batteries
17

3*3 mCBC in Price Sensitivity Analyses – pain killers
18

3*3 mCBC in Telecom – design of a postpaid portfolio
19

Mobile in MaxDiff applied to Semantics
and other tests of claims and benefit statements
20

We capture actual consumer behavior
in a mobile shopping exercise taken on mobile devices
Packaging can be
seen in detail through
a zoom-in feature
The shopping basket
selections can
be reviewed
SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”21

The exercise allows us to optimize features
of the mobile retail environment and its products
22
Product rankings in
search results
Ratings & Reviews
Price & Promotions
Images and
iconography
Product
descriptions
and sizes
24 hour
protection
Product callouts
Shipping options

Replicating online retail environments transcends
to services offered by telecom networks
23
A build-your-own post-paid subscription
environment inspired by Hi,
a subsidiary brand of KPN in NL
A build-your-own post-paid subscription
environment inspired by Tele2,
an independent price fighter brand in NL

… and by publishing houses…
24
The original website of the publishing house
to have consumers sign up for a subscription …
… converted into a conjoint choice exercise
to mimic the online environment

3*3 mCBC is a valid addition
To conclude

To conclude – mCBC is a valid addition
Do
• Take market knowledge into
account
• Use a larger sample
• Use succinct text, iconized
visuals
• Have five attributes at five
levels max
Don’t
• Use it for large CBCs
• Include research design
restrictions
• Use it for package, ad or
concept work

Next steps
• SKIM researchers will continue to validate mCBC …
• For allocation studies
• For large research designs using consideration sets
• For more complex visuals
• Keep improving UI for Mobile CBCs
• Across dominant platforms
• Translate learnings back to any CBC

Go to www.skimgroup.com/webinars
for today’s presentation slides and more!
Robin de Rooij
Senior Manager
r.derooij@skimgroup.com
@Robin_de_Rooij
Abigail Joffre
a.joffre@skimgroup.com
+1 201 963 8430
@SKIMgroup
Gerard Loosschilder
Chief Innovation Officer
g.loosschilder@skimgroup.com
@gloosschilder
#SKIMwebinar
Share your thoughts online:

Webinar Mobile Research: Are the results valid?”

Recomendados

Recomendados

Mais conteúdo relacionado

Mais de SKIM

Mais de SKIM (20)

Último

Último (20)

Webinar Mobile Research: Are the results valid?”