This document discusses the need for market research to evolve in order to keep pace with advances in technology and changes in consumer behavior. It argues that research should be more nimble, participatory, contextualized and ongoing. Specifically, it proposes a new model where research is conducted in real-time, engages participants, provides nuanced insights at scale, and continually adapts. It acknowledges this requires moving outside of traditional research comfort zones by making tradeoffs like pragmatism over purity, specificity over generalizability, authenticity over artificiality, transparency over anonymity, relationships over distance, and looking forward rather than backward. The document advocates for integrating humanistic and experimental approaches iteratively to provide both exploratory and confirmatory insights
2. written by
Manila Austin, Ph.D., Director of Research
Julie Wittes Schlack, Senior Vice President, Innovation & Design
3. executive
summary
Advances in social media, the empowerment of everyday consumers, The choice facing the industry need not be to invest blind faith in
and the need for more actionable insights fuel a mandate for market old, authoritarian research techniques or in uppity, untested new
research to do more, faster. These developments create great ones. This paper sets out to collaboratively build a foundation
opportunity for researchers to exercise strategic leadership, to for a 21st Century understanding of market research—what it can
inspire and innovate by bringing the voice of the customer to life, accomplish and how. We seek to pose some provocative questions,
to apply new insights to complex business problems, and to offer some initial thinking, and engage the industry as a whole
produce creative, timely and actionable recommendations to in an ongoing conversation about how to embrace the blurred
drive business results. boundaries between marketing and market research, and activate
But the use of social media-driven research also fuels the quality the ability to quickly garner and act on customer insight.
debate that’s been raging for years, creating worries about declining
A New Model for a New Age
response rates, questionable respondents, sample size, and
projectability. Market researchers need to consider and address The emergence of social media challenges us to recognize and
these legitimate concerns, while also recognizing the ways in which figure out how to intelligently embrace a new way of doing research—
online, social, community-based research can actually strengthen one that is sure to generate insight, to both inspire and inform,
validity and enhance quality. and to provide strategic value. We see an integrative paradigm
emerging—a 21st Century model—in which research is:
To take that leap, it’s helpful to think in terms of tradeoffs, to
understand what researchers are risking—and gaining—by • Conducted in real time, so that it’s relevant and actionable
shifting their focus and methods. • articipatory and engaging, which means adopting
P
humanistic and consumer-centric methods
TRADE BECAUSE • extured and nuanced, with the potential of getting
T
Purity for Pragmatism Pragmatic = Actionable rich detail on a really large scale
• ontinually evolving to meet new marketplace
C
General for Specific Specific = Relevant demands from consumers, clients, and competition
• ore dynamic, where we will rethink and re-invent to
M
Artificial for Natural Natural = Authentic
drive innovation on an ongoing basis
Anonymity for Transparency Transparency = Engagement Taken together, these criteria feel pretty different from the
somewhat dry language many of us grew up with. The online era
Distance for Relationship Relationship = Candor challenges many of our assumptions about data quality—validity,
projectability, bias—and it represents a significant change in how
Randomness for Purpose Purpose = Productivity we think and go about our work. As an industry—and to varying
degrees as individuals—we are being nudged, or shoved, out of
Control for Collaboration Collaboration = Creativity our comfort zone.
Looking Backward
Looking Forward = The Future
for Looking Forward
Leaving Our Comfort Zone: 21st Century Market Research | 3
4. The New Roads to
Quality Require
Challenging Old
Assumptions And
Making Informed
Trade-offs.
Quality is an urgent issue but there’s a more important
issue to do with what needs to be done going forward,
because the industry isn’t keeping pace with the change
going on around us. People are too focused on
quality, people are too focused on probability and
non-probability samples, people are too focused
on respondent engagement—this is all about making
minor changes to what we are doing right now.
Those are all necessary, but they’re not sufficient
conditions for the success of the function.
“(COCA-COLA’S) STAN STHANUNATHAN ON WHY QUALITY DOESN’T MATTER,”
RESEARCH., 22 OCTOBER 2009
Leaving Our Comfort Zone: 21st Century Market Research | 4
5.
6. Pragmatic =
Actionable
It is time to decouple the notion of “quality” from purity. Today, it is Top-down, researcher-centric approaches are best for generating
more important for research to be actionable than irrefutable. This feedback, for confirming what is already known or suspected. By
means shifting our focus—aiming not for the perfect, bias-free study, design, they do not expand a problem space nor do they generate
but for an approach that pragmatically applies a range of methods to knowledge outside of the researcher’s frame of reference. They also
generate and test hypotheses. Integrating elements of both humanistic run the risk of alienating the very people—everyday consumers—to
and experimental approaches allows us to produce timely, “good whom companies desperately need to listen. Are there circumstances
enough” research targeted to specific business needs. Rather than under which you want and need to hear from the population of
itemizing the statistical significance of individual data points, we less-engaged, more neutral customers? Of course, but they are
need to focus on synthesizing findings that are relevant, insightful, not and should not be the only game in town.
and actionable. We believe that there clearly is a place for both kinds of knowledge—
exploratory and confirmatory. But while researcher-centric
approaches are necessary for some purposes, numbers don’t tell
Integrating consumer- researcher-centric approaches the whole story, or even close to it. By grounding knowledge
generation in the humanistic tradition, we ensure that we are
focusing on the consumers’ world view and framework, not just
Consumer- researchER-
the brands’. And combining elements of both traditions in an
centric Design centric Design iterative, agile, and pragmatic way allows researchers to move
Discovery Feedback between exploring and testing, generating and confirming
Exploring Focusing
Hypothesis generation Hypothesis testing
to produce the most actionable information in support of
specific business needs.
Delivers: Delivers:
Insight and Meaning Confirmation and Numbers
We have learned that a longitudinal, iterative approach—one that
combines humanistic, person-centered approaches with more
traditional, experimentally derived ones—is most effective.
Humanistic methods (such as ethnographic-type activities that treat
participants as active co-investigators, as opposed to simply passive
survey respondents) are consumer-centric, reflecting how real people
want to engage with researchers. They are discovery-oriented and
exploratory, most suitable for uncovering connections, insights, and
nuances that lead to innovation and competitive advantage. And
iterative, as opposed to episodic research, supports researchers and
participants in an ongoing discovery process that allows everyone to
ask new questions as they uncover and reflect upon what they learn.
My call to action is that we would figure out the
way to return to the consumer’s backyard…We
have to rebuild that trust and we have to gain
much better insights than we are today…We need to
listen to them on their time, their turf, and in the
ways they want to communicate to us, not in ways
that we choose to communicate with them. And we
need to re-establish the trust and the confidence
that they deserve to have in us…
Kim Dedeker, ARF Leadership Forum 2008
Leaving Our Comfort Zone: 21st Century Market Research | 6
7.
8. SPECIFIC =
RELEVANT Market researchers have been concerned with the quality of online
data collection since the dawn of the Internet, largely due to fears
that online consumers fail to accurately represent the general
population. These concerns, however, are becoming increasingly
irrelevant for two main reasons. First, more people are accessing the
Internet, making distinctions between online and offline groups less
meaningful. Second, if relevancy of insight is an important quality
factor (which we believe it is), then researchers have more to gain
by listening to the “right” group of people than they do by trying to
generalize findings to a generic population.
There is plenty of evidence showing that, except for a few discrete
segments, the Internet population in the U.S. is quickly becoming the
general population. Many European countries’ Internet penetration is
higher than what we have in the U.S., and projections for developing
markets (with the advent of smartphones) indicate that soon,
two billion people will be online.1 Online versus offline is quickly
becoming a non-issue.
More importantly, quality research must produce relevant findings;
and we have learned that listening to targeted, specific groups of
customers is the surest road to relevancy.
If you want to deepen customer loyalty, who better to engage than
members of your brand’s loyalty program? If your goal is to broaden
your brand’s appeal, then hearing from fans of your competitors’
brands may be the most useful approach.
Researchers can be more confident in taking action when they
trust they have the right people assembled to address their specific
objectives. We advise our clients that whether or not they generalize
from communities depends on the community composition and on
the particular question they’re trying to answer. Many questions—like
the ones featured in the United Airlines example shown here—can be
explored and findings effectively generalized when the community is
specific and the target market is defined enough.
On REPRESENTATIVE SAMPLES
We recruited a community of our most valuable
customers—a relatively small population—
from our customer list. Because the community
accurately reflects my population of interest,
and because the purpose of the community is to
understand our most loyal customers, I believe
the results of my community’s research are
reflective of the larger target population. So even
though it’s not a traditional panel, I’m comfortable
using the community more quantitatively.
Dan Comenduley, Project Manager, Customer Metrics Insights,
United Airlines
1. www.internetworldstats.com/stats.htm
Leaving Our Comfort Zone: 21st Century Market Research | 8
9.
10. NATURAL =
authentic
Perhaps one of the most overlooked ways to enhance quality is Trading artificial approaches for more naturalistic (private, online,
leveraging the power of a naturalistic research setting. Natural at-home) research settings can strengthen validity and data quality,
settings promote authentic participation because the researcher especially when researchers are asking people to reveal intimate
engages people on their own terms, refraining from barging into aspects of their lives.
people’s lives or extracting them from their homes to answer We have found that the freedom and relative safety of private
questions they had no say in generating. One benefit of online online communities allow for the iterative exploration of the
research—and private communities in particular—is that it allows most intimate content. We have seen that in naturalistic settings,
people to participate on their own time, on their own terms, and regular people openly share detailed information about their
from their own homes or smartphones. They are able to use social financial situations, experiences with serious illnesses, stresses
technologies to bring their own lives to the researcher. and hopes, relationship worries, and even embarrassing quirks
and habits.
Another benefit of naturalistic settings is that they not only provide
more authentic contexts for engagement and discovery, but they
strengthen researchers’ ability to generalize findings. Participating
in focus groups and anonymous surveys requires people to step out
of their daily lives; thus findings generated from these approaches
do not always translate to real-life situations. And with so much
public distrust in how companies use electronic information (e.g.,
identity theft, subversive marketing, etc.), people are less likely
to be truly open and forthcoming if they don’t know or cannot
trust the researcher. But a willingness on the part of researchers
to model the kind of self-disclosure that we hope to elicit from our
participants, coupled with the kind of ongoing connection enabled
by social media, make it possible to build a trusting relationship
between researcher and “subjects” over time.
On intimacy
We formed a community of newly diagnosed cancer
patients and primary caregivers who participated
no matter where they lived, whatever hour of the
day and regardless of their condition. In contrast
to typical market research, the richest ‘aha’s’ came
when the patients and caregivers initiated their
own discussions, and we had the opportunity to
really just listen—observing how the members
supported each other and learning from the
stories they shared. We were a fly on the wall
in the treatment room, which for healthcare
marketers is very unusual.
Alana Brody, Former SVP Strategic Development,
National Comprehensive Cancer Network (NCCN)
Leaving Our Comfort Zone: 21st Century Market Research | 10
11.
12. transparency =
ENGagement
One way to get quality information is to trade anonymity for a
transparent, open approach. Researchers are often concerned
that findings will not be valid if participants speak freely with one
another, know who the sponsoring company is, or are active
co-participants in the research process. Our experience and our
five-year program of research-on-research suggest this is actually
not true. We have found that when companies trade in anonymity,
they gain better engagement, more textured insights, and increased
value overall.
First, transparency in research is just simply more engaging for
people. When companies are upfront in disclosing their identity,
when they invite people into the fold, and when they demonstrate
that they are truly listening, people respond in kind. The research
we have done on community member participation clearly shows
that branded communities outperform unbranded ones.2 And our
research on corporate listening shows that community members
value feeling that their voice is heard and that their contributions
are making a difference.3 Engagement is critical for quality; when
people are engaged they try harder, they do and share more, and
go to great lengths for companies when they know who they are
talking to.
Second, companies undermine the ROI of research when they fail
to be transparent. Participation is lower, researchers must concoct
and field “dummy” research to disguise their identity (which
waters down the learning agenda and wastes time), and members’
energy is often diverted into guessing games that are not valuable
for the client.
And last, many of our clients have conducted parallel studies
comparing results from communities with those generated from
blinded approaches (e.g., panel surveys, focus groups, etc.) We
have collected over 25 examples across a range of clients, and the
comparison results are consistent: findings from data collected
through a transparent, community approach are directionally
similar to those obtained by other methods.
On Anonymity
We found that in an unbranded community we were
continually challenged with talking directly to
our members without revealing our identity, and
thus being forced to find ‘creative’ ways to ask
questions and conduct ‘decoy’ research. Our
real objective was to have a direct dialogue
with our consumers and gain valuable insights
into their needs as shoppers. When we did move
forward and unveil our identity, becoming a
branded community, we found member engagement
increased and conversations become much more 2. Katrina Lerman and Manila Austin, The Fifth ‘P’ of Marketing:
relevant and valuable to both parties. Participation (Communispace whitepaper, 2007)
Michelle Laslo, Senior Manager,
Customer Strategy and Insights, PepsiCo 3. atrina Lerman and Manila Austin, What Companies Gain
K
from Listening (Communispace whitepaper, 2006)
Leaving Our Comfort Zone: 21st Century Market Research | 12
13.
14. relationship =
candor
It may seem counter-intuitive, but quality can be heightened
when researchers trade distance and objectivity for closeness and
relationship. A common fear market researchers share is the
possibility that the feedback from people participating in branded
research communities will be tainted, inflated or somehow not
trustworthy as a result of their ongoing involvement.
Our research shows that, if anything, members become more honest
and forthright as their tenure increases. Over time, members do come
to view company sponsors more positively, but this does not affect their
ability to provide valuable—and critical—feedback.
For example, we have developed a method for coding open-ended
responses for both “candor” and “richness” and have found that
members continue to provide critiques and textured detail in
their contributions, regardless of tenure.
We have also found that the relationship that develops between
company sponsors and community members can actually increase
clients’ confidence and trust in what they hear. Because communities are
transparent, composed of the “right people” for the business objective,
and because companies really know who they are hearing from, clients
feel more comfortable acting on the advice of the research community.
On candor
In a past job at a major food company, we had
members react to a new product concept early
in development. Internally, we really loved the
concept we were fielding, and had high hopes
for it. Members responded quickly to the concept
test—within 72 hours—and they hated it, totally
rejecting the idea. But it wasn’t just a gut
response. Because they felt they knew us, were
invested in us, and didn’t want to see us screw
up, they also provided us with ample, very clear
feedback on why. Our loyal users saw a fatal flaw
in the product that we had missed in our own
excitement. Based on their responses, we pulled
the product idea within three weeks, saving the
company costs on further development. In this
case our community helped us fail faster, and
allowed us to feel good about the decision
because we knew we could trust that
critical feedback.
Adrian C. Bing-Zaremba, Consumer Insights and Market Intelligence,
Boehringer Ingelheim Consumer Health Care Products
Leaving Our Comfort Zone: 21st Century Market Research | 14
15.
16. purpose =
productivity
Concern about professional respondents has been acute in recent We also need to re-evaluate our fear of the dreaded practice
years, but it is time to realize that quality requires trading in the effect—the idea that repeated participation in research necessarily
myth of the “fresh,” unpracticed consumer, and focus instead on creating erodes quality. Recent research and our experience suggest just
purposeful relationships with people to help them do a better job of what the opposite.
they are already doing anyway. The notion that there is a “random” There is mounting evidence that practiced research participants—
population of people in the world that do not take surveys, answer people who are motivated and engaged, and well-versed in how to
marketers’ questions, post and read reviews, or engage with brands on best contribute—actually produce better results. A study released
a regular basis is wishful thinking in the 21st Century. Consumers today by the ARF’s Online Research Quality Council actually found
are extremely marketing savvy; and if there are untapped consumers out that increasing panel membership lowered “bad” survey taking
there, then they certainly aren’t representative of the general population. behavior (such as straight-lining or speeding).4 “Professional”
respondents, then, do not necessarily threaten—and may actually
improve—quality.
This sounds counter-intuitive, but we have also found that
experience participating in research—especially when the purpose
is transparent—produces better results. Members are motivated,
proficient, and simply more productive.
This is certainly the case with new product development
communities, but it is also true for insight communities.
Our participation research also shows no relationship between
greater monetary incentives and increased participation.5
Intrinsic motivation drives engagement. There is something
inherently energizing about a shared purpose and goal-directed
activity; when research participants know why they are being
asked to make electronic collages, take videos of a family dinner,
or brainstorm ideas, they are motivated to do a better job.
On practice
I like problem solving, and this community is about
problem solving, in terms of identifying what people
need, what they want to see done differently, and
how you can meet their expectations.
After a few months, I guess the [incentive] became
tiresome and the conversation kind of addictive. 4. obert Walker, Raymond Pettit and Joel Rubinson, Foundations
R
I like many of the topics and hearing guys’ thoughts of Quality Knowledge Brief (The Advertising Research
on them, as well as having the opportunity to give Foundation, 2009)
my own perspective.
Members of two Communispace Communities
5. atrina Lerman and Manila Austin, The Fifth ‘P’ of Marketing:
K
Participation (Communispace whitepaper, 2007)
Leaving Our Comfort Zone: 21st Century Market Research | 16
17.
18. collaboration =
creativity
Marketers must also consider trading sterility and one-to-one tactics for
dynamic collaboration based on many-to-many interactions. To reap
the benefits of collaborative creation, however, researchers need to get
comfortable with—and figure out how to leverage—the group dynamics
that naturally arise when people get together for any purpose. The
likelihood that research participants will influence one another
throughout the research process is an understandable concern;
because interaction is encouraged and happens transparently in online
communities, we worry that members are subject to “group think.”
But the days of isolated, pristine research are over. In this era of tweeting
and lifestreaming and rating and review sites and human billboards,
everyone is subject to influence from their peers. Rather than attempting
to isolate people or control their interaction (or worse, constraining the
naturalistic community setting by virtually hiding responses or forcing
anonymity), we need to ask ourselves how to observe influence behavior
and learn from it.
The alli® example demonstrates the creative potential in building on
group processes for breakthrough solutions.
On Influence
When GSK Consumer Healthcare introduced alli, we
did so knowing that this was a product that could
elicit some intense emotional responses. Our private
communities gave us a chance to pose questions of
users and non-users alike, but more importantly, to
see what questions alli non-users posed of users,
and of how passionate alli consumers answered
them, described their own experiences, and made
their own recommendations. I’d like to think that we
were visionary market researchers doing cutting-
edge work, but honestly, I think we were just being
realistic about the fact that consumers have
unprecedented opportunity to influence one
another online. So our goal wasn’t to pre-empt
‘group think’ so much as to understand it, to see
the peer-to-peer influence process in action and
learn from it.
Andrea Harkins, Manager, Integrated Insights,
GlaxoSmithKline consumer healthcare
Leaving Our Comfort Zone: 21st Century Market Research | 18
19.
20. looking forward =
the future The industry’s historical focus on producing irrefutable, nationally
representative data points is meaningful only if we limit the role of
market research to testing and confirmation. But if market research
is to win a permanent seat at the executive table, if it is to be integral
to brand strategy, then it has got to be about creation, not just
prediction. It is time to trade a backward-looking and confirmatory
stance for a forward-looking and generative approach.
It is no longer clear in today’s long-tailed, filtered, personalized
world that it is actually, scientifically possible to accurately predict
behavior. But what we can do is co-create with our consumers,
rapidly, ideally one step ahead of them, but at least with them.
Markets are becoming more diverse and will continue to change
rapidly. So generating insights and engaging in co-creation
upstream in the development process and doing it in an agile
way (with short time frames and a focus on niche markets as
they emerge)—these are the ways market researchers will keep
pace with customers and “go where they go.” And those market
researchers who can “do” upstream creation as well as prediction
will play strategic roles in driving business.
On Co-Creation
Scholastic Book Clubs recently won a Forrester
Research Groundswell award for our work with
a customer community comprised of parents and
teachers. Our goal was to redesign the Book Club
flyer in an effort to improve the way parents, kids,
and teachers find and buy the right book for the
right child. By zooming out and exploring not just
how teachers evaluate books, but how parents
evaluate their kids’ readiness and interest in
reading, this group helped us effectively redesign
our iconic flyer. This fruitful co-creation couldn’t
have happened if we had simply solicited feedback
on flyer designs from one group or the other in
isolation. The transparency and opportunity
for mutual influence, along with active, visible
facilitation is what made this process so productive.
Judy Newman, President of Scholastic Book Clubs
Leaving Our Comfort Zone: 21st Century Market Research | 20
21. trading off
can mean
trading up.
The industry has moved away from just being
seen as a quasi-scientific activity, providing
hard quantitative measurement that is detached
from the creative process and the complexities
of intuitive decision-making. Today, it is seen as
also embracing a much more pragmatic approach
that requires high levels of creativity and
imagination in order to tease out key insights.
DVL Smith, University of Hertfordshire, U.K.
It’s about understanding the human condition. We’re
too focused on understanding consumption behavior
and shopping behavior. We need to understand the
human condition, which you’ll only know by observing,
listening, synthesizing and deducing.
“(COCA-COLA’S) STAN STHANUNATHAN ON WHY QUALITY DOESN’T MATTER,”
RESEARCH., 22 OCTOBER 2009
Leaving Our Comfort Zone: 21st Century Market Research | 21
22. summary
The kinds of results we have observed over the years are specific to communities as we do them
at Communispace, where insights are generated in the context of continuous, longitudinal,
intimate, purpose-driven groups, and where community members forge real relationships
with one another and with our clients over time.
But beyond our own experience, we generally believe that an online, iterative, consumer-centric
approach mitigates some of the risks and challenges of conventional market research, can actually
enhance quality, and uncovers relevant insights quickly in a way that is fun and authentic for real
people. By leveraging emerging technologies that foster connection, researchers can avoid
the pitfalls of barging into peoples’ lives and instead meet them where they are. As a result,
research efforts are likely to yield more spontaneous and revealing insights. And as an added
advantage, there are also efficiencies and cost-savings researchers can achieve by capturing a
high volume of rich, open-ended data at a relatively low cost.
21st century method Potential Gain
Engaged, motivated participants who
Human, transparent approaches
generate higher quality data
Consumer-centric settings (leveraging online, “Naturalistic” settings that feel safe, maximize
mobile, and other technologies) comfort, and encourage intimacy
Collecting an unprecedentedly large number
Large scale “qualitative”
of open-ended data at a relatively low cost
Research findings that are relevant, timely,
Fast, targeted inquiries
and actionable
Deep knowledge of participants as real people,
Building relationships that leading to greater insight and increased
endure over time confidence (you can trust that you really
know the people participating)
By using humanistic, transparent approaches—in essence, by encouraging consumers to become
engaged in the form as well as the substance of the research—we get really engaged, motivated
participants. By involving customers as actors, not just as subjects; by bringing their voices into
every organizational function, market researchers will enable consumer-led growth. They’ll
ensure that their companies generate solutions that are relevant to customers—in design,
function, packaging, and messaging—and in so doing, drive growth and innovation.
Leaving Our Comfort Zone: 21st Century Market Research | 22