SlideShare uma empresa Scribd logo
1 de 19
Baixar para ler offline
Building a next
generation marketing
measurement system:
the framework
M er kl e Tho u g hT l eader s hip se r i e s




   A Database Marketing Agency
Building a next generation marketing measurement
system: the framework


executive summary:
marketers struggle more than ever to develop accurate, reliable,
and consistent metrics around the performance of marketing
spend. relying on incorrect metrics creates a snowball effect that compounds
measurement errors along the entire marketing decision process. decisions on
how much to spend on marketing in total, how much to spend by media, and how
to coordinate a multi-media program are all directly influenced by the metrics
used. for these reasons, the importance of evaluating your current measurement
framework and building a solid foundation can’t be understated. n

Marketers already employ processes to link a purchase with a marketing activity to create metrics on
marketing performance. These marketing measurement systems fall into one of two camps, each with
its own shortcomings:
	 •	 Top-down	attribution	approaches	that	provide	such	a	high	level	view	that	they	fail	to	be	actionable,	or	
	 •	 Bottom-up	attribution	approaches	that	present	a	collection	of	very	detailed	information	but	fail	to	
     integrate into a single view.

Better	solutions	do	exist	and	can	be	achieved	through	sophisticated	analytical	and	technological	
techniques.	Merkle	introduces	a	next	generation	measurement	system	based	on	Probabilistic
Contribution (PC) scores (see definition on page 5). rather than attempting to link a purchase
to	a	marketing	activity,	a	contribution	score	is	created,	by	integrating	top-down	and	bottom-up	
measurement systems, for each factor that potentially influenced the purchase. The key advantage is
integration	-	bridging	multiple	media/channels	and	points	of	time	for	a	single	purchase	to	provide	a	
comprehensive view of the consumption pattern.

This white paper, the second in a series of four on marketing optimization, addresses the critical
elements	needed	to	build	a	next	generation	marketing	measurement	framework.	A	third	paper	will	
discuss the data, infrastructure, and software tool requirements. and a fourth and final white paper will
outline how to effectively integrate this system throughout the marketing decision making process.




                              Building a next generation marketing measurement system: The framework | © 2008 Merkle   2
Building a next generation marketing measurement
system: the framework


five factors fuel marketing’s roi dilemma
What makes answering such fundamental questions around the ROI for marketing spend so difficult?
Some of the main reasons for this include:

  1. Consumers face daily ad bombardment. studies consistently show that consumers are exposed to
	 	 a	mind-numbing	one	to	two	thousand	advertising	impressions	per	day.	The	sheer	volume	of	
     advertising makes drawing a line from messaging to consumer activity all the more difficult.

  2. fragmentation of media. The nearly monolithic media channels of previous decades have
	 	 splintered	in	the	age	of	the	internet,	on-demand	television,	and	satellite	radio.	For	instance,	the	
	 	 number	of	television	channels	increased	over	a	hundred	times	in	the	last	thirty	years.	The	growing	
     consumer trend of multitasking while accessing multiple media – especially prevalent in young
	 	 consumers	–	throws	a	wrench	into	marketers’	ability	to	attribute	action	to	marketing	message.			

  3. failure to consider the complexity of consumers’ decision process. a purchase is not directly
	 	 tied	to	the	most	recent	advertising	exposure,	despite	the	fact	that	most	organizations	attribute	sales	
     to marketing spend in this manner today. purchase decisions are exceedingly complex and
	 	 consumers	are	likely	to	be	influenced	by	multiple	advertisements	over	time	and	different	types	of	
     media.

 4. Consumers are in control. Consumers have more choice than ever over when and how they
    consume media, making it that much more difficult for marketers to even know if a message
    was received.

  5. organizational challenges. Most	marketing	groups	organize	by	media	type	creating	a	silo	effect	
	 	 (e.g.	brand	marketing,	direct	marketing,	and	interactive/online	marketing).	These	silos	each	develop	
     their own goals, metrics, infrastructures, and partners to maximize the impact within the media they
     can control.




                             Building a next generation marketing measurement system: The framework | © 2008 Merkle   3
Building a next generation marketing measurement
system: the framework


Critical elements of marketing measurement
Marketing	organizations	require	the	following	key	elements	to	effectively	build	and	implement	next	
generation marketing measurement:

	 1.	Analytical	acumen	to	custom-build	a	solution	to	link	all	marketing	activities	to	consistent
     performance metrics (i.e. measurement framework)

	 2.	Data	availability:	Access	to	a	complete,	accurate,	and	frequently	updated	marketing	data	universe

	 3.	Infrastructure	(Media	management	database)

	 4.	Integration	into	business	decision	making	(belief/buy-in	and	integration	through	software	tools)	and	
	 	 integration	across	all	lines	of	business.

Terms Used In This Paper
Standardized	language	will	be	used	in	this	paper	to	provide	relevance	across	different	industries.	A	few	
notable	cases	are	highlighted	below.	Standardized	terms	are	defined	as	follows:

outcome of interest	–	This	paper	will	use	“purchase”	to	represent	a	generic	term	that	could	be	applied	
to any performance measurement a company may use. Throughout the discussion, purchase can easily
be	replaced	with	response,	new	customer,	revenue,	profit,	LTV,	or	any	other	metric	that	may	be	used	to	
calculate desired outcomes. While it is true that some details related to techniques and implementation
may	change	depending	on	the	metric	used,	the	discussion	throughout	this	paper	will	be	broad	enough	
to encompass any of these metrics.

media/Channel	–	A	distinction	will	be	made	between	media	and	channel,	with	media	representing	
communication flowing from the company to the consumer, and channel representing communication
from	the	consumer	to	the	company.	Additionally,	media	will	be	discussed	in	three	classes;	direct,	mass,	
and interactive.

top-down and Bottom-up	–	The	top-down	and	bottom-up	approaches	are	designed	to	represent	
two	common	ways	marketing	organizations	measure	the	performance	of	their	media.	The	top-down	
approach	starts	with	all	media	spend	and	then	breaks	the	media	spend	into	media,	region,	and	
timeframe.	This	is	the	broadest	perspective	commonly	used	in	media	mix	analyses,	but	rarely	takes	very	
detailed information into account (individual level data, creative, quality of advertising, etc.). in contrast,
the	bottom-up	approach	starts	with	the	very	detailed	level	of	information.	This	approach	is	common	in	
direct and interactive media analyses (consumer segments, predictive models, decile analyses, etc).




                              Building a next generation marketing measurement system: The framework | © 2008 Merkle   4
Building a next generation marketing measurement
system: the framework


attribution	–	The	term	“attribution”	represents	any	process	designed	to	link	a	marketing	activity	with	
an	outcome	of	interest	(i.e.	purchase).	Typically,	an	attribution	process	attempts	to	link	the	purchase	with	
a	single	marketing	activity	or	campaign.	More	sophisticated,	and	less	common,	attribution	processes	
apply	probabilities	across	marketing	activities	to	capture	cross-media	influence	more	accurately.

This	paper	outlines	a	new	approach	to	marketing	measurement	by:

	 1.	Setting	the	stage	through	a	discussion	of	typical	attribution	processes	and	the	current	challenges	of	
     marketing measurement systems.

  2. introducing analytical framework for the next generation marketing measurement system. This
	 	 paper	describes	a	new	metric	called	Probabilistic Contribution score (PC score) used as the key
     metric to link marketing activities with the outcomes of interest.

 3. providing guidelines on the analytical approach required to calculate pC scores across all marketing
    activities.

 Probabilistic Contribution Score:
 The Probabilistic Contribution Score (PC Score) is a value assigned to any activity or event (marketing or
 otherwise) that influences the consumer toward a desired action (i.e. purchase a product). PC Score derives its
 name from:
 •	 Probabilistic – The sum of all the values for any given purchase equals
 •	 Contribution – Each value represents the relative contribution or influence specific activities or events have
    on consumers’ decision making process
 •	 Score – The values are designed to be informative within a given purchase or when rolled up to
    summarized reports



Current Challenges facing marketing measurement systems
If	asked,	many	organizations	claim	to	have	robust	and	reliable	processes	to	attribute	sales	to	media	
spend.		Forrester	Research	conducted	a	survey	in	2008	that	concluded	that	69%	of	marketers	believe	
they	are	at	least	somewhat	effective	at	measuring	marketing	ROI.	It	has	been	Merkle’s	experience,	
however,	that	after	digging	beneath	the	surface,	most	marketing	measurement	systems	fall	far	short	of	
robust	or	reliable,	especially	when	trying	to	forecast	marketing	impact	into	the	future.	The	2008	ANA/
MMA	Marketing	Accountability	Survey	showed	that	only	10%	of	marketers	felt	they	could	forecast	
the	effect	of	a	10%	cut	in	budget,	and	just	14%	said	that	senior	management	in	their	companies	had	
confidence	in	their	firms’	marketing	forecasts.		CFO’s	are	even	more	skeptical	of	the	numbers.	Ninety	
percent		of	CFO’s	don’t	use	marketing	ROI	metrics	to	help	set	marketing	budgets.		“They	don’t	believe	
the	numbers,”	said	Jeffrey	Marshall,	the	retired	editor	in	chief	of	Financial Executive magazine.




                                Building a next generation marketing measurement system: The framework | © 2008 Merkle   5
Building a next generation marketing measurement
system: the framework


Another	recent	Forrester	study	highlights	some	of	the	top	barriers	marketing	organizations	currently	
face	to	improving	marketing	ROI	(see	figure	1).	Many	of	the	challenges	Forrester	identifies	are	directly	
linked	to	the	four	challenges	addressed	by	this	framework	(analytic	sophistication,	data	access	and	
accuracy,	technology,	common	metrics	and	business	integration).

Figure 1
   marketers face hurdles in staffing, data, technology, and common metrics




Source:	Forrester	Research,	“Database	Marketers	Evolve	Their	ROI	Measurement”




                                      Building a next generation marketing measurement system: The framework | © 2008 Merkle   6
Building a next generation marketing measurement
system: the framework


two Common approaches to marketing measurement
Marketers already employ processes to link a purchase to a marketing activity to create metrics on
performance. These marketing measurement systems fall into two camps:
	 •	 Top-down	attribution	process	-	providing	such	a	high	level	view	that	they	fail	to	be	actionable,	or	
	 •	 Bottom-up	attribution	process	-	presenting	a	collection	of	very	detailed	systems	that	fail	to	integrate	
     into a single view.

Overview of typical top-down attribution processes
The	top-down	approach	starts	with	the	entire	marketing	budget	and	then	looks	at	how	the	marketing	
budget	is	split	between	major	categories	(media,	regions,	products,	etc).	The	advantage	of	this	
approach is that it provides a framework to compare the performance of all marketing spend with
common	performance	metrics	to	one	another.	This	property	is	crucial	to	the	building	of	a	robust	
marketing measurement system, and is the reason why the approach outlined later in this paper uses
the	top-down	approach	as	the	starting	point.	This	type	of	analysis	is	often	termed	media	mix	analysis.

Figure 2
  top-down and Bottom-up approaches




                              Building a next generation marketing measurement system: The framework | © 2008 Merkle   7
Building a next generation marketing measurement
system: the framework


Certain industries, like consumer product goods and pharmaceutical companies, have a long history of
using	media	mix	modeling	to	help	determine	marketing	spend.		These	industries	tend	to	have	in-direct	
relationship to their consumers and involve purchasing of less expensive products conducive to a short
purchase	consideration	time	(see	figure	3).		These	industries	developed	media	mix	modeling	capabilities	
early	on	partly	because	it	was	one	of	the	few	ways	to	understand	marketing	performance	when	there	
is	no	direct	to	consumer	relationship.		These	industries	also	benefit	from	standardized	data	sources	
(like pos data) which have allowed standardized media mix modeling methodologies to develop. as
relationships	move	more	towards	direct-to-consumer	and	the	purchase	consideration	increases,	the	data	
and	methodology	required	for	media	mix	modeling	becomes	more	customized	for	each	company.		This	
customization requires more investment to develop, and therefore it is typically the companies with
significant	marketing	spend	that	can	justify	the	modeling	work.		

Figure 3
   Choosing the measurement approach
      PurChase Consideration time




                                                                     most established top-down
                                                                     measurement systems

                                    non-direCt                                                        direCt to Consumer




*Note – the company placements above are for illustration purposes only. In some cases a company can be put in multiple areas due to different
product offerings. For example, a Dell desktop computer would be in the top right (as shown) whereas a replacement ink cartridge for Dell printers
would be in the lower right.




                                                 Building a next generation marketing measurement system: The framework | © 2008 Merkle   8
Building a next generation marketing measurement
system: the framework


However,	there	are	some	significant	weaknesses	to	the	top-down	approach,	including:	

  1. it takes too long. it is not uncommon for this type of analysis to take over six months to complete.
	 	 The	deliverable	is	often	a	presentation	of	how	media	performed	for	the	analysis	time	period	along	
     with recommended changes. By the time the recommendations are presented, the marketing
	 	 landscape	may	have	already	changed,	making	the	analysis	outdated	before	it	is	even	presented.	
	 	 Another	way	to	view	this	pitfall	is	that	media	mix	work	is	often	managed	on	a	project	basis,	as	
     opposed to an ongoing system.

  2. it fails to take into account the specifics within each media.	There	are	a	number	of	options	and	
	 	 choices	when	spending	dollars	within	a	media.	If	the	budget	for	that	media	is	being	cut	back	
     due to historical poor performance, then perhaps the allocation of spend within the
	 	 media	should	be	changed,	not	necessarily	the	size	of	the	budget.	For	example,	there	are	many	
     decisions and analyse within a direct mail program that are not adequately summarized
	 	 by	simply	when	just	total	dollars	spent.

  3. lack of integration with media execution. Media mix analyses often conclude with
	 	 recommendations	on	how	to	shift	marketing	dollars	between	available	media.	It	is	rare	for	these	
     analyses to address the complexities associated with the execution of each media. of particular
	 	 importance	are	inventory	and	cost	constraints.	For	example,	a	recommendation	to	increase	spend	
	 	 in	spot	TV	may	require	introducing	new	day	parts	or	networks	which	typically	changes	the	price	to	
     purchase the media. as cost per impression increases, the relative effectiveness of that media is
     likely going to decrease.

These	weaknesses	can	be	summarized	by	the	project	vs.	process	nature	of	the	analyses.		These	analyses	
are	often	conducted	ad-hoc	without	becoming	part	of	the	marketing	measurements	systems.		By	not	
being	fully	implemented	into	the	marketing	culture,	these	types	of	results	rank	as	merely	interesting	–	
but	not	relevant	enough	to	change	marketing	strategy	and	spend.		Additionally,	because	the	top-down	
measurement	approach	fails	to	take	specifics	into	account	there	is	a	need	for	the	more	detailed	bottom-
up approaches.

Overview of bottom-up typical attribution processes
The	bottom-up	approach	starts	with	detailed	information	within	a	media	and	then	establishes	a	system	
to measure performance of segments or campaigns within the media. The nature of the data and
the	type	of	metrics	calculated	vary	depending	on	the	media	discussed	(which	is	part	of	the	problem).	
Some	metrics	are	efficiency	based	(cost	per	impression)	while	others	are	performance	based	(cost	per	
purchase,	ROI).	It	is	more	common	to	see	performance	based	metrics	available	where	it	is	easier	to	
connect media spend directly to individual level purchases.




                             Building a next generation marketing measurement system: The framework | © 2008 Merkle   9
Building a next generation marketing measurement
system: the framework


Direct	and	indirect	attribution	are	two	commonly	used	practices	as	the	foundation	for	bottom-up	type	
metrics.	A	brief	overview	along	with	some	examples	and	pitfalls	are	highlighted	below.

direct attribution	is	often	classified	as	an	attribution	process	that	uses	some	kind	of	unique	identifier	
to link a purchase to a specific marketing activity. examples include asking customers to provide a
specific	code	from	the	back	of	their	catalog	,	a	unique	1-800	number	that	is	only	used	for	a	particular	
run	of	a	television	advertisement,	or	tracking	a	click	on	a	banner	ad	by	directing	to	a	specific	landing	
page.	While	each	of	the	examples	above	has	their	own	merit,	they	ultimately	fail	in	producing	a	
reliable	attribution	process.		They	fail	because	the	direct	attribution	is	only	recording	the	last	marketing	
exposure	before	the	purchase,	but	not	necessarily	what	is	driving	the	purchase	intent	in	the	first	place.		
A	common	example	is	unique	1-800’s	on	TV	ads.		A	TV	ad	may	drive	desire	to	purchase,	but	people	
don’t	use	the	unique	1-800	number	on	the	ad,	but	rather	search	for	the	product	on	Google,	go	to	a	
landing	page,	and	then	use	that	unique	1-800	number.		The	TV	ad	is	given	less	credit	than	it	deserves,	
and search engine marketing gets more credit than it deserves.

indirect attribution,	or	business	rule	attribution,	uses	some	assumptions	to	link	a	responder	to	a	
marketing	effort.	For	example,	a	unique	1-800	number	is	not	used	but	we	know	that	the	consumer	
received	a	catalog	two	weeks	prior	to	the	purchase.		In	this	case	we	use	indirect	attribution	to	assume	
the catalog drove the purchase. This process has some advantages and disadvantages over direct
attribution.

Indirect	attribution	allows	a	marketer	to	attribute	a	purchase	when	direct	attribution	is	not	possible.		
However,	indirect	attribution	rules	come	down	to	little	more	than	an	educated	guess.	How	long	should	
the	time	window	be?	What	if	we	know	the	consumer	received	a	mail	piece	one	week	ago?	One	month	
ago?	Six	months	ago?	This	decision	will	drastically	impact	the	perceived	performance	of	the	direct	mail	
campaigns. even worse, the pattern feeds on itself (up to a point). if it looks like direct mail performs
well then a company will do more direct mail, which leads to an even higher likelihood that a consumer
who purchases your product received a mail piece within the time window. This assumption inflates
direct mail performance metrics even further. eventually the direct mail performance metrics will drop
off	as	volume	increases,	leading	to	a	perceived	equilibrium	relative	to	other	marketing	options.	However	
this	equilibrium	is	typically	far	from	the	true	equilibrium.

a common pitfall for organizations is the sense of sophistication associated with very complex direct
and	indirect	attribution	rules.	The	attribution	rules	continually	become	more	refined	and	complicated,	
leading	the	marketer	to	believe	they	have	a	world-class	marketing	attribution	system	and	therefore	are	
doing very well at linking purchases to marketing activities.




                              Building a next generation marketing measurement system: The framework | © 2008 Merkle   10
Building a next generation marketing measurement
system: the framework


Most	of	the	problems	associated	with	direct	and	indirect	attribution	systems	can	be	placed	into	two	
general	categories:	cross-media/channel	tracking	and	effects	over	time.

  1. Cross-media/channel tracking	–	Most	attribution	systems	attempt	to	link	a	purchase	to	a	single	
	 	 marketing	communication,	whereas	in	reality	consumers	do	not	just	use	one	source	of	information	to	
     make a purchasing decision.

  2. effects over time	–	multiple	advertising	exposures	may	contribute	to	the	ultimate	purchase	even	
	 	 within	a	single	media,	but	most	attribution	systems	will	just	attribute	the	purchase	to	a	single	
     instance.

	 	 Current	marketing	measurement	systems	ultimately	leave	CMOs	with	two	problems:

	 	 	       •	 an inability to see the whole picture.	Today’s	systems	force	CMOs	to	look	at	either	broad	
	 	 	       	 brush	metrics	that	fail	to	consider	specifics	of	individual	media	or	at	detailed	metrics	within	a	
	 	 	       	 media	that	fail	to	translate	into	metrics	that	can	be	compared	across	media.	

	   	   	   •	   a disconnect between budget planning and execution.	Typically,	the	budget	planning	
	   	   	   	    process	has	two	steps:	first,	each	media	or	channel	is	allocated	a	certain	budget	to	spend	and	
	   	   	   	    second,	each	media	or	channel	attempts	to	gain	the	best	performance	it	can	from	the	
	   	   	   	    allocated	budget.	This	process	effectively	creates	a	major	planning	constraint	where	the	
	   	   	   	    budget	by	media	is	considered	fixed.	Each	media	may	be	doing	the	best	it	can	given	the	
	   	   	   	    budget,	but	the	process	is	sub-optimal	when	considering	all	marketing	spend	together.	
	   	   	   	    The	optimal	process	would	enable	a	fluid	budget	allocation	by	media	depending	on	
	   	   	   	    performance,	opportunity,	and	cross-media	interactions.	Of	course	the	budget	allocated	to	
	   	   	   	    each	media	can	be	changed	over	time	but	the	process	to	do	so	is	typically	arduous,	requiring	
	   	   	   	    an	effort	to	prove	the	performance	benefit	of	a	media	with	respect	to	other	media,	which	
	   	   	   	    becomes	futile	since	there	are	no	common	metrics	to	compare.




                                    Building a next generation marketing measurement system: The framework | © 2008 Merkle   11
Building a next generation marketing measurement
system: the framework


introducing a next generation measurement system
The next generation measurement system forgoes the classical approach of attempting to link a
purchase	to	a	marketing	activity.	Instead,	a	contribution	score	is	created	for	each	factor	that	potentially	
influenced	that	purchase.	The	key	advantage	of	the	probabilistic	contribution	score	(PC	Score)	is	that	
it	bridges	multiple	media/channels	and	points	of	time	for	a	single	purchase.	While	the	scores	have	
little	bearing	on	measuring	media	for	a	single	purchase,	the	reporting	makes	perfect	sense	when	
summarized.

Determining Probabilistic Contributions
The	goal	of	the	next-generation	system	is	to	assign	a	PC	Score	for	every	marketing	communication	that	
a	consumer	may	have	been	exposed	to.	The	sum	of	the	PC	Scores	for	any	given	consumer	purchase	will	
equal	100%.	Depending	on	the	scope	of	the	measurement	system,	PC	Scores	can	be	computed	for	the	
following	categories	and	non-marketing	activities	(the	list	below	is	not	exclusive):
	 •	 Mass	media
	 •	 Direct	media
	 •	 Pricing
	 •	 Promotion	(i.e.	sales	or	other	special	offers)
	 •	 Creative,	messaging,	versioning
	 •	 Brand	awareness	or	baseline	effect
	 •	 Natural	and	economic	environmental	factors
	 •	 Competitive	actions	and	spend
	 •	 Legislative	or	regulatory	changes
	 •	 Service	levels	and	customer	satisfaction	ratings

The scope of categories a particular organization will want to tackle depends on the purpose, amount
of	available	data,	sophistication	of	analytical	skill	set,	and	organizational	readiness.	Media	mix	modeling	
would	be	an	accurate	description	if	this	approach	is	limited	to	mass	and	direct	media.	Marketing	mix	
modeling	is	a	more	suitable	descriptor	if	pricing,	promotion,	and	environmental	factors	are	considered.	
In	our	experience,	just	using	the	model	above	to	incorporate	mass	and	direct	media	alone	will	produce	
a	system	more	advanced	than	the	majority	of	measurement	systems	today.	Merkle	would	recommend	
adding	a	number	of	other	factors	deemed	to	have	the	most	impact.	These	components	usually	include	a	
combination	of	pricing,	promotion,	brand/baseline	effect,	and	competitive	spend.

assuming a system is in place to assign the pC score for every purchase, then extracting meaningful
information	from	the	system	is	a	matter	of	rolling	the	data	up	to	the	appropriate	level.	For	example,	if	
we	just	want	to	know	what	media	provided	the	best	ROI,	we	would	roll	up	the	PC	Scores	associated	
with	every	purchase	over	a	specific	period	of	time.	The	result	will	show	total	number	of	units	sold	for	
each media used. once the total cost for each media is factored in we can get the roi for each media.
A	similar	roll	up	logic	can	be	applied	to	geographies,	customer	segments,	products,	etc.


                              Building a next generation marketing measurement system: The framework | © 2008 Merkle   12
Building a next generation marketing measurement
system: the framework


Probabilistic Contribution Requires an Integrated Approach
The	process	to	assign	PC	Scores	requires	a	combined	top-down	and	bottom-up	approach.	The	process	
is	a	two-stage	process,	with	the	top-down	approach	laying	the	rough	baseline	and	the	bottom-up	
approach	refining	the	weights	where	possible.	

The	process	of	integrating	the	top-down	and	bottom-up	approaches	is	best	understood	through	the	
following	simple	example.	For	simplicity,	assume	the	following	events	take	place:

	 1.	The	marketer	uses	only	two	media	to	sell	their	product,	TV	and	direct	mail.

	 2.	This	company	places	three	spot	TV	advertisements	in	a	particular	region	at	weeks	1,	3,	and	8.	

	 3.	Additionally,	the	company	executes	a	direct	mail	campaign	at	week	6.	

	 4.	A	purchase	is	made	at	week	9.

	 5.	The	company	uses	unique	1-800	numbers	to	track	the	performance	of	every	marketing	activity.

	 6.	The	consumer	who	made	the	purchase	did	not	use	one	of	the	unique	1-800	numbers	available	but	
     was mailed a dM piece 2 weeks prior to purchase.

The	top-down	and	bottom-up	approaches	ultimately	produce	very	different	answers	to	the	question	of	
what marketing activity drove that purchase.

Bottom-up approach scenario
The	bottom-up	approach	would	likely	use	business	rules	around	the	1-800	numbers.	For	example,	the	
following	rules	could	be	applied:

	 1.	If	the	consumer	uses	one	of	the	unique	1-800	numbers	to	make	the	purchase,	then	attribute	that	
	 	 purchase	to	the	marketing	activity	using	that	1-800	number.

	 2.	If	a	number	other	than	one	of	the	unique	1-800	numbers	is	used,	then	check	to	see	if	the	customer	
	 	 was	mailed	a	Direct	Mail	piece	within	the	last	three	months.	If	they	were	mailed	then	attribute	the	
	 	 response	to	the	mail	piece	via	indirect	attribution	process.

	 3.	If	neither	1	or	2	apply,	then	allocate	the	purchase	to	a	brand	awareness	bucket.

Since	the	consumer	did	not	use	a	unique	1-800	number	but	was	mailed	within	the	three	month	time	
period leading up to the purchase, we could conclude that the purchase was due to the direct mail
piece.




                             Building a next generation marketing measurement system: The framework | © 2008 Merkle   13
Building a next generation marketing measurement
system: the framework


top-down approach scenario
Another	way	to	approach	this	process	is	to	use	the	top-down	or	media	mix	modeling	approach.	In	this	
approach,	we	estimate	the	effective	media	exposure	through	an	ad-stock	transformation	(see	figure	
4). The concept is that a marketing activity can produce purchases over an extended period of time.
For	example,	a	TV	advertisement	could	cause	a	consumer	to	call	and	purchase	immediately,	or	could	
lead	to	the	consumer	purchasing	weeks	or	months	later.	Using	ad-stock	transformations	we	create	the	
effective	media	exposures	for	each	media	at	any	given	time.	Figure	4	shows	this	process,	where	the	
red	line	represents	the	effective	TV	exposure	over	time	and	the	blue	line	represents	the	effective	direct	
mail	exposure	over	time.	Since	the	purchase	occurred	at	week	9	we	can	multiply	the	effective	media	
exposure	by	their	coefficients	estimated	through	the	media	mix	models	to	get	probabilities	that	the	
purchase	was	due	to	each	possible	marketing	activity.	In	this	case	we	may	conclude	that	the	purchase	
was	60%	likely	due	to	the	TV	exposures,	10%	due	to	direct	mail	exposure,	and	30%	due	to	neither	(i.e.	
brand	awareness).

Figure 4
  top-down or media mix modeling approach


                                                                                            PurChase made
         effeCtiVe exPosure




                                                        week


The	two	approaches	listed	above	produce	nearly	opposite	answers.	The	top-down	concludes	that	TV	
had	the	biggest	influence	in	the	purchase,	whereas	the	bottom-up	concludes	that	the	purchase	was	due	
to	the	direct	mail	piece.	The	top-down	approach	more	effectively	integrates	the	influence	of	multiple	
media	over	multiple	time	points,	but	fails	to	take	into	account	some	detail-level	information	(i.e.	if	that	




                              Building a next generation marketing measurement system: The framework | © 2008 Merkle   14
Building a next generation marketing measurement
system: the framework


consumer	received	a	mail	piece	or	not).	On	the	other	hand,	the	bottom-up	approach	takes	the	detail-
level	information	into	account	but	uses	rigid	business	rules	to	conclude	an	all	or	nothing	answer.

an integrated approach to measurement
Merkle	suggests	an	approach	that	utilizes	the	best	of	both	methodologies	(see	figure	5).	The	top-down	
approach	is	used	as	the	baseline	since	it	has	the	desirable	properties	of	a	consistent	performance	
metric	across	media	and	naturally	takes	into	account	cross-media	impact	and	impacts	over	time.	But	the	
result	of	the	top-down	approach	is	modified	based	on	the	known,	detail-level	information	available.	In	
the	given	example,	we	would	start	with	the	top-down	solution	but	then	adjust	the	probabilities	by	the	
detail-level	information	known.	For	example,	we	know	two	important	pieces	of	information	about	the	
consumer: first, we know that the consumer did receive a mail piece two weeks prior to purchase, and,
we know that the consumer was scored as a decile two name using a likelihood to purchase predictive
model.	Given	this	information,	the	integrated	outcome	is	50%	likelihood	due	to	DM,	20%	due	to	TV,	
and	30%	due	to	general	brand	awareness.




                            Building a next generation marketing measurement system: The framework | © 2008 Merkle   15
Building a next generation marketing measurement
system: the framework


Figure 5
  an integrated approach utilizes the Best of Both methodologies

         aPProaCh                                                   logiC                                                 result


 Top-Down Approach                                                                    PurChase made
                                                                                                      Likely	responded	to	TV	(60%)
                          effeCtiVe exPosure




                                                                                                      May	have	responded	to	DM	(10%)
 •	Media	mix	modeling                                                                                 Neither	TV	nor	DM	(30%,	i.e.		
 •	Aggregated	data                                                                                    Brand)




                                                                     week
                                                        Quality




 Integrated Approach      Media	exposure	detail	above	AND                                             Likely	responded	to	DM	(50%)
                          Customer	received	mail	piece	at	time	point	6                                May	have	responded	to	TV	(20%)
 •	Top-Down	enhanced	     Customer	was	decile	2	name                                                  Neither	TV	nor	DM	(30%,	i.e.		Brand)
 	 with	Bottom-up         ….
                          ….
                                                                                              integrated
                                                          Quality




 Bottom-up Approach       If	used	unique	1-800	number	then	direct	                                    Responded	to	DM	(100%)
                          attribution	(TV	or	DM)
 •	Business	rules         Else	indirect	attribution	(known	exposure	
 •	Atomic	data            to	DM)
                          Else ….

                          •	Did	receive	mail	piece	within	response	
                          	 window
                          •	Did	not	use	unique	1-800	number
                          •	So	attributed	to	DM	through	indirect	
                          	 attribution	(match-back)


This	integrated	approach	retains	the	key	advantage	of	both	the	top-down	(consistent	metric)	and	
bottom-up	(use	of	detailed	information)	approaches.




                                               Building a next generation marketing measurement system: The framework | © 2008 Merkle   16
Building a next generation marketing measurement
system: the framework


Guidelines for creating PC Scores
The specifics to create the pC scores vary widely depending on the situation, so a description of
a	detailed	process	to	create	the	scores	will	not	be	attempted	here.	Some	general	guidelines	are	
described	below,	listed	by	category.

top-down influence (media mix modeling)
	 •	 Media	mix	models	should	be	fit	at	the	smallest	geographic	level	possible.	DMA-level	model	
     is ideal.

	 •	 Caution	should	be	used	with	respect	to	small	sample	size	for	models	at	a	small	geographic	level.	
	 	 Bayesian	Shrinkage	or	other	adjustment	factors	should	be	considered.

	 •	 It	is	important	to	keep	the	media	mix	models	based	on	as	recent	data	as	possible.	Consider	having	a	
	 	 semi-automated	model	building	process	if	many	models	are	used	(geographic	models,	product	
     models, channel models, etc).

Bottom-up influence (Qualitative data)
	 •	 The	adjustments	to	the	PC	Scores	based	on	qualitative	data	may	be	analytically	driven	or	decided	
	 	 based	on	industry	or	company	experts.	Remember	that	creation	of	PC	Scores	is	part	art	and	part	
     science

	 •	 The	impact	of	qualitative	data	varies	depending	on	the	media.	Knowing	that	somebody	opened	an	
	 	 e-mail	and	clicked	through	to	your	website	is	very	concrete	and	so	would	have	more	impact	than	
     what commercial ran in a dMa (since we don’t even know if that individual saw the commercial).

	 •	 Because	the	qualitative	data	can	change	quickly,	to	have	an	easy	method	to	integrate	new	types	of	
     qualitative data into the models (new creative, for example).

testing and Validation
	 •	 Because	there	is	no	‘right’	answer,	assessing	accuracy	can	be	a	challenging	process.

	 •	 When	possible,	use	controlled	tests	to	validate	influence	of	each	media.	For	example,	a	direct	mail	
	 	 campaign	with	a	random	holdout	sample	will	provide	the	‘true’	influence	of	that	direct	mail	piece	
     since all other factors are controlled for through the randomization.

	 •	 Creditability	is	established	over	time.	Evidence	can	be	gathered	multiple	ways,	the	best	of	which	
	 	 being	when	an	in	market	test	design	produces	the	results	forecast	by	the	media	mix	models.




                             Building a next generation marketing measurement system: The framework | © 2008 Merkle   17
Building a next generation marketing measurement
system: the framework


summary and Conclusions
Marketers	have	an	unprecedented	opportunity	to	increase	the	ROI	on	marketing	programs	by	
implementing	a	next	generation	measurement	system.		By	utilizing	the	best	elements	of	both	the	
high-level,	top-down	approach	and	the	detailed,	bottom-up	approach,	marketers	achieve	a	much	more	
accurate	view	of	their	customers	purchasing	behavior.	

The	top-down	approach	is	used	as	the	baseline	since	it	has	the	desirable	properties	of	a	consistent	
performance	metric	across	media	and	naturally	takes	into	account	cross-media	impact	and	impacts	over	
time.	But	the	result	of	the	top-down	approach	is	modified	based	on	the	known,	detail-level	information	
gleaned	form	the	bottom-up	approach.		The	powerful	combination	of	these	two	measurement	systems	
and	probabilistic	contribution	scores	provides	a	much	clearer	and	reliable	marketing	metrics.		

our next paper will focus on the data, infrastructure, and software toolset required to operationalize the
integrated measurement system. The fourth and final paper will focus on integrating the measurement
framework	into	the	business	decision	making	and	marketing	execution	process	to	ensure	value	is	driven	
from the solution.

How Merkle Can Help
Merkle	works	with	several	clients	to	develop	the	infrastructure	and	analytics	to	enable	the	quantification	
of	brand	engagement	across	their	prospect	and	customer	base	and	make	information-based	decisions	
on	their	brand	equity.	

Merkle	specializes	in	information-based	marketing	strategies	and	is	one	of	the	nation’s	leading	database	
marketing	firms.	With	a	proven	track	record	in	developing	winning	strategies	based	on	information	
insight	for	large	consumer-focused	organizations,	Merkle	works	with	many	of	the	nation’s	leading	
businesses,	including	Procter	&	Gamble,	Dell,	Capital	One,	GEICO,	and	DIRECTV.	




                             Building a next generation marketing measurement system: The framework | © 2008 Merkle
Building a next generation marketing measurement
system: the framework


about the author
scott nuernberger is senior director, Quantitative solutions. scott has over eight years of experience
in developing and implementing analytical solutions to marketing programs for many different
companies,	including	GEICO,	AEGON,	Nationwide	Insurance,	MBNA,	Fidelity,	and	Dell.	Prior	to	joining	
Merkle, scott worked for american express as a statistician and modeler and taught graduate students
statistical methods and experimental design at Cornell university. scott has dual Bs degrees in Brain
and Cognitive sciences and statistics from The university of rochester, a Ms degree in statistics from
Cornell	University,	and	an	MBA	from	Johns	Hopkins	University.




                            Building a next generation marketing measurement system: The framework | © 2008 Merkle   18

Mais conteúdo relacionado

Mais procurados

Key performance indicators for sales
Key performance indicators for salesKey performance indicators for sales
Key performance indicators for salesThe Bitter Business
 
Sales manager kpi examples
Sales manager kpi examplesSales manager kpi examples
Sales manager kpi examplessudaphutagarci
 
Bi in telecom through kpi’s
Bi in telecom through kpi’sBi in telecom through kpi’s
Bi in telecom through kpi’sSai Venkatesh
 
How to Build an End-to-End Process for Defining and Delivering on Outcomes
How to Build an End-to-End Process for Defining and Delivering on OutcomesHow to Build an End-to-End Process for Defining and Delivering on Outcomes
How to Build an End-to-End Process for Defining and Delivering on OutcomesGainsight
 
The Customer Success Metrics That Matter
The Customer Success Metrics That MatterThe Customer Success Metrics That Matter
The Customer Success Metrics That MatterOpsPanda
 
Kpi in telecommunication
Kpi in telecommunicationKpi in telecommunication
Kpi in telecommunicationbaluiabrows
 
How Box Scaled to Millions of Engagements Using Tech Touch
 How Box Scaled to Millions of Engagements Using Tech Touch How Box Scaled to Millions of Engagements Using Tech Touch
How Box Scaled to Millions of Engagements Using Tech TouchGainsight
 
What Does It Really Mean To Build A Customer Culture?
What Does It Really Mean To Build A Customer Culture?What Does It Really Mean To Build A Customer Culture?
What Does It Really Mean To Build A Customer Culture?Totango
 
Configure Price Quote as an Agent for Change
Configure Price Quote as an Agent for ChangeConfigure Price Quote as an Agent for Change
Configure Price Quote as an Agent for ChangeApttus
 
Revenue Operations Analytics: A Strategic Blueprint
Revenue Operations Analytics: A Strategic BlueprintRevenue Operations Analytics: A Strategic Blueprint
Revenue Operations Analytics: A Strategic BlueprintKwanzoo Inc
 
The State of the Customer Success Profession 2018 - Survey Results First Look
The State of the Customer Success Profession 2018 - Survey Results First LookThe State of the Customer Success Profession 2018 - Survey Results First Look
The State of the Customer Success Profession 2018 - Survey Results First LookTotango
 
Considerations and examples for key performance indicators design
Considerations and examples for key performance indicators designConsiderations and examples for key performance indicators design
Considerations and examples for key performance indicators designJAMSO
 
Anaplan SPM webinar series, part 4: Motivating the right behavior with incent...
Anaplan SPM webinar series, part 4: Motivating the right behavior with incent...Anaplan SPM webinar series, part 4: Motivating the right behavior with incent...
Anaplan SPM webinar series, part 4: Motivating the right behavior with incent...Anaplan
 
Customer Engagement Marketing eBook
Customer Engagement Marketing eBookCustomer Engagement Marketing eBook
Customer Engagement Marketing eBookOpsPanda
 
Customer Success with Technical Customers
Customer Success with Technical CustomersCustomer Success with Technical Customers
Customer Success with Technical CustomersGainsight
 
Sales and Operations Planning (S&OP) Overview
Sales and Operations Planning (S&OP) OverviewSales and Operations Planning (S&OP) Overview
Sales and Operations Planning (S&OP) OverviewMichael Ryan
 
What Great Sales & Operations Planning (S&OP) Feels Like!
What Great Sales & Operations Planning (S&OP) Feels Like!What Great Sales & Operations Planning (S&OP) Feels Like!
What Great Sales & Operations Planning (S&OP) Feels Like!Steelwedge
 
Quotient Corporate Profile 2009.11.Desc
Quotient Corporate Profile 2009.11.DescQuotient Corporate Profile 2009.11.Desc
Quotient Corporate Profile 2009.11.DescQuotient Management
 

Mais procurados (20)

KPI Workshop
KPI WorkshopKPI Workshop
KPI Workshop
 
Key performance indicators for sales
Key performance indicators for salesKey performance indicators for sales
Key performance indicators for sales
 
Sales manager kpi examples
Sales manager kpi examplesSales manager kpi examples
Sales manager kpi examples
 
Bi in telecom through kpi’s
Bi in telecom through kpi’sBi in telecom through kpi’s
Bi in telecom through kpi’s
 
How to Build an End-to-End Process for Defining and Delivering on Outcomes
How to Build an End-to-End Process for Defining and Delivering on OutcomesHow to Build an End-to-End Process for Defining and Delivering on Outcomes
How to Build an End-to-End Process for Defining and Delivering on Outcomes
 
The Customer Success Metrics That Matter
The Customer Success Metrics That MatterThe Customer Success Metrics That Matter
The Customer Success Metrics That Matter
 
Kpi in telecommunication
Kpi in telecommunicationKpi in telecommunication
Kpi in telecommunication
 
How Box Scaled to Millions of Engagements Using Tech Touch
 How Box Scaled to Millions of Engagements Using Tech Touch How Box Scaled to Millions of Engagements Using Tech Touch
How Box Scaled to Millions of Engagements Using Tech Touch
 
Marketing Automation 101
Marketing Automation 101Marketing Automation 101
Marketing Automation 101
 
What Does It Really Mean To Build A Customer Culture?
What Does It Really Mean To Build A Customer Culture?What Does It Really Mean To Build A Customer Culture?
What Does It Really Mean To Build A Customer Culture?
 
Configure Price Quote as an Agent for Change
Configure Price Quote as an Agent for ChangeConfigure Price Quote as an Agent for Change
Configure Price Quote as an Agent for Change
 
Revenue Operations Analytics: A Strategic Blueprint
Revenue Operations Analytics: A Strategic BlueprintRevenue Operations Analytics: A Strategic Blueprint
Revenue Operations Analytics: A Strategic Blueprint
 
The State of the Customer Success Profession 2018 - Survey Results First Look
The State of the Customer Success Profession 2018 - Survey Results First LookThe State of the Customer Success Profession 2018 - Survey Results First Look
The State of the Customer Success Profession 2018 - Survey Results First Look
 
Considerations and examples for key performance indicators design
Considerations and examples for key performance indicators designConsiderations and examples for key performance indicators design
Considerations and examples for key performance indicators design
 
Anaplan SPM webinar series, part 4: Motivating the right behavior with incent...
Anaplan SPM webinar series, part 4: Motivating the right behavior with incent...Anaplan SPM webinar series, part 4: Motivating the right behavior with incent...
Anaplan SPM webinar series, part 4: Motivating the right behavior with incent...
 
Customer Engagement Marketing eBook
Customer Engagement Marketing eBookCustomer Engagement Marketing eBook
Customer Engagement Marketing eBook
 
Customer Success with Technical Customers
Customer Success with Technical CustomersCustomer Success with Technical Customers
Customer Success with Technical Customers
 
Sales and Operations Planning (S&OP) Overview
Sales and Operations Planning (S&OP) OverviewSales and Operations Planning (S&OP) Overview
Sales and Operations Planning (S&OP) Overview
 
What Great Sales & Operations Planning (S&OP) Feels Like!
What Great Sales & Operations Planning (S&OP) Feels Like!What Great Sales & Operations Planning (S&OP) Feels Like!
What Great Sales & Operations Planning (S&OP) Feels Like!
 
Quotient Corporate Profile 2009.11.Desc
Quotient Corporate Profile 2009.11.DescQuotient Corporate Profile 2009.11.Desc
Quotient Corporate Profile 2009.11.Desc
 

Semelhante a Next Generation Marketing Measurement

Nine Novel Tactics for Software Product Managment in the New Digital Age
Nine Novel Tactics for Software Product Managment in the New Digital AgeNine Novel Tactics for Software Product Managment in the New Digital Age
Nine Novel Tactics for Software Product Managment in the New Digital AgeCognizant
 
Rethinking the four_ps[1]
Rethinking the four_ps[1]Rethinking the four_ps[1]
Rethinking the four_ps[1]Amanda Knox
 
Analysis of Marketing Mix Modeling – Advantages and Disadvantages
Analysis of Marketing Mix Modeling – Advantages and DisadvantagesAnalysis of Marketing Mix Modeling – Advantages and Disadvantages
Analysis of Marketing Mix Modeling – Advantages and DisadvantagesVirtue Analytics
 
Media mix modeling with social media roi bla
Media mix modeling with social media roi blaMedia mix modeling with social media roi bla
Media mix modeling with social media roi blaBLA101
 
In-Product Marketing: A Game-Changer for Customer-Minded Companies
In-Product Marketing: A Game-Changer for Customer-Minded CompaniesIn-Product Marketing: A Game-Changer for Customer-Minded Companies
In-Product Marketing: A Game-Changer for Customer-Minded CompaniesCognizant
 
ASSIGNMENT PROJECT FRONT SHEET CIM Membership Number Module Title
ASSIGNMENT PROJECT FRONT SHEET CIM Membership Number  Module TitleASSIGNMENT PROJECT FRONT SHEET CIM Membership Number  Module Title
ASSIGNMENT PROJECT FRONT SHEET CIM Membership Number Module TitleAudrey Britton
 
The Smart Cube | Marketing Mix Modeling: An Old Remedy for New Ills
The Smart Cube | Marketing Mix Modeling: An Old Remedy for New IllsThe Smart Cube | Marketing Mix Modeling: An Old Remedy for New Ills
The Smart Cube | Marketing Mix Modeling: An Old Remedy for New IllsMelissa Luongo
 
Meaningful metrics for annual marketing plans
Meaningful metrics for annual marketing plansMeaningful metrics for annual marketing plans
Meaningful metrics for annual marketing plansThomas Emrich
 
IMC Planning Process
IMC Planning ProcessIMC Planning Process
IMC Planning Processgyaanmasti
 
Electronics Digital Strategy03011GBEN.PDF
Electronics Digital Strategy03011GBEN.PDFElectronics Digital Strategy03011GBEN.PDF
Electronics Digital Strategy03011GBEN.PDFMarcel Baron
 
Electronics social marketing_digital strategy
Electronics social marketing_digital strategyElectronics social marketing_digital strategy
Electronics social marketing_digital strategyMarcel Baron
 
EY Report_Digital Marketing Plan
EY Report_Digital Marketing PlanEY Report_Digital Marketing Plan
EY Report_Digital Marketing PlanNikoletta Kaiara
 
Introduction to marketing analytics.pptx
Introduction to marketing analytics.pptxIntroduction to marketing analytics.pptx
Introduction to marketing analytics.pptxIshuGupta84
 
MKT 571 TUTOR Begins Education / www.mkt571edu.com
MKT 571 TUTOR Begins Education / www.mkt571edu.comMKT 571 TUTOR Begins Education / www.mkt571edu.com
MKT 571 TUTOR Begins Education / www.mkt571edu.comSpencerLStuart
 
Know Your Data 1. The ArnetMiner citation dataset (provide.docx
Know Your Data 1. The ArnetMiner citation dataset (provide.docxKnow Your Data 1. The ArnetMiner citation dataset (provide.docx
Know Your Data 1. The ArnetMiner citation dataset (provide.docxDIPESH30
 
Results from recent key projects
Results from recent key projectsResults from recent key projects
Results from recent key projectsWilliam Tangalos
 
Responsys forrester the-rise_of_the_customer
Responsys forrester the-rise_of_the_customerResponsys forrester the-rise_of_the_customer
Responsys forrester the-rise_of_the_customerJenna Beck
 
Econometrics Explained - IPA Report
Econometrics Explained - IPA ReportEconometrics Explained - IPA Report
Econometrics Explained - IPA ReportThink Ethnic
 

Semelhante a Next Generation Marketing Measurement (20)

Nine Novel Tactics for Software Product Managment in the New Digital Age
Nine Novel Tactics for Software Product Managment in the New Digital AgeNine Novel Tactics for Software Product Managment in the New Digital Age
Nine Novel Tactics for Software Product Managment in the New Digital Age
 
OLD SPICE
OLD SPICEOLD SPICE
OLD SPICE
 
Marketing information system
Marketing information systemMarketing information system
Marketing information system
 
Rethinking the four_ps[1]
Rethinking the four_ps[1]Rethinking the four_ps[1]
Rethinking the four_ps[1]
 
Analysis of Marketing Mix Modeling – Advantages and Disadvantages
Analysis of Marketing Mix Modeling – Advantages and DisadvantagesAnalysis of Marketing Mix Modeling – Advantages and Disadvantages
Analysis of Marketing Mix Modeling – Advantages and Disadvantages
 
Media mix modeling with social media roi bla
Media mix modeling with social media roi blaMedia mix modeling with social media roi bla
Media mix modeling with social media roi bla
 
In-Product Marketing: A Game-Changer for Customer-Minded Companies
In-Product Marketing: A Game-Changer for Customer-Minded CompaniesIn-Product Marketing: A Game-Changer for Customer-Minded Companies
In-Product Marketing: A Game-Changer for Customer-Minded Companies
 
ASSIGNMENT PROJECT FRONT SHEET CIM Membership Number Module Title
ASSIGNMENT PROJECT FRONT SHEET CIM Membership Number  Module TitleASSIGNMENT PROJECT FRONT SHEET CIM Membership Number  Module Title
ASSIGNMENT PROJECT FRONT SHEET CIM Membership Number Module Title
 
The Smart Cube | Marketing Mix Modeling: An Old Remedy for New Ills
The Smart Cube | Marketing Mix Modeling: An Old Remedy for New IllsThe Smart Cube | Marketing Mix Modeling: An Old Remedy for New Ills
The Smart Cube | Marketing Mix Modeling: An Old Remedy for New Ills
 
Meaningful metrics for annual marketing plans
Meaningful metrics for annual marketing plansMeaningful metrics for annual marketing plans
Meaningful metrics for annual marketing plans
 
IMC Planning Process
IMC Planning ProcessIMC Planning Process
IMC Planning Process
 
Electronics Digital Strategy03011GBEN.PDF
Electronics Digital Strategy03011GBEN.PDFElectronics Digital Strategy03011GBEN.PDF
Electronics Digital Strategy03011GBEN.PDF
 
Electronics social marketing_digital strategy
Electronics social marketing_digital strategyElectronics social marketing_digital strategy
Electronics social marketing_digital strategy
 
EY Report_Digital Marketing Plan
EY Report_Digital Marketing PlanEY Report_Digital Marketing Plan
EY Report_Digital Marketing Plan
 
Introduction to marketing analytics.pptx
Introduction to marketing analytics.pptxIntroduction to marketing analytics.pptx
Introduction to marketing analytics.pptx
 
MKT 571 TUTOR Begins Education / www.mkt571edu.com
MKT 571 TUTOR Begins Education / www.mkt571edu.comMKT 571 TUTOR Begins Education / www.mkt571edu.com
MKT 571 TUTOR Begins Education / www.mkt571edu.com
 
Know Your Data 1. The ArnetMiner citation dataset (provide.docx
Know Your Data 1. The ArnetMiner citation dataset (provide.docxKnow Your Data 1. The ArnetMiner citation dataset (provide.docx
Know Your Data 1. The ArnetMiner citation dataset (provide.docx
 
Results from recent key projects
Results from recent key projectsResults from recent key projects
Results from recent key projects
 
Responsys forrester the-rise_of_the_customer
Responsys forrester the-rise_of_the_customerResponsys forrester the-rise_of_the_customer
Responsys forrester the-rise_of_the_customer
 
Econometrics Explained - IPA Report
Econometrics Explained - IPA ReportEconometrics Explained - IPA Report
Econometrics Explained - IPA Report
 

Mais de Amita Paul

Amita paul vc2018_final
Amita paul vc2018_finalAmita paul vc2018_final
Amita paul vc2018_finalAmita Paul
 
Maximize your Event Management with ObjectiveMarketer
Maximize your Event Management with ObjectiveMarketerMaximize your Event Management with ObjectiveMarketer
Maximize your Event Management with ObjectiveMarketerAmita Paul
 
ObjectiveMarketer Overview
ObjectiveMarketer OverviewObjectiveMarketer Overview
ObjectiveMarketer OverviewAmita Paul
 
ObjectiveMarketer Introduction
ObjectiveMarketer IntroductionObjectiveMarketer Introduction
ObjectiveMarketer IntroductionAmita Paul
 
Getting Started Tutorial
Getting Started TutorialGetting Started Tutorial
Getting Started TutorialAmita Paul
 
Report: SXSW 2010 - What Twitter #Tags Reveal
Report: SXSW 2010 - What Twitter #Tags RevealReport: SXSW 2010 - What Twitter #Tags Reveal
Report: SXSW 2010 - What Twitter #Tags RevealAmita Paul
 
Anderson analytics meng_2010_trend_report_final
Anderson analytics meng_2010_trend_report_finalAnderson analytics meng_2010_trend_report_final
Anderson analytics meng_2010_trend_report_finalAmita Paul
 

Mais de Amita Paul (7)

Amita paul vc2018_final
Amita paul vc2018_finalAmita paul vc2018_final
Amita paul vc2018_final
 
Maximize your Event Management with ObjectiveMarketer
Maximize your Event Management with ObjectiveMarketerMaximize your Event Management with ObjectiveMarketer
Maximize your Event Management with ObjectiveMarketer
 
ObjectiveMarketer Overview
ObjectiveMarketer OverviewObjectiveMarketer Overview
ObjectiveMarketer Overview
 
ObjectiveMarketer Introduction
ObjectiveMarketer IntroductionObjectiveMarketer Introduction
ObjectiveMarketer Introduction
 
Getting Started Tutorial
Getting Started TutorialGetting Started Tutorial
Getting Started Tutorial
 
Report: SXSW 2010 - What Twitter #Tags Reveal
Report: SXSW 2010 - What Twitter #Tags RevealReport: SXSW 2010 - What Twitter #Tags Reveal
Report: SXSW 2010 - What Twitter #Tags Reveal
 
Anderson analytics meng_2010_trend_report_final
Anderson analytics meng_2010_trend_report_finalAnderson analytics meng_2010_trend_report_final
Anderson analytics meng_2010_trend_report_final
 

Next Generation Marketing Measurement

  • 1. Building a next generation marketing measurement system: the framework M er kl e Tho u g hT l eader s hip se r i e s A Database Marketing Agency
  • 2. Building a next generation marketing measurement system: the framework executive summary: marketers struggle more than ever to develop accurate, reliable, and consistent metrics around the performance of marketing spend. relying on incorrect metrics creates a snowball effect that compounds measurement errors along the entire marketing decision process. decisions on how much to spend on marketing in total, how much to spend by media, and how to coordinate a multi-media program are all directly influenced by the metrics used. for these reasons, the importance of evaluating your current measurement framework and building a solid foundation can’t be understated. n Marketers already employ processes to link a purchase with a marketing activity to create metrics on marketing performance. These marketing measurement systems fall into one of two camps, each with its own shortcomings: • Top-down attribution approaches that provide such a high level view that they fail to be actionable, or • Bottom-up attribution approaches that present a collection of very detailed information but fail to integrate into a single view. Better solutions do exist and can be achieved through sophisticated analytical and technological techniques. Merkle introduces a next generation measurement system based on Probabilistic Contribution (PC) scores (see definition on page 5). rather than attempting to link a purchase to a marketing activity, a contribution score is created, by integrating top-down and bottom-up measurement systems, for each factor that potentially influenced the purchase. The key advantage is integration - bridging multiple media/channels and points of time for a single purchase to provide a comprehensive view of the consumption pattern. This white paper, the second in a series of four on marketing optimization, addresses the critical elements needed to build a next generation marketing measurement framework. A third paper will discuss the data, infrastructure, and software tool requirements. and a fourth and final white paper will outline how to effectively integrate this system throughout the marketing decision making process. Building a next generation marketing measurement system: The framework | © 2008 Merkle 2
  • 3. Building a next generation marketing measurement system: the framework five factors fuel marketing’s roi dilemma What makes answering such fundamental questions around the ROI for marketing spend so difficult? Some of the main reasons for this include: 1. Consumers face daily ad bombardment. studies consistently show that consumers are exposed to a mind-numbing one to two thousand advertising impressions per day. The sheer volume of advertising makes drawing a line from messaging to consumer activity all the more difficult. 2. fragmentation of media. The nearly monolithic media channels of previous decades have splintered in the age of the internet, on-demand television, and satellite radio. For instance, the number of television channels increased over a hundred times in the last thirty years. The growing consumer trend of multitasking while accessing multiple media – especially prevalent in young consumers – throws a wrench into marketers’ ability to attribute action to marketing message. 3. failure to consider the complexity of consumers’ decision process. a purchase is not directly tied to the most recent advertising exposure, despite the fact that most organizations attribute sales to marketing spend in this manner today. purchase decisions are exceedingly complex and consumers are likely to be influenced by multiple advertisements over time and different types of media. 4. Consumers are in control. Consumers have more choice than ever over when and how they consume media, making it that much more difficult for marketers to even know if a message was received. 5. organizational challenges. Most marketing groups organize by media type creating a silo effect (e.g. brand marketing, direct marketing, and interactive/online marketing). These silos each develop their own goals, metrics, infrastructures, and partners to maximize the impact within the media they can control. Building a next generation marketing measurement system: The framework | © 2008 Merkle 3
  • 4. Building a next generation marketing measurement system: the framework Critical elements of marketing measurement Marketing organizations require the following key elements to effectively build and implement next generation marketing measurement: 1. Analytical acumen to custom-build a solution to link all marketing activities to consistent performance metrics (i.e. measurement framework) 2. Data availability: Access to a complete, accurate, and frequently updated marketing data universe 3. Infrastructure (Media management database) 4. Integration into business decision making (belief/buy-in and integration through software tools) and integration across all lines of business. Terms Used In This Paper Standardized language will be used in this paper to provide relevance across different industries. A few notable cases are highlighted below. Standardized terms are defined as follows: outcome of interest – This paper will use “purchase” to represent a generic term that could be applied to any performance measurement a company may use. Throughout the discussion, purchase can easily be replaced with response, new customer, revenue, profit, LTV, or any other metric that may be used to calculate desired outcomes. While it is true that some details related to techniques and implementation may change depending on the metric used, the discussion throughout this paper will be broad enough to encompass any of these metrics. media/Channel – A distinction will be made between media and channel, with media representing communication flowing from the company to the consumer, and channel representing communication from the consumer to the company. Additionally, media will be discussed in three classes; direct, mass, and interactive. top-down and Bottom-up – The top-down and bottom-up approaches are designed to represent two common ways marketing organizations measure the performance of their media. The top-down approach starts with all media spend and then breaks the media spend into media, region, and timeframe. This is the broadest perspective commonly used in media mix analyses, but rarely takes very detailed information into account (individual level data, creative, quality of advertising, etc.). in contrast, the bottom-up approach starts with the very detailed level of information. This approach is common in direct and interactive media analyses (consumer segments, predictive models, decile analyses, etc). Building a next generation marketing measurement system: The framework | © 2008 Merkle 4
  • 5. Building a next generation marketing measurement system: the framework attribution – The term “attribution” represents any process designed to link a marketing activity with an outcome of interest (i.e. purchase). Typically, an attribution process attempts to link the purchase with a single marketing activity or campaign. More sophisticated, and less common, attribution processes apply probabilities across marketing activities to capture cross-media influence more accurately. This paper outlines a new approach to marketing measurement by: 1. Setting the stage through a discussion of typical attribution processes and the current challenges of marketing measurement systems. 2. introducing analytical framework for the next generation marketing measurement system. This paper describes a new metric called Probabilistic Contribution score (PC score) used as the key metric to link marketing activities with the outcomes of interest. 3. providing guidelines on the analytical approach required to calculate pC scores across all marketing activities. Probabilistic Contribution Score: The Probabilistic Contribution Score (PC Score) is a value assigned to any activity or event (marketing or otherwise) that influences the consumer toward a desired action (i.e. purchase a product). PC Score derives its name from: • Probabilistic – The sum of all the values for any given purchase equals • Contribution – Each value represents the relative contribution or influence specific activities or events have on consumers’ decision making process • Score – The values are designed to be informative within a given purchase or when rolled up to summarized reports Current Challenges facing marketing measurement systems If asked, many organizations claim to have robust and reliable processes to attribute sales to media spend. Forrester Research conducted a survey in 2008 that concluded that 69% of marketers believe they are at least somewhat effective at measuring marketing ROI. It has been Merkle’s experience, however, that after digging beneath the surface, most marketing measurement systems fall far short of robust or reliable, especially when trying to forecast marketing impact into the future. The 2008 ANA/ MMA Marketing Accountability Survey showed that only 10% of marketers felt they could forecast the effect of a 10% cut in budget, and just 14% said that senior management in their companies had confidence in their firms’ marketing forecasts. CFO’s are even more skeptical of the numbers. Ninety percent of CFO’s don’t use marketing ROI metrics to help set marketing budgets. “They don’t believe the numbers,” said Jeffrey Marshall, the retired editor in chief of Financial Executive magazine. Building a next generation marketing measurement system: The framework | © 2008 Merkle 5
  • 6. Building a next generation marketing measurement system: the framework Another recent Forrester study highlights some of the top barriers marketing organizations currently face to improving marketing ROI (see figure 1). Many of the challenges Forrester identifies are directly linked to the four challenges addressed by this framework (analytic sophistication, data access and accuracy, technology, common metrics and business integration). Figure 1 marketers face hurdles in staffing, data, technology, and common metrics Source: Forrester Research, “Database Marketers Evolve Their ROI Measurement” Building a next generation marketing measurement system: The framework | © 2008 Merkle 6
  • 7. Building a next generation marketing measurement system: the framework two Common approaches to marketing measurement Marketers already employ processes to link a purchase to a marketing activity to create metrics on performance. These marketing measurement systems fall into two camps: • Top-down attribution process - providing such a high level view that they fail to be actionable, or • Bottom-up attribution process - presenting a collection of very detailed systems that fail to integrate into a single view. Overview of typical top-down attribution processes The top-down approach starts with the entire marketing budget and then looks at how the marketing budget is split between major categories (media, regions, products, etc). The advantage of this approach is that it provides a framework to compare the performance of all marketing spend with common performance metrics to one another. This property is crucial to the building of a robust marketing measurement system, and is the reason why the approach outlined later in this paper uses the top-down approach as the starting point. This type of analysis is often termed media mix analysis. Figure 2 top-down and Bottom-up approaches Building a next generation marketing measurement system: The framework | © 2008 Merkle 7
  • 8. Building a next generation marketing measurement system: the framework Certain industries, like consumer product goods and pharmaceutical companies, have a long history of using media mix modeling to help determine marketing spend. These industries tend to have in-direct relationship to their consumers and involve purchasing of less expensive products conducive to a short purchase consideration time (see figure 3). These industries developed media mix modeling capabilities early on partly because it was one of the few ways to understand marketing performance when there is no direct to consumer relationship. These industries also benefit from standardized data sources (like pos data) which have allowed standardized media mix modeling methodologies to develop. as relationships move more towards direct-to-consumer and the purchase consideration increases, the data and methodology required for media mix modeling becomes more customized for each company. This customization requires more investment to develop, and therefore it is typically the companies with significant marketing spend that can justify the modeling work. Figure 3 Choosing the measurement approach PurChase Consideration time most established top-down measurement systems non-direCt direCt to Consumer *Note – the company placements above are for illustration purposes only. In some cases a company can be put in multiple areas due to different product offerings. For example, a Dell desktop computer would be in the top right (as shown) whereas a replacement ink cartridge for Dell printers would be in the lower right. Building a next generation marketing measurement system: The framework | © 2008 Merkle 8
  • 9. Building a next generation marketing measurement system: the framework However, there are some significant weaknesses to the top-down approach, including: 1. it takes too long. it is not uncommon for this type of analysis to take over six months to complete. The deliverable is often a presentation of how media performed for the analysis time period along with recommended changes. By the time the recommendations are presented, the marketing landscape may have already changed, making the analysis outdated before it is even presented. Another way to view this pitfall is that media mix work is often managed on a project basis, as opposed to an ongoing system. 2. it fails to take into account the specifics within each media. There are a number of options and choices when spending dollars within a media. If the budget for that media is being cut back due to historical poor performance, then perhaps the allocation of spend within the media should be changed, not necessarily the size of the budget. For example, there are many decisions and analyse within a direct mail program that are not adequately summarized by simply when just total dollars spent. 3. lack of integration with media execution. Media mix analyses often conclude with recommendations on how to shift marketing dollars between available media. It is rare for these analyses to address the complexities associated with the execution of each media. of particular importance are inventory and cost constraints. For example, a recommendation to increase spend in spot TV may require introducing new day parts or networks which typically changes the price to purchase the media. as cost per impression increases, the relative effectiveness of that media is likely going to decrease. These weaknesses can be summarized by the project vs. process nature of the analyses. These analyses are often conducted ad-hoc without becoming part of the marketing measurements systems. By not being fully implemented into the marketing culture, these types of results rank as merely interesting – but not relevant enough to change marketing strategy and spend. Additionally, because the top-down measurement approach fails to take specifics into account there is a need for the more detailed bottom- up approaches. Overview of bottom-up typical attribution processes The bottom-up approach starts with detailed information within a media and then establishes a system to measure performance of segments or campaigns within the media. The nature of the data and the type of metrics calculated vary depending on the media discussed (which is part of the problem). Some metrics are efficiency based (cost per impression) while others are performance based (cost per purchase, ROI). It is more common to see performance based metrics available where it is easier to connect media spend directly to individual level purchases. Building a next generation marketing measurement system: The framework | © 2008 Merkle 9
  • 10. Building a next generation marketing measurement system: the framework Direct and indirect attribution are two commonly used practices as the foundation for bottom-up type metrics. A brief overview along with some examples and pitfalls are highlighted below. direct attribution is often classified as an attribution process that uses some kind of unique identifier to link a purchase to a specific marketing activity. examples include asking customers to provide a specific code from the back of their catalog , a unique 1-800 number that is only used for a particular run of a television advertisement, or tracking a click on a banner ad by directing to a specific landing page. While each of the examples above has their own merit, they ultimately fail in producing a reliable attribution process. They fail because the direct attribution is only recording the last marketing exposure before the purchase, but not necessarily what is driving the purchase intent in the first place. A common example is unique 1-800’s on TV ads. A TV ad may drive desire to purchase, but people don’t use the unique 1-800 number on the ad, but rather search for the product on Google, go to a landing page, and then use that unique 1-800 number. The TV ad is given less credit than it deserves, and search engine marketing gets more credit than it deserves. indirect attribution, or business rule attribution, uses some assumptions to link a responder to a marketing effort. For example, a unique 1-800 number is not used but we know that the consumer received a catalog two weeks prior to the purchase. In this case we use indirect attribution to assume the catalog drove the purchase. This process has some advantages and disadvantages over direct attribution. Indirect attribution allows a marketer to attribute a purchase when direct attribution is not possible. However, indirect attribution rules come down to little more than an educated guess. How long should the time window be? What if we know the consumer received a mail piece one week ago? One month ago? Six months ago? This decision will drastically impact the perceived performance of the direct mail campaigns. even worse, the pattern feeds on itself (up to a point). if it looks like direct mail performs well then a company will do more direct mail, which leads to an even higher likelihood that a consumer who purchases your product received a mail piece within the time window. This assumption inflates direct mail performance metrics even further. eventually the direct mail performance metrics will drop off as volume increases, leading to a perceived equilibrium relative to other marketing options. However this equilibrium is typically far from the true equilibrium. a common pitfall for organizations is the sense of sophistication associated with very complex direct and indirect attribution rules. The attribution rules continually become more refined and complicated, leading the marketer to believe they have a world-class marketing attribution system and therefore are doing very well at linking purchases to marketing activities. Building a next generation marketing measurement system: The framework | © 2008 Merkle 10
  • 11. Building a next generation marketing measurement system: the framework Most of the problems associated with direct and indirect attribution systems can be placed into two general categories: cross-media/channel tracking and effects over time. 1. Cross-media/channel tracking – Most attribution systems attempt to link a purchase to a single marketing communication, whereas in reality consumers do not just use one source of information to make a purchasing decision. 2. effects over time – multiple advertising exposures may contribute to the ultimate purchase even within a single media, but most attribution systems will just attribute the purchase to a single instance. Current marketing measurement systems ultimately leave CMOs with two problems: • an inability to see the whole picture. Today’s systems force CMOs to look at either broad brush metrics that fail to consider specifics of individual media or at detailed metrics within a media that fail to translate into metrics that can be compared across media. • a disconnect between budget planning and execution. Typically, the budget planning process has two steps: first, each media or channel is allocated a certain budget to spend and second, each media or channel attempts to gain the best performance it can from the allocated budget. This process effectively creates a major planning constraint where the budget by media is considered fixed. Each media may be doing the best it can given the budget, but the process is sub-optimal when considering all marketing spend together. The optimal process would enable a fluid budget allocation by media depending on performance, opportunity, and cross-media interactions. Of course the budget allocated to each media can be changed over time but the process to do so is typically arduous, requiring an effort to prove the performance benefit of a media with respect to other media, which becomes futile since there are no common metrics to compare. Building a next generation marketing measurement system: The framework | © 2008 Merkle 11
  • 12. Building a next generation marketing measurement system: the framework introducing a next generation measurement system The next generation measurement system forgoes the classical approach of attempting to link a purchase to a marketing activity. Instead, a contribution score is created for each factor that potentially influenced that purchase. The key advantage of the probabilistic contribution score (PC Score) is that it bridges multiple media/channels and points of time for a single purchase. While the scores have little bearing on measuring media for a single purchase, the reporting makes perfect sense when summarized. Determining Probabilistic Contributions The goal of the next-generation system is to assign a PC Score for every marketing communication that a consumer may have been exposed to. The sum of the PC Scores for any given consumer purchase will equal 100%. Depending on the scope of the measurement system, PC Scores can be computed for the following categories and non-marketing activities (the list below is not exclusive): • Mass media • Direct media • Pricing • Promotion (i.e. sales or other special offers) • Creative, messaging, versioning • Brand awareness or baseline effect • Natural and economic environmental factors • Competitive actions and spend • Legislative or regulatory changes • Service levels and customer satisfaction ratings The scope of categories a particular organization will want to tackle depends on the purpose, amount of available data, sophistication of analytical skill set, and organizational readiness. Media mix modeling would be an accurate description if this approach is limited to mass and direct media. Marketing mix modeling is a more suitable descriptor if pricing, promotion, and environmental factors are considered. In our experience, just using the model above to incorporate mass and direct media alone will produce a system more advanced than the majority of measurement systems today. Merkle would recommend adding a number of other factors deemed to have the most impact. These components usually include a combination of pricing, promotion, brand/baseline effect, and competitive spend. assuming a system is in place to assign the pC score for every purchase, then extracting meaningful information from the system is a matter of rolling the data up to the appropriate level. For example, if we just want to know what media provided the best ROI, we would roll up the PC Scores associated with every purchase over a specific period of time. The result will show total number of units sold for each media used. once the total cost for each media is factored in we can get the roi for each media. A similar roll up logic can be applied to geographies, customer segments, products, etc. Building a next generation marketing measurement system: The framework | © 2008 Merkle 12
  • 13. Building a next generation marketing measurement system: the framework Probabilistic Contribution Requires an Integrated Approach The process to assign PC Scores requires a combined top-down and bottom-up approach. The process is a two-stage process, with the top-down approach laying the rough baseline and the bottom-up approach refining the weights where possible. The process of integrating the top-down and bottom-up approaches is best understood through the following simple example. For simplicity, assume the following events take place: 1. The marketer uses only two media to sell their product, TV and direct mail. 2. This company places three spot TV advertisements in a particular region at weeks 1, 3, and 8. 3. Additionally, the company executes a direct mail campaign at week 6. 4. A purchase is made at week 9. 5. The company uses unique 1-800 numbers to track the performance of every marketing activity. 6. The consumer who made the purchase did not use one of the unique 1-800 numbers available but was mailed a dM piece 2 weeks prior to purchase. The top-down and bottom-up approaches ultimately produce very different answers to the question of what marketing activity drove that purchase. Bottom-up approach scenario The bottom-up approach would likely use business rules around the 1-800 numbers. For example, the following rules could be applied: 1. If the consumer uses one of the unique 1-800 numbers to make the purchase, then attribute that purchase to the marketing activity using that 1-800 number. 2. If a number other than one of the unique 1-800 numbers is used, then check to see if the customer was mailed a Direct Mail piece within the last three months. If they were mailed then attribute the response to the mail piece via indirect attribution process. 3. If neither 1 or 2 apply, then allocate the purchase to a brand awareness bucket. Since the consumer did not use a unique 1-800 number but was mailed within the three month time period leading up to the purchase, we could conclude that the purchase was due to the direct mail piece. Building a next generation marketing measurement system: The framework | © 2008 Merkle 13
  • 14. Building a next generation marketing measurement system: the framework top-down approach scenario Another way to approach this process is to use the top-down or media mix modeling approach. In this approach, we estimate the effective media exposure through an ad-stock transformation (see figure 4). The concept is that a marketing activity can produce purchases over an extended period of time. For example, a TV advertisement could cause a consumer to call and purchase immediately, or could lead to the consumer purchasing weeks or months later. Using ad-stock transformations we create the effective media exposures for each media at any given time. Figure 4 shows this process, where the red line represents the effective TV exposure over time and the blue line represents the effective direct mail exposure over time. Since the purchase occurred at week 9 we can multiply the effective media exposure by their coefficients estimated through the media mix models to get probabilities that the purchase was due to each possible marketing activity. In this case we may conclude that the purchase was 60% likely due to the TV exposures, 10% due to direct mail exposure, and 30% due to neither (i.e. brand awareness). Figure 4 top-down or media mix modeling approach PurChase made effeCtiVe exPosure week The two approaches listed above produce nearly opposite answers. The top-down concludes that TV had the biggest influence in the purchase, whereas the bottom-up concludes that the purchase was due to the direct mail piece. The top-down approach more effectively integrates the influence of multiple media over multiple time points, but fails to take into account some detail-level information (i.e. if that Building a next generation marketing measurement system: The framework | © 2008 Merkle 14
  • 15. Building a next generation marketing measurement system: the framework consumer received a mail piece or not). On the other hand, the bottom-up approach takes the detail- level information into account but uses rigid business rules to conclude an all or nothing answer. an integrated approach to measurement Merkle suggests an approach that utilizes the best of both methodologies (see figure 5). The top-down approach is used as the baseline since it has the desirable properties of a consistent performance metric across media and naturally takes into account cross-media impact and impacts over time. But the result of the top-down approach is modified based on the known, detail-level information available. In the given example, we would start with the top-down solution but then adjust the probabilities by the detail-level information known. For example, we know two important pieces of information about the consumer: first, we know that the consumer did receive a mail piece two weeks prior to purchase, and, we know that the consumer was scored as a decile two name using a likelihood to purchase predictive model. Given this information, the integrated outcome is 50% likelihood due to DM, 20% due to TV, and 30% due to general brand awareness. Building a next generation marketing measurement system: The framework | © 2008 Merkle 15
  • 16. Building a next generation marketing measurement system: the framework Figure 5 an integrated approach utilizes the Best of Both methodologies aPProaCh logiC result Top-Down Approach PurChase made Likely responded to TV (60%) effeCtiVe exPosure May have responded to DM (10%) • Media mix modeling Neither TV nor DM (30%, i.e. • Aggregated data Brand) week Quality Integrated Approach Media exposure detail above AND Likely responded to DM (50%) Customer received mail piece at time point 6 May have responded to TV (20%) • Top-Down enhanced Customer was decile 2 name Neither TV nor DM (30%, i.e. Brand) with Bottom-up …. …. integrated Quality Bottom-up Approach If used unique 1-800 number then direct Responded to DM (100%) attribution (TV or DM) • Business rules Else indirect attribution (known exposure • Atomic data to DM) Else …. • Did receive mail piece within response window • Did not use unique 1-800 number • So attributed to DM through indirect attribution (match-back) This integrated approach retains the key advantage of both the top-down (consistent metric) and bottom-up (use of detailed information) approaches. Building a next generation marketing measurement system: The framework | © 2008 Merkle 16
  • 17. Building a next generation marketing measurement system: the framework Guidelines for creating PC Scores The specifics to create the pC scores vary widely depending on the situation, so a description of a detailed process to create the scores will not be attempted here. Some general guidelines are described below, listed by category. top-down influence (media mix modeling) • Media mix models should be fit at the smallest geographic level possible. DMA-level model is ideal. • Caution should be used with respect to small sample size for models at a small geographic level. Bayesian Shrinkage or other adjustment factors should be considered. • It is important to keep the media mix models based on as recent data as possible. Consider having a semi-automated model building process if many models are used (geographic models, product models, channel models, etc). Bottom-up influence (Qualitative data) • The adjustments to the PC Scores based on qualitative data may be analytically driven or decided based on industry or company experts. Remember that creation of PC Scores is part art and part science • The impact of qualitative data varies depending on the media. Knowing that somebody opened an e-mail and clicked through to your website is very concrete and so would have more impact than what commercial ran in a dMa (since we don’t even know if that individual saw the commercial). • Because the qualitative data can change quickly, to have an easy method to integrate new types of qualitative data into the models (new creative, for example). testing and Validation • Because there is no ‘right’ answer, assessing accuracy can be a challenging process. • When possible, use controlled tests to validate influence of each media. For example, a direct mail campaign with a random holdout sample will provide the ‘true’ influence of that direct mail piece since all other factors are controlled for through the randomization. • Creditability is established over time. Evidence can be gathered multiple ways, the best of which being when an in market test design produces the results forecast by the media mix models. Building a next generation marketing measurement system: The framework | © 2008 Merkle 17
  • 18. Building a next generation marketing measurement system: the framework summary and Conclusions Marketers have an unprecedented opportunity to increase the ROI on marketing programs by implementing a next generation measurement system. By utilizing the best elements of both the high-level, top-down approach and the detailed, bottom-up approach, marketers achieve a much more accurate view of their customers purchasing behavior. The top-down approach is used as the baseline since it has the desirable properties of a consistent performance metric across media and naturally takes into account cross-media impact and impacts over time. But the result of the top-down approach is modified based on the known, detail-level information gleaned form the bottom-up approach. The powerful combination of these two measurement systems and probabilistic contribution scores provides a much clearer and reliable marketing metrics. our next paper will focus on the data, infrastructure, and software toolset required to operationalize the integrated measurement system. The fourth and final paper will focus on integrating the measurement framework into the business decision making and marketing execution process to ensure value is driven from the solution. How Merkle Can Help Merkle works with several clients to develop the infrastructure and analytics to enable the quantification of brand engagement across their prospect and customer base and make information-based decisions on their brand equity. Merkle specializes in information-based marketing strategies and is one of the nation’s leading database marketing firms. With a proven track record in developing winning strategies based on information insight for large consumer-focused organizations, Merkle works with many of the nation’s leading businesses, including Procter & Gamble, Dell, Capital One, GEICO, and DIRECTV. Building a next generation marketing measurement system: The framework | © 2008 Merkle
  • 19. Building a next generation marketing measurement system: the framework about the author scott nuernberger is senior director, Quantitative solutions. scott has over eight years of experience in developing and implementing analytical solutions to marketing programs for many different companies, including GEICO, AEGON, Nationwide Insurance, MBNA, Fidelity, and Dell. Prior to joining Merkle, scott worked for american express as a statistician and modeler and taught graduate students statistical methods and experimental design at Cornell university. scott has dual Bs degrees in Brain and Cognitive sciences and statistics from The university of rochester, a Ms degree in statistics from Cornell University, and an MBA from Johns Hopkins University. Building a next generation marketing measurement system: The framework | © 2008 Merkle 18