This is a revision to a presentation I gave at Hyper Island Master Class, and shared with a lunch & learn session hosted by Prudential Ad

1. MEASURE WHAT MATTERS
understanding measurement & roi for digital experiences

2. @FARRAHBOSTIC

3. 4 THINGS:
WHY
WHAT
HOW
(AND HOW TO SELL IT)

4. 1. WHY YOU SHOULD MEASURE

5. BINGO
TIME
SIGN-
KPI UPS
ON CPM IMPRESSIONS
SITE
CPA CPC REACH &
FREQUENCY LIKES ENGAGEMENT
PAGE
LTV SENTIMENT
ANALYSIS ROI VIEWS FOLLOWS
MENTIONS CTR DOWNLOADS
BRAND MEDIA MIX
ATTRIBUTES MODELING
SHARE
SALIENCE LIKEABILITY RECALL OF ??
WALLET

6. A WORD OF CAUTION
I notice increasing reluctance on
the part of marketing executives
to use judgment; they are coming
to rely too much on research, and
they use it as a drunkard uses a
lamp post: for support, rather
than for illumination.
David Ogilvy

7. WHY DO METRICS MATTER?
1. Metrics reduce arguments based on opinion.
2. Metrics give you answers about what really works.
3. Metrics show you where you’re strong.
4. Metrics allow you to test anything you want.
5. Clients love metrics.
http://bokardo.com/talks/metrics-driven-design-sxsw.pdf

8. WHAT ELSE CAN MEASUREMENT HELP YOU DO?
UNDERSTAND CONTEXT OF BRAND & PRODUCT
DETECT DEMAND EXPLORE CONSUMER SENTIMENT
MENTIONS
Now we can easily assess pre- Spontaneous product comments Digital makes it is easier to
existing demand among our target: and mentions reveal sentiment detect problems or possible
how easy is user acquisition going around them, and help us areas for improvement related
to be? overcome existing boundaries for products/service use.
product adoption.
DEFINE CONSUMERS’ OBSERVE SOCIAL GESTURES & CONTENT
UNCOVER CONSUMER MOTIVATION
MULTIPLE IDENTITIES INTERACTION
Digital traces reveal important Understanding different Our targets’ gift-giving of content
drivers of human behavior that identities our audience assumes gives us insights into where we
we can use in shaping brand’s lets us help them in their role- can insert brand into their social
online actions. playing and is also useful in exchanges.
targeting.
via @andjelicaaa

9. THERE ARE ONLY 3 RULES OF MEASUREMENT
Your metrics will be as UNIQUE as your business.
If metrics aren’t ACTIONABLE, they aren’t worth using.
DON’T TRY TO MEASURE EVERYTHING.

10. MEASURE WHAT MATTERS... TO WHOM?
To your To your
BUSINESS PROSPECT

11. IT’S EASY TO FOCUS ON VANITY METRICS
“Numbers that make us
look good but don’t really
help make decisions.”
Eric Ries,
The Lean Start-up

12. WHAT’S A VANITY METRIC?
A lot of the time, we
measure for a show
reel, instead of for
business results.
http://bokardo.com/talks/metrics-driven-design-sxsw.pdf

13. WHAT MATTERS IS VALUE CREATION
To your To your
BUSINESS PROSPECT
CONVERSION ENGAGEMENT

14. 2. WHAT YOU SHOULD MEASURE

15. USE ‘PIRATE’ METRICS
Acquisition: users from channels come
to site, landing page, widget, etc
Activation: users enjoy 1st visit, "happy"
user experience
*
Retention: users re-visit multiple times
Referral: users like experience enough *
to refer others via email, links, likes,
blogs, widgets, word-of-mouth, etc
Revenue: users buy, download, sign-up
http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version

16. WHAT ARE SOME ‘VALUE’ METRICS?
To your To your
BUSINESS PROSPECT
CONVERSION ENGAGEMENT

17. Acquisition - drawing people
into the brand experience
To your
BUSINESS Revenue - converting visitors
into customers
Referral - converting
customers into advocates
CONVERSION

18. BEWARE THE CTR...
The starting metric for Acquisition isn’t CTR or “Site
Visits”...
It’s CPA: Cost Per Acquisition
(e.g., how much did you spend to get a new user)
But this by itself is irrelevant
If the CPA is higher than the lifetime value (LTV) of a new
user, you’re in trouble

19. Activation - people enjoyed the
experience...
To your
Retention - enough to come PROSPECT
back more than once...
Referral - and recommended
the experience to others
ENGAGEMENT

20. HOW DO YOU MEASURE RETENTION & REFERRAL?
Returning
Visitors
Registered Users Likes, +1s, RTs
Frequency Mentions
Time on Site Email referrals
Daily Active
Users

21. VALUE CREATION =
CREATING CUSTOMERS
To your To your
BUSINESS PROSPECT
CONVERSION ENGAGEMENT

22. SO WHAT’S MORE IMPORTANT?
Brand growth comes from Acquisition.
Contrary to popular belief, acquisition is actually cheaper,
and more beneficial to the brand, than retention.
Marketing should be geared toward Acquisition.
Product & service design should be geared toward
Activation & Retention.

23. BUT WHAT ABOUT THE OTHER STUFF?
Brand attributes?
Loyalty?
Share of wallet?
Media mix modeling?

25. THERE ARE MARKETING LAWS
Double Jeopardy Law: the lower market share brands in a market
have far fewer buyers in a time period and also lower brand loyalty.
Retention Double Jeopardy: All brands lose some buyers; this loss is
proportionate with market share.
Pareto Law is 60/20 (not 80/20): Slightly more than half of a brand’s
sales come from the top 20% of customers.
Law of Buyer Moderation: Heavy buyers sometimes buy less often,
light buyers sometimes buy more often, non-buyers sometimes
become buyers.
Law of Prototypicality: Image attributes that describe the product
category score higher than less prototypical attributes
Duplication of Purchase Law: A brand’s customer base overlaps with
the customer base of other brands, in line with their market share.

26. AND SOME INCONVENIENT TRUTHS
User bases seldom vary:
Rival brands sell to very similar customer bases.
Attitudes and brand beliefs reflect behavioral loyalty:
Consumers know and say more about brands they use often,
and they think and say little about brands they do not use.
Usage drives attitude:
Consumers like what they buy.

27. 3. TOOLS YOU CAN USE FOR DIGITAL
EXPERIENCE MEASUREMENT

28. APPS & SITES HELP ANSWER QUESTIONS...
I NEVER FELT LIKE I
WAS LEARNING
ANYTHING IMPORTANT

29. WHICH ONE IS MORE SUCCESSFUL?
@37SIGNALS
http://www.slideshare.net/stueccles/lean-startup-metrics

30. WHEN DOES SUCCESS HAPPEN?
EXTENDED SIGN-UP FUNNEL
http://www.slideshare.net/stueccles/lean-startup-metrics

31. WHERE IS SUCCESS COMING FROM?
COHORT METRICS - CREATED
PROJECT
http://www.slideshare.net/stueccles/lean-startup-metrics

32. DON’T HAVE ACCESS TO GOOGLE ANALYTICS?
First -
get access. :)
Second -
try QuantCast:
Gives high-level
insight into traffic
sources, frequency
and volume to site,
and basic
demographic and
socio-economic data
about your visitors.
And it’s free.

33. OR THERE’S CHARTBEAT
For $10/month, try
ChartBeat. It provides
real-time tracking and
engagement metrics like
user paths, time spent
with content, and micro-
actions like clicks,
downloads, and scrolls
that show whether people
are actively engaging with
the site content.

34. HACK THE METRICS
Use unique links - through services like Bit.ly - to
track the behavior that a social mention or display
ad or site/page leads to.
Stop paying for CPMs - pay for conversions (but
define clearly what a ‘conversion’ is, and design ad
units & digital experiences for conversion
moments).

35. REMEMBER RULE 3 -
DON’T MEASURE EVERYTHING -
[WE’RE LOOKING FOR WHAT MAKES MEANINGFUL DIFFERENCES]

36. TITLE
Main Idea
•Bullet
•Bullet

37. TITLE
Main Idea
•Bullet
•Bullet

38. TITLE
Main Idea
•Bullet
•Bullet
GOOGLE BING

39. 4. HOW DO YOU ‘SELL IT’

40. HOW TO TALK TO CLIENTS & COLLEAGUES ABOUT METRICS
Setting Goals Accountability Transparency
Assessing
Optimizing Learning
Effectiveness
Good measurement practices improve:
client-agency relationships
insight
design
results

41. SETTING REASONABLE KPI’S
What are the business objectives?
What are the campaign objectives?
What are the channels/touchpoints?
What is the campaign experience & message?
What do we expect people to do?

42. SUMMING UP
Measure what will teach you the most and help you make the
most important decisions.
Use bespoke, not off-the-shelf, metrics and measurement
tools.
Focus on ‘metrics’ that connect to value creation for the
business and drive brand growth.
Don’t measure everything.

43. THAT WAS FUN.