My presentation for the Measurement & ROI session at the NYC Hyper Island Master Class in JU

1. MEASURE WHAT MATTERS
understanding measurement & roi for digital experiences

2. @FARRAHBOSTIC

3. 4 THINGS:
WHY
WHAT
HOW
(AND HOW TO SELL IT)

4. 1. WHY YOU SHOULD MEASURE

5. BINGO
TIME
KPI SIGN-UPS ON CPM IMPRESSIONS
SITE
CPA CPC REACH &
FREQUENCY LIKES ENGAGEMENT
SENTIMENT PAGE
LTV ANALYSIS ROI FOLLOWS
VIEWS
MENTIONS CTR DOWNLOADS

6. A WORD OF CAUTION
I notice increasing reluctance on
the part of marketing executives
to use judgment; they are coming
to rely too much on research,
and they use it as a drunkard
uses a lamp post: for support,
rather than for illumination.
David Ogilvy

7. WHY DO METRICS MATTER?
1. Metrics reduce arguments based on opinion.
2. Metrics give you answers about what really works.
3. Metrics show you where you’re strong.
4. Metrics allow you to test anything you want.
5. Clients love metrics.
http://bokardo.com/talks/metrics-driven-design-sxsw.pdf

8. WHAT ELSE CAN MEASUREMENT HELP YOU DO?
EXPLORE CONSUMER UNDERSTAND CONTEXT OF
DETECT DEMAND
SENTIMENT BRAND & PRODUCT MENTIONS
Now we can easily assess pre-existing Spontaneous product comments and Digital makes it is easier to detect
demand among our target: how easy is mentions reveal sentiment around them, problems or possible areas for
user acquisition going to be? and help us overcome existing boundaries improvement related products/service
for product adoption. use.
UNCOVER CONSUMER DEFINE CONSUMERS’ OBSERVE SOCIAL GESTURES &
MOTIVATION MULTIPLE IDENTITIES CONTENT INTERACTION
Digital traces reveal important Understanding different identities our
Our targets’ gift-giving of content gives us
drivers of human behavior that we audience assumes lets us help them in
insights into where we can insert brand
can use in shaping brand’s online their role-playing and is also useful in
into their social exchanges.
actions. targeting.
via @andjelicaaa

9. THERE ARE ONLY 3 RULES OF MEASUREMENT
Your metrics will be as UNIQUE as your business.
If metrics aren’t ACTIONABLE, they aren’t worth using.
DON’T TRY TO MEASURE EVERYTHING.

10. MEASURE WHAT MATTERS... TO WHOM?
To your To your
BUSINESS CUSTOMER

11. IT’S EASY TO FOCUS ON VANITY METRICS
“Numbers that make us
look good but don’t really
help make decisions.”
Eric Ries,
The Lean Start-up

12. WHAT’S A VANITY METRIC?
A lot of the time, we
measure for a show
reel, instead of for
business results.
http://bokardo.com/talks/metrics-driven-design-sxsw.pdf

13. WHAT MATTERS IS VALUE CREATION
To your To your
BUSINESS CUSTOMER
CONVERSION ENGAGEMENT

14. 2. WHAT YOU SHOULD MEASURE

15. USE ‘PIRATE’ METRICS
Acquisition: users from channels come
to site, landing page, widget, etc
*
Activation: users enjoy 1st visit, "happy" *
user experience
Retention: users re-visit multiple times
Referral: users like experience enough
to refer others via email, links, likes,
blogs, widgets, word-of-mouth, etc
Revenue: users buy, download, sign-up
http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version

16. WHAT ARE SOME ‘VALUE’ METRICS?
To your To your
BUSINESS CUSTOMER
CONVERSION ENGAGEMENT

17. Acquisition - drawing people into
the brand experience
To your
BUSINESS Revenue - converting visitors into
customers
CONVERSION Referral - converting customers into
advocates

18. BEWARE THE CTR...
The main metric for Acquisition isn’t CTR or “Site Visits”...
It’s CPA: Cost Per Acquisition
(e.g., how much did you spend to get a new user)
But this by itself is irrelevant
If the CPA is higher than the lifetime value (LTV) of a new user,
you’re in trouble

19. Activation - people enjoyed the
experience...
To your
Retention - enough to come back CUSTOMER
more than once...
Referral - and recommended the ENGAGEMENT
experience to others

20. HOW DO YOU MEASURE ACTIVATION?
A net promoter of +50 or better is a great score.
But it’s not good enough on its own - you have to ask people why they gave that score
And then put that information to use

21. HOW DO YOU MEASURE RETENTION & REFERRAL?
Returning Visitors
Registered Users Likes, +1s, RTs
Frequency Mentions
Time on Site Email referrals
Daily Active Users

22. VALUE CREATION =
CUSTOMERS BECOME ADVOCATES
To your To your
BUSINESS CUSTOMER
CONVERSION ENGAGEMENT

23. 3. HOW YOU SHOULD MEASURE

24. APPS & SITES HELP ANSWER QUESTIONS...
I NEVER FELT LIKE I
WAS LEARNING
ANYTHING IMPORTANT

25. WHICH ONE IS MORE SUCCESSFUL?
@37SIGNALS
http://www.slideshare.net/stueccles/lean-startup-metrics

26. WHEN DOES SUCCESS HAPPEN?
EXTENDED SIGN-UP FUNNEL
http://www.slideshare.net/stueccles/lean-startup-metrics

27. WHERE IS SUCCESS COMING FROM?
COHORT METRICS - CREATED
PROJECT
http://www.slideshare.net/stueccles/lean-startup-metrics

28. DON’T HAVE ACCESS TO GOOGLE ANALYTICS?
First -
get access. :)
Second -
try QuantCast:
Gives high-level
insight into traffic
sources, frequency
and volume to site,
and basic
demographic and
socio-economic data
about your visitors.
And it’s free.

29. OR THERE’S CHARTBEAT
For $10/month, try
ChartBeat. It provides
real-time tracking and
engagement metrics like
user paths, time spent
with content, and micro-
actions like clicks,
downloads, and scrolls
that show whether people
are actively engaging with
the site content.

30. FACEBOOK
Dashboard for a brand
page provides
information on traffic,
content, user activity.
And starts to give a
sense of more than just
likes & mentions - the
‘reach’ of the users who
are liking & talking
about you on Facebook.

31. FACEBOOK
The advertising
dashboard has been
updated to focus on
social metrics - like
‘social reach’ (the
absolute number of
people who saw an ad
with social context -
names/pictures of
friends who connected
with the ad’s
destination) and
‘connection’s (social
actions triggered by an
ad, like Likes, app
installs, event RSVPs)
http://www.secondshares.com/2011/05/10/facebook-advertising-embracing-the-second-internet-ethos/

32. TWITTER
Twitter has not yet released its Analytics
tool... So in the meantime - you can use
twittercounter.com and tools like
hootsuite/tweetdeck to track mentions &
conversations

33. REMEMBER RULE 3 -
DON’T MEASURE EVERYTHING -
[WE’RE LOOKING FOR WHAT MAKES MEANINGFUL DIFFERENCES]

34. TITLE
Main Idea
•Bullet
•Bullet

35. TITLE
Main Idea
•Bullet
•Bullet

36. TITLE
Main Idea
•Bullet
•Bullet
GOOGLE BING

37. 4. HOW DO YOU ‘SELL IT’

38. HOW TO TALK TO CLIENTS & COLLEAGUES ABOUT METRICS
Setting Goals Accountability Transparency
Assessing
Optimizing Learning
Effectiveness
Good measurement practices improve:
client-agency relationships
insight
design
results

39. SETTING REASONABLE KPI’S
What are the business objectives?
What are the campaign objectives?
What are the channels/touchpoints?
What is the campaign experience & message?
What do we expect people to do?

40. A CREDIT CARD BRAND
The business objective:
add new cardmembers
The campaign objective:
use social media to drive new
cardmember sign-ups
The touchpoints:
Facebook brand page +
Foursquare integration +
Mobile-optimized Apply site

41. A CREDIT CARD BRAND
What is the most
important KPI/success
metric?
What would you measure/
track to determine if you
are succeeding?
What experiments would
you run to find the best
route to success?

42. A CREDIT CARD BRAND
What matters is acquisition -
new cardmembers.
Acquisition comes from
qualified leads completing
applications - the result of
smaller conversions:
Revenue,
•A larger prospect pool Retention &
•Prospects to qualified Referral matter too:
leads
• Cardmembers to
•Qualified leads to active spenders
applicants
• Cardmembers to
advocates

43. THANK YOU!