Understanding Lean Analytics (and how analytics helps businesses win)

Understanding
Lean Analytics
and how analytics
helps business win

Ben Yoskovitz
@byosko
byosko@gmail.com
#leananalytics

http://leananalyticsbook.com

Monday, March 18, 13

More wins than losses

GoInstant
1st startup Year One Labs
Started blogging
Standout Jobs
Big pivot

1996 1998 2001 2006 2007 2010 2011

The “I got too comfy” years Failed $0


I BELIEVE IN
GUTS + DATA


What we’ll cover today

• Lean Startup
• Key aspects of metrics
• Lean Analytics framework
• The One Metric That Matters
• The Lean Analytics Cycle
• Exploring business models
• Analytics in marketing
• Trends in analytics


LEAN STARTUP


Build Measure Learn

Eric Ries
http://theleanstartup.com


THREE ENGINES OF GROWTH

Sticky

Virality

Paid
http://www.ﬂickr.com/photos/fftang/4941889792/sizes/l/in/photostream/


The Lean Canvas (leancanvas.com)


BASICS OF
ANALYTICS


Analytics is the measurement of
movement towards your
business goals.


Things you can explain to others
about metrics

* Qualitative vs. Quantitative
* Exploratory vs. Reporting
* Vanity vs. Actionable
* Leading vs. Lagging


Qualitative Quantitative
Unstructured, Numbers and stats;
anecdotal, revealing, hard facts but less
hard to aggregate. insight.

Warm and fuzzy. Cold and hard.


DISCOVER QUALITATIVELY
AND

PROVE QUANTITATIVELY


Exploratory Reporting
Speculative, trying to Predictable, keeping
find unexpected or you abreast of
interesting insights. normal, managerial
operations.

http://www.flickr.com/photos/50755773@N06/5415295449/ http://www.flickr.com/photos/elwillo/4737933662/

Case study: From friends to moms

• Started as Circle of Friends
• Grew to 10M users

BUT ENGAGEMENT SUCKED


Case study: Moms are crazy (in a good way!)

• Messages to one another were on average 50% longer.
• 115% more likely to attach a picture to a post they wrote.
• 110% more likely to engage in a threaded (i.e. deep) conversation.
• Friends, once invited, were 50% more likely to become engaged users.
• 180% more likely to click on Facebook news feed items.
• 60% more likely to accept invitations to the app.

ENGAGEMENT WAS GREAT


Vanity Actionable
Picks a
direction.

Makes you feel
good, but doesn’t
change how you’ll
act.

VANITY METRICS ARE BAD

A metric from the early, foolish days of the Web.
Hits
Count people instead.
Marginally better than hits. Unless you’re displaying
Page views
ad inventory, count people.
Is this one person visiting a hundred times, or are a
Visits
hundred people visiting once? Fail.
This tells you nothing about what they did, why they
Unique visitors
stuck around, or if they left.

Followers/ Count actions instead. Find out how many followers
friends/likes will do your bidding.


http://www.ﬂickr.com/photos/circasassy/7858155676/

If it won’t change
how you behave,
it’s a

bad
metric.


LEADING LAGGING
Number today Historical metric
that shows metric that shows how
tomorrow-makes you’re doing-
the news reports the news

http://www.ﬂickr.com/photos/vegaseddie/3310041214/sizes/l/in/photostream/


What mode of e-commerce are you?
How many of
your customers Then you are in this Your customers will You are just
Focus on
buy a second time mode buy from you like
in 90 days?

Low CAC,
1-15% Acquisition Once 70% high
of retailers checkout

15-30% Hybrid 2-2.5 20% Increasing
per year of retailers returns

Loyalty,
>30% Loyalty >2.5 10% inventory
per year of retailers expansion

(Thanks to Kevin Hillstrom for this.)

ANALYTICS SUPERPOWERS
(or what the heck is growth hacking?)

http://www.ﬂickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/


10000

1000

100

10

1
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec

Ice cream consumption Drownings

Correlated Causal
Two variables that An independent
change in similar ways, factor that directly
perhaps because they impacts a dependent
are linked to something one.
else.
Summer
al

Ca
us

us
Ca

al
Correlated
Ice cream Drowning
consumption

Causality is a superpower, because it lets you change
the future.

Correlation lets you Causality lets you
predict the future change the future

“I will have 420 engaged “If I can make more ﬁrst-
users and 75 paying time visitors stay on for
customers next month.” 17 minutes I will increase
sales in 90 days.”

Optimize the
Find correlation Test causality
causal factor

LEAN ANALYTICS
FRAMEWORK


simplify.

http://www.ﬂickr.com/photos/josefeliciano/3849557951/sizes/l/in/photostream/


YOUR BASIC
THE STAGE OF
BUSINESS
YOUR STARTUP
MODEL
How you make $$ Lifecycle

• E-commerce • Empathy
• SaaS • Stickiness
• Free mobile app • Virality
• Media site • Revenue
• Collaborative content site • Scale
• Two-sided marketplace


LEAN ANALYTICS “GATE” NEEDED TO
STAGES MOVE FORWARD
I’ve found a real, poorly-met need
EMPATHY that a reachable market faces.

I’ve ﬁgured out how to solve the
problem in a way they will adopt
STICKINESS and pay for.

I’ve built the right product/features/
GROWTH RATE

functionality that keeps users
VIRALITY
around.

The users and features fuel growth
REVENUE organically and artiﬁcially.

I’ve found a sustainable, scalable
business with the right margins in a
SCALE
healthy ecosystem.


Case study: Buffer goes from
Stickiness to Scale (through Revenue)

• Stage: Scale
• Model: SaaS (consumer)
• Popular social sharing application.
• Focused primarily on customer
acquisition
• Charged from day one


Buffer charges early to prove people
want the problem solved

20% of visitors create an account
(acquisition / Empathy)
of sign-ups become active
64% (start of Stickiness)
of sign-ups return in the 1st month
60% (engagement / Stickiness)
of sign-ups are active after 6 months
20% (engagement / Stickiness)
convert from free to paid
2% (Revenue)


How it all comes together

The business you’re in

E- 2-sided Mobile User-gen
SaaS Media
commerce market app content
Empathy
The stage you’re at

Stickiness

Virality

Revenue

Scale


How it all comes together

The business you’re in

SaaS Media
Empathy
The stage you’re at

One Metric
Stickiness

Virality

Revenue That Matters.
Scale


Case study: SEOmoz reduces the
KPIs it tracks

• Stage: Scale
• Model: SaaS
• SEO toolkit (product suite)
• Reduced KPIs to focus on Net Adds


Net Adds = “health of the business”
indicator

If Net Adds: Why & Next Steps:


indicator

• Was a marketing campaign successful?
• Was churn lowered?
• Were customer complaints lowered?
• Was a product upgrade valuable?


indicator


• How can we acquire more valuable customers?
• Can we increase site conversion?
• How can we lower churn?
• What product features can increase engagement?


indicator


• How can we acquire more valuable customers?
• Can we increase site conversion?
• How can we lower churn?
• What product features can increase engagement?

• Are the new customers not the right segment?
• Did a marketing campaign fail?
• Are too many customers churning?
• Did a product upgrade fail to impress or cause issues?


DRAW A LINE IN THE SAND


HighScore House deﬁnes an “active user”


What’s your OMTM?

SaaS Media

Empathy Interviews; qualitative results; quantitative scoring; surveys

Loyalty, Inventory, Engagement, Downloads, Content, Traffic, visits,
Stickiness conversion listings churn churn, virality spam returns

CAC, shares, Inherent WoM, app Invites, Content
Virality reactivation
SEM, sharing
virality, CAC ratings, CAC sharing virality, SEM

(Money from transactions) (Money from active users) (Money from ad clicks)

Transaction, Transactions, Upselling, CLV, Ads, CPE, affiliate
Revenue CLV commission CAC, CLV ARPDAU donations %, eyeballs

Affiliates, Other API, magic Spinoffs, Analytics, Syndication,
Scale white-label verticals #, mktplace publishers user data licenses


METRICS
ARE LIKE
SQUEEZE
TOYS


YOUR GOAL IS TO MAKE
FASTER,
MORE INTELLECTUALLY
HONEST DECISIONS
AND

EMPOWER YOUR
ORGANIZATION
TO DO THE SAME


What these have in common:
The Lean Analytics Cycle


Pick a KPI


Pick a KPI Draw a line
in the sand


in the sand

Find a
potential
improvement


in the sand

Find a
potential
improvement

Without
data: make a
good guess


in the sand

Find a
potential
improvement

Without With data:
data: make a ﬁnd a
good guess commonality


in the sand

Find a
potential
improvement

Without With data:

Hypothesis


in the sand

Find a
potential
improvement

Without With data:

Hypothesis
Make changes
in production

in the sand

Find a
potential
improvement

Without With data:
Design a test

Hypothesis
Make changes
in production

in the sand

Find a
potential
improvement

Without With data:
Design a test
Measure
the results Hypothesis
Make changes
in production

in the sand

Find a
potential
improvement

Did we move
the needle? Without With data:
Design a test
Measure
Make changes
in production

Success! Pick a KPI Draw a line
in the sand

Find a
potential
improvement

Did we move
Design a test
Measure
Make changes
in production

in the sand
Pivot or
give up Find a
potential
improvement

Did we move
Design a test
Measure
Make changes
in production

in the sand
Pivot or
give up Draw a new line Find a
potential
improvement

Did we move
Design a test
Measure
Make changes
in production

in the sand
Pivot or
give up Draw a new line Find a
potential
Try again improvement

Did we move
Design a test
Measure
Make changes
in production

EXPLORING
BUSINESS MODELS


MEDIA COMPANIES

• Audience & churn (how many people visit the site, and how
loyal they are)
• Ad inventory (the number of impressions that can be monetized)
• Ad rates (sometimes measured in cost-per-engagement--
essentially how much a site can make from those impressions based
on the content it covers and the people who visit)
• Click-through rates (how many of the impressions actually turn
into money)
• The content/advertising balance (the balance of ad
inventory ad rates and content that maximizes overall performance


E-COMMERCE COMPANIES

• Conversion rate (the # of visitors who buy something)
• Purchases / year (the # of purchases made by each customer per year)
• Average shopping cart size (the amount of money spent / purchase)
• Cost of customer acquisition (the money spent to get someone to buy something)
• Revenue / customer (the lifetime value of each customer)
• Top keywords driving trafﬁc to the site (those terms that people are looking for,
and associate with you--a clue to adjacent products or markets)

• Top search terms (both those that lead to revenue, and those that don’t have any
results)

• Effectiveness of recommendation engines (how likely a visitor is to add a
recommended product to their cart)

• Virality (word of mouth, and sharing per visitor)
• Mailing list effectiveness (click-through rates and ability to make buyers return and
buy)


Case study: WineExpress increases
revenues

• Stage: Revenue
• Model: E-commerce
• Exclusive wine shop partner of the
Wine Enthusiast catalog and website
• “Wine of the day” page is highly
trafﬁcked, needed optimization


Case study: Before and after

41%
increase in
revenue per
visitor


Don’t forget the real world

Shipping time, stock availability,
logistics, ratings, and other factors
have a real impact on most e-
commerce companies.

Shipping time, stock availability,
DON’T logistics, ratings, and other factors
FORGET THE have a real impact on most e-
REAL WORLD commerce companies.


Outside In: The Power
of Putting Customers
at the Center of Your
Business

http://www.amazon.com/Outside-Putting-Customers-Center-Business/dp/0547913982


ANALYTICS IN
MARKETING


HOW IT USUALLY HAPPENS

• Some CxO catches on to the latest thing from talking to someone in
social media, or tries out the latest cool app (Vine, Pinterest)
• That person rushes into the ofﬁce, exclaiming, “We need to do a
Pinterest campaign now!”
• The marketing team scrambles to come up with a campaign
• They use that campaign to back into a metric
• They run the campaign and report the results (maybe!)

...which of course means no line in the sand, no
discipline, and attempts to move metrics that may not
be core to the business in the ﬁrst place.


HOW IT SHOULD BE DONE

• There’s a known business objective / problem to solve
• There’s a key metric for it (OMTM)
• There’s a goal (line in the sand)
• We come up with a campaign or effort to move the needle
(hypothesis)
• We decide the medium that we want to try that on
(campaign details)
• We execute
• We measure and adjust (learn & iterate)


TRENDS IN
ANALYTICS


From: To:
Aggregate Individual
Segment Cohort
Generic Vertical
Silos Aggregation
Daily Realtime
Reports Exceptions
Pages Events
Funnels Inﬂuences
Desktop Mobile
Accounting Predictive


Aggregate to individual
Aggregate to individual

e.g. KISSmetrics

Segment to cohort
Segment to cohort

e.g. Mixpanel

Generic to vertical
Generic to vertical

SaaS' Mobile' Publishing'

e.g. Totango, Flurry, Parse.ly

Silos to aggregation


Daily to realtime
Daily to realtime

Remember: right time is better than real time

Reports to exceptions
Reports to exceptions


Pages to events
Pages to events

e.g. Chartbeat

Funnels to inﬂuences
Funnels to influences


Desktop to mobile

e.g. Keen.io

Reporting to predictive

e.g. Synapsify

Once, a leader convinced others in
the absence of data.


Now, a leader knows what
questions to ask.


Thank you.
follow me. email me.
@byosko

byosko@gmail.com
instigatorblog.com

subscribe.
ORDER!

leananalyticsbook.com


Understanding Lean Analytics (and how analytics helps businesses win)

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (12)

Semelhante a Understanding Lean Analytics (and how analytics helps businesses win)

Semelhante a Understanding Lean Analytics (and how analytics helps businesses win) (20)

Último

Último (20)

Understanding Lean Analytics (and how analytics helps businesses win)