I have given this talk at LeanCamp Barcelona and roughly the same at Hackernews London. It's an extended and updated version of the previous LeanCamp London talk. Feedback highly welcome.

1. METRICS
LESSONS
LEARNED
@andreasklinger
#LEANCAMP | @andreasklinger

2. Andreas Klinger
CWTFO of LOOKK
@andreasklinger
Slides: http://www.slideshare.net/andreasklinger

3. STARTuP
LOOKK | CONDÉ NAST | 10-14-2011 | 3

4. STARTuP
BEFORE PRODuCT
MARKET FIT
LOOKK | CONDÉ NAST | 10-14-2011 | 4

5. STARTuP
BEFORE PRODuCT
MARKET FIT
“BLuE OCEAN”
LOOKK | CONDÉ NAST | 10-14-2011 | 5

6. STARTuP
BEFORE PRODuCT
MARKET FIT
“BLuE OCEAN” ESS
M ADN
LOOKK | CONDÉ NAST | 10-14-2011 | 6

7. www.LOOKK.com

8. 4 Biggest (development) Problems
we had/have:
+1
?
OvER NO BLOCKED uNCLEAR
ENgINEERED DATA PROCESSES uSER vALuE
AND OvER DRIvEN PROP
CONCEPTED STARTuP SOLuTION? (PRE-PMF)
FEATuRES CuLTuRE KANBAN
BOARD SOLuTION?
SOLuTION? SOLuTION? CuSTDEv
FEATuRE gOINg MENTAL
ROLLOuTS IN ON METRICS.
A/B
LOOKK | CONDÉ NAST | 10-14-2011 | 8

9. 4 Biggest (development) Problems
we had/have:
+1
?
OvER NO BLOCKED uNCLEAR
ENgINEERED DATA PROCESSES uSER vALuE
AND OvER DRIvEN PROP
CONCEPTED STARTuP SOLuTION? (PRE-PMF)
FEATuRES CuLTuRE KANBAN
BOARD SOLuTION?
SOLuTION? SOLuTION? CuSTDEv
FEATuRE gOINg MENTAL +
ROLLOuTS IN ON METRICS. uSERgROuP
A/B IN FB
LOOKK | CONDÉ NAST | 10-14-2011 | 9

10. BuZZ IS OK YOu?
NuMBERS
ARE OK
FEEDBACK
IS OK
MEDIA
IS OK
LOOKK | CONDÉ NAST | 10-14-2011 | 10

11. BuZZ IS OK YOu?
SOMETHINg
IS WRONg
NuMBERS
ARE OK TRAFFIC
SPIKES
FEEDBACK
IS OK NO STICKYNESS
MEDIA PuSHES
IS OK DON’T LIFT
TRAFFIC
LOOKK | CONDÉ NAST | 10-14-2011 | 11

12. BuZZ IS OK YOu?
SOMETHINg
IS WRONg
NuMBERS
ARE OK TRAFFIC
SPIKES
FEEDBACK
IS OK NO STICKYNESS
MEDIA PuSHES
IS OK DON’T LIFT
TRAFFIC
vANITY WHAT
ACTIONS
WORK?
LOOKK | CONDÉ NAST | 10-14-2011 | 12

13. vANITY METRICS
Always goes up. ESPECIALLY:
- Registered users
- Facebook Fans

14. vANITY METRICS
Always goes up. ESPECIALLY:
- Registered users
- Facebook Fans
Not actionable…

15. vANITY METRICS v2
OF THIS MONTH
- visitors / Month Tell you nothing
- users / Month
Usually more related to external
factors (PR, Search, etc)

16. vANITY METRICS v2
OF THIS MONTH
- visitors / Month Tell you nothing
- users / Month
Usually more related to external
factors (PR, Search, etc)
* Tell you nothing about the health of your business or
product
It’s a like a shop saying “we do good… there are now more people on the street”
… and not actionable

17. BEFORE
PRODuCT MARKET FIT
FOCuS ON
RETENTION
#LEANCAMP | @andreasklinger

18. BEFORE PMF
FOCuS ON RETENTION
http://www.ashmaurya.com/2010/07/3-rules-to-
actionable-metrics/
Summary:
In Discovery/validation
Focus on Retention & Activation
In validation/growth
Focus on Revenue and Referral

19. THE REASON:
RETENTION
=
f(uSERHAPPINESS)
#LEANCAMP | @andreasklinger

20. RETENTION =
f(uSERHAPPINESS)
Actually you don’t want to measure retention.
You want to measure user Happiness.
Especially before PMF.
Rentention is the best signal for
userhappiness.
But if you can - narrow down your KPIs
to show user happiness and
the health of your product.
If you are multi sided (eg. buyers/sellers)
segment each group seperately
IF YOu WILL DO ONE THINg ONLY:
FOCuS ON uSER HAPPINESS

21. KPI’S
NEED TO BE
YOuR HEALTH
MONITOR
#LEANCAMP | @andreasklinger

22. KPI’S
NEED TO BE
YOuR HEALTH
MONITOR
What numbers do actually show you the (real)
health of your product?
What numbers could show user happiness?
E.g. Crashpadder.com (privat room rentals)
calculates an activity/happiness score for each Host.
And then creates groups (cohorts) for each city and
compares over time.

23. IN PRODuCT DISCOvERY
MARKETINg
CAN HuRT
YOuR
NuMBERS
#HNLONDON | @andreasklinger

24. (WRONg) MARKETINg
STuNTS (CAN) HuRT YOuR
(ACTIONABLE) NuMBERS
E.g. PR BuRSTS OR COMPETITIONS
- Created visits in google Analytics
- Created userentries in Database
- Disappeared.
- Lower your retention and activation (%) rates
- Defocus you from understanding our product.
Example: LOOKK
We asked users to vote designers in a competition.
ass about LOOKK.
They just created “Dataschmutz”.
useless data that obfuscates your real user behaviour

25. DRILL DOWN
TO REMOvE
DATASCHMuTZ
#HNLONDON | @andreasklinger

26. DRILL DOWN TO REMOvE
DATASCHMuTZ*
Q1: What numbers really show the health of your
Q2: How can you make them more “stable” against
Example:
Base your activation and retention KPIs on visits of
If you are focusing on your onboarding experience (eg.
services not communities) focus base them on the user
registration date.
* Schmutz (noun, german/jiddish) means “dirt”

27. THE METRICS FAIRY SAYS…
METRICS
NEED
TO
HuRT
#LEANCAMP | @andreasklinger

28. METRICS
NEED TO HuRT
If the metrics you focus on do not hurt you
everytime you look at them and you are too
embarassed to show them around.
They are either
a) not narrowed down enough
or
b) nothing you should focus on.

29. START WITH
AARRR
EXAMPLE - LOOKK Nice to look at. useless to work with.
ACQuISITION - user registered Eg.
ACTIvATION - user voted (eg 90%) We changed activiation from
“Registered users that voted”
RETENTION - user visited Again
to
REFERRAL - … “Registered users that voted
REvENuE - … for more than two designers”
Everybody else is just voting for a friend.
Now:
AARRRR = THE PAIN
THE PAIN = ACTIONABLE

30. “AARRR” IS THE
SOuND OF PAIN
ACQuISITION
You need a way to reach the user.
(Dave is wrong. A visit is just fog.s)
ACTIvATION
The user has seen the real core of your
product and used it.
(segment in user groups if multifaced)
RETENTION
The user did an action that leads to value. (if
possible close to money or coreaction)
REFERRAL
Not only shared but invited users came.
REvENuE
Money.

31. TOO MANY
METRICS
ARE
BAD FOR
YOuR EYES
FOCuS
#LEANCAMP | @andreasklinger

32. TOO MANY
METRICS
ARE
BAD FOR
YOuR EYES
On your dashboards only watch those
metrics you are currently iterating with
your product on.
There are always numbers going up and
Don’t go crazy because of that.
Focus your product discovery around
assumptions. Align your metrics to that.
Example: At LOOKK we removed revenue
from our dashboards while focusing on
newsfeed and community features.

33. LESSONS
LEARNED
TOOLS
#LEANCAMP | @andreasklinger

34. TOOLS
TL;DR: All Suck.

35. TOOLS
TL;DR: All Suck.
…but are useful for certain parts:

36. TOOLS
TL;DR: All Suck.
…but are useful for certain parts:
Kissmetrics, Mixpanel = People Analysis
Your own Database = drilldown
Be consistent throughout all tools:
- Naming of goals
- Logging and Naming of user types
(eg. user, Designer, Admin, visitor)

37. TOOLS
TL;DR: All Suck.
…but are useful for certain parts:
Kissmetrics, Mixpanel = People Analysis
Your own Database = drilldown
There are three things you want to do:
* Insights (how many…)
* Accounting (compare birdeye weeks/months)
* Comparing (eg. A/B or focused cohorts)

38. TOOLS
google
Analytics
- Referrals
- Optimize Online Marketing,
- Bird eye view Website
- Suck for funnel analysis (no ad-hoc eg)
- Suck for consistancy (goals are easy to break)
Don’t do funnel testing in gA
- only track the last step
Dashboards rock.
Don’t go too crazy with customization
Unfortunately single
- it will break.
player only (yet)
READ: Cohorts in google Analytics
by http://danhilltech.tumblr.com/

39. TOOLS
KissMetrics
- Insights on Conversions
- Quick reports on features/pages
- Suck (yet) at dashboards
use it for funnel optimization.
“View this people” feature is genius If you are a SaaS use it for everything ;)
Don’t expect too much
and also test mixpanel.

40. TOOLS
LOOKK
Butler
- SQL = quick, Tables = cheap
- You can drilldown to your
actionable KPIs
- Dashboards for team
members
- Suck at bird-eye view, not ad-
Questions:
Who are the most popular
designers if you only count
people who vote for more than
one designer?
users?

41. TOOLS
LOOKK
Butler
Team dashboards are easy to do

42. BuT…
ALWAYS REMEMBER
IN PRE-PMF
CuSTDEv > METRICS
#LEANCAMP | @andreasklinger

43. THANKS.
PLEASE
SHARE
YOuR
LESSONS
@ANDREASKLINgER
Presentation slides: http://slideshare.net/andreasklinger