SlideShare uma empresa Scribd logo
1 de 16
Baixar para ler offline
7 Practical Tips
on Lean Analytics
Janne Aukia — 17 Feb 2015
1. Lean Analytics
requires a Lean project
Illustration from commons.wikimedia.org/wiki/File:Kepler-solar-system-1.png
Challenging for a consultancy?
Lean Analytics works well with
Lean Startup, Lean UX and

Agile projects
Question designs, question
authorities, ask difficult questions,
be prepared to kill features.
Requires freedom to experiment
2. Write down a success
metric for every feature
Was this feature used more than
Google+ login?
If feature isn’t successful, 

just kill it.
Measure only actionable
metrics.
Is the feature for acquisition,
retention or revenue?
3. You need lots of
users for an A/B test
Typically at least 10k events,
such as visits or people using a
feature.
Doesn’t work for small
startups.
4. Make big changes
Illustration from http://quod.lib.umich.edu/m/moa/agb8710.0001.001/26
If you cannot see the changes
from 2 meters away, most likely
your AB test won’t either.
Google A/B tested 41 shades of
blue. Don’t do that — leads to
local optimum.
Illustration from http://www.doxsey.net/blog/too-practically-minded-to-be-of-any-theological-good/
5. Measure your tests
correctly
Never terminate early

a running test
Use a split test calculator, 

for example:
http://getdatadriven.com/ab-significance-test
6. Push through radical
changes with A/B
7. Use the right tools
Google Analytics can be used for
tracking events but it is
painful.
Mixpanel more suited for the
job. Consider Keen.io, Heap as
well.
1.Lean Analytics
requires a Lean
project
2.Specify success
metric for every
feature
3.You need lots of
users for an A/B
test
4.Make big changes
5.Measure your tests
correctly
6.Push through radical
changes with A/B
7.Use the right tools
Summary

Mais conteúdo relacionado

Mais procurados

Mais procurados (11)

Why should I care about the Minimum Viable Product (MVP)
Why should  I care about the Minimum Viable Product  (MVP)Why should  I care about the Minimum Viable Product  (MVP)
Why should I care about the Minimum Viable Product (MVP)
 
Lean Startup Hardware Examples
Lean Startup Hardware ExamplesLean Startup Hardware Examples
Lean Startup Hardware Examples
 
Agile Problem, Commercial Solution
Agile Problem, Commercial SolutionAgile Problem, Commercial Solution
Agile Problem, Commercial Solution
 
Product Management Ethics in A.I. by Yammer's former Dir. of Product
Product Management Ethics in A.I. by Yammer's former Dir. of ProductProduct Management Ethics in A.I. by Yammer's former Dir. of Product
Product Management Ethics in A.I. by Yammer's former Dir. of Product
 
Annotating Images for Machine Learning Models: 5 Common Misconceptions
Annotating Images for Machine Learning Models: 5 Common MisconceptionsAnnotating Images for Machine Learning Models: 5 Common Misconceptions
Annotating Images for Machine Learning Models: 5 Common Misconceptions
 
Value-Focused Prioritization & Decision-Making
Value-Focused Prioritization & Decision-MakingValue-Focused Prioritization & Decision-Making
Value-Focused Prioritization & Decision-Making
 
Different faces of MVP
Different faces of MVPDifferent faces of MVP
Different faces of MVP
 
Metrics for design decisions_Diana Prokusheva
Metrics for design decisions_Diana ProkushevaMetrics for design decisions_Diana Prokusheva
Metrics for design decisions_Diana Prokusheva
 
CampusSDN2017 - Jawdat: Product Management and Agile Development
CampusSDN2017 - Jawdat: Product Management and Agile DevelopmentCampusSDN2017 - Jawdat: Product Management and Agile Development
CampusSDN2017 - Jawdat: Product Management and Agile Development
 
May 2021 Embedded Vision Summit Opening Remarks (May 26)
May 2021 Embedded Vision Summit Opening Remarks (May 26)May 2021 Embedded Vision Summit Opening Remarks (May 26)
May 2021 Embedded Vision Summit Opening Remarks (May 26)
 
Impact Mapping with Innovation Games (R)
Impact Mapping with Innovation Games (R)Impact Mapping with Innovation Games (R)
Impact Mapping with Innovation Games (R)
 

Semelhante a 7 Practical Tips on Lean Analytics

1. no2.The Waterfall model has some disadvantages.Agile software.pdf
1. no2.The Waterfall model has some disadvantages.Agile software.pdf1. no2.The Waterfall model has some disadvantages.Agile software.pdf
1. no2.The Waterfall model has some disadvantages.Agile software.pdf
anjaliselectionahd
 
Agile Gurugram 2023 I Elevating Agile Teams with Generative AI- Paramjit Jolly
Agile Gurugram 2023 I Elevating Agile Teams with Generative AI- Paramjit JollyAgile Gurugram 2023 I Elevating Agile Teams with Generative AI- Paramjit Jolly
Agile Gurugram 2023 I Elevating Agile Teams with Generative AI- Paramjit Jolly
AgileNetwork
 

Semelhante a 7 Practical Tips on Lean Analytics (20)

Stop throwing money out the window
Stop throwing money out the windowStop throwing money out the window
Stop throwing money out the window
 
Agile and data driven product development oleh Dhiku VP Product KMK Online
Agile and data driven product development oleh Dhiku VP Product KMK OnlineAgile and data driven product development oleh Dhiku VP Product KMK Online
Agile and data driven product development oleh Dhiku VP Product KMK Online
 
Maximising likelihood of success: Applying Product Management to AI/ML/DS pr...
Maximising likelihood of success:  Applying Product Management to AI/ML/DS pr...Maximising likelihood of success:  Applying Product Management to AI/ML/DS pr...
Maximising likelihood of success: Applying Product Management to AI/ML/DS pr...
 
Feature Prioritization Techniques for an Agile PMs by Microsoft PM
Feature Prioritization Techniques for an Agile PMs by Microsoft PMFeature Prioritization Techniques for an Agile PMs by Microsoft PM
Feature Prioritization Techniques for an Agile PMs by Microsoft PM
 
Experimentation at Blue Apron (webinar)
Experimentation at Blue Apron (webinar)Experimentation at Blue Apron (webinar)
Experimentation at Blue Apron (webinar)
 
1. no2.The Waterfall model has some disadvantages.Agile software.pdf
1. no2.The Waterfall model has some disadvantages.Agile software.pdf1. no2.The Waterfall model has some disadvantages.Agile software.pdf
1. no2.The Waterfall model has some disadvantages.Agile software.pdf
 
Remote, unmoderated usability and user testing.
Remote, unmoderated usability and user testing.Remote, unmoderated usability and user testing.
Remote, unmoderated usability and user testing.
 
Why Adopt Analytic Driven Testing?
Why Adopt Analytic Driven Testing?Why Adopt Analytic Driven Testing?
Why Adopt Analytic Driven Testing?
 
Experiment Management for the Enterprise
Experiment Management for the EnterpriseExperiment Management for the Enterprise
Experiment Management for the Enterprise
 
Jan Moons talks at World Usability Day 2015 in Antwerp
Jan Moons talks at World Usability Day 2015 in AntwerpJan Moons talks at World Usability Day 2015 in Antwerp
Jan Moons talks at World Usability Day 2015 in Antwerp
 
How AI Can Be Leveraged In All Aspects Of Testing
How AI Can Be Leveraged In All Aspects Of TestingHow AI Can Be Leveraged In All Aspects Of Testing
How AI Can Be Leveraged In All Aspects Of Testing
 
prod-dev-management.pptx
prod-dev-management.pptxprod-dev-management.pptx
prod-dev-management.pptx
 
Munkhzorig - Digital Transformation
Munkhzorig - Digital TransformationMunkhzorig - Digital Transformation
Munkhzorig - Digital Transformation
 
Agile Analytics
Agile AnalyticsAgile Analytics
Agile Analytics
 
PMI CH AMM2023 - Bye Bye Project Manager - SwissQ.pptx
PMI CH AMM2023 - Bye Bye Project Manager - SwissQ.pptxPMI CH AMM2023 - Bye Bye Project Manager - SwissQ.pptx
PMI CH AMM2023 - Bye Bye Project Manager - SwissQ.pptx
 
Agile Methods: Fact or Fiction
Agile Methods: Fact or FictionAgile Methods: Fact or Fiction
Agile Methods: Fact or Fiction
 
PCC2 - How do I incorporate Apple-like design into my products?
PCC2 - How do I incorporate Apple-like design into my products?PCC2 - How do I incorporate Apple-like design into my products?
PCC2 - How do I incorporate Apple-like design into my products?
 
Agile Gurugram 2023 I Elevating Agile Teams with Generative AI- Paramjit Jolly
Agile Gurugram 2023 I Elevating Agile Teams with Generative AI- Paramjit JollyAgile Gurugram 2023 I Elevating Agile Teams with Generative AI- Paramjit Jolly
Agile Gurugram 2023 I Elevating Agile Teams with Generative AI- Paramjit Jolly
 
The app trail how ideas move out of the drawing board onto the app store
The app trail how ideas move out of the drawing board onto the app storeThe app trail how ideas move out of the drawing board onto the app store
The app trail how ideas move out of the drawing board onto the app store
 
3 Optimisation Decks : WAW Copenhagen - 27 Feb 2013
3 Optimisation Decks : WAW Copenhagen - 27 Feb 20133 Optimisation Decks : WAW Copenhagen - 27 Feb 2013
3 Optimisation Decks : WAW Copenhagen - 27 Feb 2013
 

Último

%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
masabamasaba
 
%+27788225528 love spells in Knoxville Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Knoxville Psychic Readings, Attraction spells,Br...%+27788225528 love spells in Knoxville Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Knoxville Psychic Readings, Attraction spells,Br...
masabamasaba
 
Abortion Pills In Pretoria ](+27832195400*)[ 🏥 Women's Abortion Clinic In Pre...
Abortion Pills In Pretoria ](+27832195400*)[ 🏥 Women's Abortion Clinic In Pre...Abortion Pills In Pretoria ](+27832195400*)[ 🏥 Women's Abortion Clinic In Pre...
Abortion Pills In Pretoria ](+27832195400*)[ 🏥 Women's Abortion Clinic In Pre...
Medical / Health Care (+971588192166) Mifepristone and Misoprostol tablets 200mg
 
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
masabamasaba
 
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
chiefasafspells
 
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
VictoriaMetrics
 
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Medical / Health Care (+971588192166) Mifepristone and Misoprostol tablets 200mg
 
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
masabamasaba
 

Último (20)

%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
 
%in Benoni+277-882-255-28 abortion pills for sale in Benoni
%in Benoni+277-882-255-28 abortion pills for sale in Benoni%in Benoni+277-882-255-28 abortion pills for sale in Benoni
%in Benoni+277-882-255-28 abortion pills for sale in Benoni
 
%+27788225528 love spells in Knoxville Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Knoxville Psychic Readings, Attraction spells,Br...%+27788225528 love spells in Knoxville Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Knoxville Psychic Readings, Attraction spells,Br...
 
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park %in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
 
Abortion Pills In Pretoria ](+27832195400*)[ 🏥 Women's Abortion Clinic In Pre...
Abortion Pills In Pretoria ](+27832195400*)[ 🏥 Women's Abortion Clinic In Pre...Abortion Pills In Pretoria ](+27832195400*)[ 🏥 Women's Abortion Clinic In Pre...
Abortion Pills In Pretoria ](+27832195400*)[ 🏥 Women's Abortion Clinic In Pre...
 
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
 
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
 
Artyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptxArtyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptx
 
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
WSO2Con2024 - From Code To Cloud: Fast Track Your Cloud Native Journey with C...
 
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
 
What Goes Wrong with Language Definitions and How to Improve the Situation
What Goes Wrong with Language Definitions and How to Improve the SituationWhat Goes Wrong with Language Definitions and How to Improve the Situation
What Goes Wrong with Language Definitions and How to Improve the Situation
 
Architecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the pastArchitecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the past
 
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
 
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
 
%in Midrand+277-882-255-28 abortion pills for sale in midrand
%in Midrand+277-882-255-28 abortion pills for sale in midrand%in Midrand+277-882-255-28 abortion pills for sale in midrand
%in Midrand+277-882-255-28 abortion pills for sale in midrand
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
 
WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...
WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...
WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...
 
WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?
 
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
 
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park %in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
 

7 Practical Tips on Lean Analytics

  • 1. 7 Practical Tips on Lean Analytics Janne Aukia — 17 Feb 2015
  • 2. 1. Lean Analytics requires a Lean project Illustration from commons.wikimedia.org/wiki/File:Kepler-solar-system-1.png
  • 3. Challenging for a consultancy? Lean Analytics works well with Lean Startup, Lean UX and
 Agile projects Question designs, question authorities, ask difficult questions, be prepared to kill features. Requires freedom to experiment
  • 4. 2. Write down a success metric for every feature
  • 5. Was this feature used more than Google+ login? If feature isn’t successful, 
 just kill it. Measure only actionable metrics. Is the feature for acquisition, retention or revenue?
  • 6. 3. You need lots of users for an A/B test
  • 7. Typically at least 10k events, such as visits or people using a feature. Doesn’t work for small startups.
  • 8. 4. Make big changes Illustration from http://quod.lib.umich.edu/m/moa/agb8710.0001.001/26
  • 9. If you cannot see the changes from 2 meters away, most likely your AB test won’t either. Google A/B tested 41 shades of blue. Don’t do that — leads to local optimum. Illustration from http://www.doxsey.net/blog/too-practically-minded-to-be-of-any-theological-good/
  • 10. 5. Measure your tests correctly
  • 11. Never terminate early
 a running test Use a split test calculator, 
 for example: http://getdatadriven.com/ab-significance-test
  • 12. 6. Push through radical changes with A/B
  • 13.
  • 14. 7. Use the right tools
  • 15. Google Analytics can be used for tracking events but it is painful. Mixpanel more suited for the job. Consider Keen.io, Heap as well.
  • 16. 1.Lean Analytics requires a Lean project 2.Specify success metric for every feature 3.You need lots of users for an A/B test 4.Make big changes 5.Measure your tests correctly 6.Push through radical changes with A/B 7.Use the right tools Summary