O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Validation and hypothesis based product management by Abdallah Al-Khalidi

522 visualizações

Publicada em

Prioritization and validation are key activities for successful product management. As a product manager, how will you succeed if you are not able to prioritize correctly? How do you determine what feature to release next and if it is the correct feature to build in the first place?

This talk will cover principle methods and frameworks for feature validation and prioritization and is recommended for product managers and people working in product.

Publicada em: Tecnologia
  • Entre para ver os comentários

Validation and hypothesis based product management by Abdallah Al-Khalidi

  1. 1. S§ Validation and Hypothesis based Product Management
  2. 2. Focus • Root cause for problems. • Ditches ideas that irrelevant. Why should you care? Feasible • Solves the real problem. • No wasted efforts. Objectivity • Eliminate personal bias for what causes what. Replicable • Results can be used as facts. • Results are durable. User-Centric • Providing real value to users.
  3. 3. The Approach: Step by Step Process (Scientific Method) 1- Observations 2- Hypothesis 3- Experiment 4- Refine 5- Validate Questions to answer. Observing a trend or a problem. What is a hypothesis? How to build one? & Good vs Bad hypothesis. How to test hypothesis? What data to look at? Refine hypothesis How to measure the effect? Success metrics! FYI: Scientific Method was founded by Ibn Al Haytham around 250 years ago.
  4. 4. Observations Observing a trend or a problem.
  5. 5. Sources for trends or problems Customer Service SalesOnline Discussions Product Vision
  6. 6. Sources for trends or problems
  7. 7. Impact Based Prioritization
  8. 8. Prioritization Methods Relative Weighting RICE Analysis Based on business metric Based on usage frequency Perfect for new features Perfect for optimizing existing features Considers development effort Considers development effort Reach, Impact, Confidence, Effort
  9. 9. Relative Weighting Prioritization Criteria Evaluation Factors Increase Sales; maximize ROI Increase customer trust Establish a competitive advantage Improve productivity Cost of development Value Score
  10. 10. Prioritized List Issue Type and ID ROI ICS ECA IP Total Value Value Percent Estimate Cost Percent Priority Add FB sign up Feature 10 Cash on Delivery Bug 3 Feature 7 1 6 1 3 3 1 6 1 6 6 1 1 1 8 1 1 6 6 6 3 11 11 14 11 21 10 20 20 40 55 Add Gmail sign up 3 6 6 6 21 15 Pin Address on map 6 6 3 6 21 35 Bug 5 3 3 3 3 12 25 Feature 8 1 3 6 1 11 40 8% 8% 11% 8% 16% 16% 16% 9% 8% 4% 8% 8% 15% 21% 6% 13% 10% 15% 2 1 1.37 0.54 0.76 2.66 1.23 0.9 0.54
  11. 11. RICE Analysis
  12. 12. Prioritized List
  13. 13. 1- Observations 2- Hypothesis 3- Experiment 4- Refine 5- Validate The Approach: Step by Step Process Questions to answer. Observing a trend or a problem. What is a hypothesis? How to build one? & Good vs Bad hypothesis. How to test hypothesis? What data to look at? Refine hypothesis How to measure the effect? Success metrics!
  14. 14. Hypothesis What is a hypothesis? How to build it? & Good vs Bad one.
  15. 15. What is a hypothesis? "If _____[I do this] _____, then _____[this]_____ will happen." If I added gmail login, then number of registered users will increase. If I include Cash on Delivery payment method during checkout, then number of purchases will increase. If I enable customers to pin their location on a map, then number of purchases will increase.
  16. 16. Hypotheses Tips Before you make a hypothesis, you have to clearly identify the question you are interested in studying. The question comes first A hypothesis is a statement, not a question Your hypothesis is not the scientific question in your project. The hypothesis is an educated, testable prediction about what will happen. Make it clear A good hypothesis is written in clear and simple language. Keep the variables in mind A good hypothesis defines the variables in easy-to-measure terms, like who the participants are, what changes during the testing. Make sure your hypothesis is “testable” Don't bite off more than you can chew! To prove or disprove your hypothesis, you need to be able to do an experiment and take measurements to see how two things are related. Make sure your hypothesis is a specific statement relating to a single experiment.
  17. 17. Good vs bad hypothesis Good Hypothesis Bad Hypothesis Testable Simple Written as a statement Establishes the participants & variables Predicts effect Not testable Not simply explained Written as a questions Doesn’t identify participants & variables Cannot use to predict effect
  18. 18. Good vs bad hypothesis Good Hypothesis Bad Hypothesis If I added gmail login, then number of registered users will increase. If I include Cash on Delivery payment method during checkout, then number of purchases will increase. If I enable customers to pin their location on a map, then number of purchases will increase. Would adding more login options increase number of registered users? Cash on delivery payment is requested by users We have 10,000 calls from users to track their orders
  19. 19. The point is to prove or disprove a hypothesis Disproving a hypothesis matters as much as proving it. Both leads to form better conclusions about the problem at hand!
  20. 20. Sources for trends or problems
  21. 21. 1- Observations 2- Hypothesis 3- Experiment 4- Refine 5- Validate The Approach: Step by Step Process Questions to answer. Observing a trend or a problem. What is a hypothesis? How to build one? & Good vs Bad hypothesis. How to test hypothesis? What data to look at? Refine hypothesis How to measure the effect? Success metrics!
  22. 22. Experiment How to test hypothesis? What data to look at?
  23. 23. Creating a mathematical model What questions do you want to answer? What data do you want to look for? Where the data you want is available? How to use the data available? How to verify the data? Will adding Gmail login improve sign up? How many customers have signed up using Gmail email? Customers data base Find % of customers signed up with Gmail. Gmail customers / all customers = % of gmail customers. Consider removing dummy created accounts
  24. 24. Creating a mathematical model What questions do you want to answer? What data do you want to look for? Where the data you want is available? How to use the data available? How to verify the data? Will adding Gmail login improve sign up? How many customers have signed up using Gmail email? Customers data base Find % of customers signed up with Gmail. Gmail customers / all customers = % of gmail customers. Consider removing dummy created accounts 150K 150K / 1.3M = 12%
  25. 25. 1- Observations 2- Hypothesis 3- Experiment 4- Refine 5- Validate The Approach: Step by Step Process Questions to answer. Observing a trend or a problem. What is a hypothesis? How to build one? & Good vs Bad hypothesis. How to test hypothesis? What data to look at? Refine hypothesis How to measure the effect? Success metrics!
  26. 26. Refine Refine mathematical model, removing bias!
  27. 27. Will adding Gmail login improve sign up? How many customers have signed up using Gmail email? Find % of customers signed up with Gmail. Gmail customers / all customers = % of gmail customers. Consider removing dummy created accounts 150K 150K / 1.3M = 12% What questions do you want to answer? What data do you want to look for? Where the data you want is available? How to use the data available? How to verify the data? Creating a mathematical model Customers data base
  28. 28. We live in a dynamic world, so always consider things in a timely manner!
  29. 29. Creating a mathematical model What questions do you want to answer? What data do you want to look for? Where the data you want is available? How to use the data available? How to verify the data? Will adding Gmail login improve sign up? How many customers have signed up using Gmail email in the past 6 months? Customers data base Find % of customers signed up with Gmail in the past 6 months. Gmail sign ups / all sign ups in the past 6 months = % of gmail customers. Consider removing dummy created accounts 120K 120K / 600K = 25%
  30. 30. 1- Observations 2- Hypothesis 3- Experiment 4- Refine 5- Validate The Approach: Step by Step Process Questions to answer. Observing a trend or a problem. What is a hypothesis? How to build one? & Good vs Bad hypothesis. How to test hypothesis? What data to look at? Refine hypothesis based on test. How to measure the effect? Success metrics!
  31. 31. Validate How to measure the effect? Success metrics!
  32. 32. What is a success metric? Metrics are a lens into your product’s health and performance.
  33. 33. Success Metric
  34. 34. Measure the uplift
  35. 35. 1- Observations 2- Hypothesis 3- Experiment 4- Refine 5- Validate The Approach: Step by Step Process Questions to answer. Observing a trend or a problem. What is a hypothesis? How to build one? & Good vs Bad hypothesis. How to test hypothesis? What data to look at? Refine hypothesis. How to measure the effect? Success metrics!
  36. 36. Thank you Abdallah Al-Khalidi Linkedin: abzkhaldi | Twitter: @abzkhaldi Email: abdallah.khalidi@gmail.com

×