O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Agile Metrics

1.120 visualizações

Publicada em

My keynote at Agile Canvass conference in Chennai today. In this talk, I also talk about my "Crawl-Walk-Run-Fly" metrics framework

Publicada em: Software

Agile Metrics

  1. 1. Agile Metrics Tathagat Varma Sr Director, Strategy & Ops, WalmartLabs
  2. 2. Disclaimer These are my personal views! Pic: https://motaen.com/wallpapers/source/id/51908
  3. 3. In today’s world, mere opinions are not enough anymore!
  4. 4. Why measure?
  5. 5. Metrics help improve visibility, and plan more realistically
  6. 6. Unfortunately, most software metrics are often “watermelon metrics”
  7. 7. “How does a project get to be a year late? …One day at a time.” Fred Brooks, The Mythical Man Month, 1975
  8. 8. “When one hears of disastrous schedule slippage in a project, he imagines that a series of major calamities must have befallen it. Usually, however, the disaster is due to termites, not tornadoes; and the schedule has slipped imperceptibly but inexorably. Indeed, major calamities are easier to handle; one responds with major force, radical reorganization, the invention of new approaches. The whole team rises to the occasion. But the day-by-day slippage is harder to recognize, harder to prevent, harder to make up. Yesterday a key man was sick, and a meeting couldn't be held. Today the machines are all down, because lightning struck the building's power transformer. Tomorrow the disk routines won't start testing, because the first disk is a week late from the factory. Snow, jury duty, family problems, emergency meetings with customers, executive audits— the list goes on and on. Each one only postpones some activity by a half-day or a day. And the schedule slips, one day at a time.”
  9. 9. Most common approach is over- measurement which leads to no meaningful decision- making
  10. 10. So how to design, capture and leverage metrics effectively?
  11. 11. Models are… well, just models!
  12. 12. Metrics are not panacea!
  13. 13. People will invariably (always?) interpret metrics to suit themselves!
  14. 14. Metrics can be (and most probably will be!) gamed!!!
  15. 15. Good metrics (often look bad but) drive improvement
  16. 16. Simplicity is the key!
  17. 17. Metrics without business alignment = ignorance is bliss!
  18. 18. A great metrics dashboard works just like an Air Traffic Controller Pic: https://www.telegraph.co.uk/travel/travel-truths/Are-airlines-exaggerating-flight-times-so-theyre-never-late/
  19. 19. Correlation doesn’t imply causation!
  20. 20. Horses for courses it is!
  21. 21. Metrics need to show the trajectory to be useful
  22. 22. Thinking in systems might be a good idea! • Measure adoption efforts • Examples: #trainings •
  23. 23. Unit of measurement is almost as important as the purpose for which it is used!
  24. 24. Know your audience!
  25. 25. Identify the “vital few” instead of the “trivial many” Pic: https://www.financialexpress.com/industry/sme/what-startups-can-learn-from-paretos-8020-rule/1122669/
  26. 26. Bad news is sometimes (often?) good news!
  27. 27. What is your “North Star Metric”?
  28. 28. Great metrics help identify the root cause and lead us towards the right solution!
  29. 29. Know the difference and use it accordingly: Accuracy vs Precision
  30. 30. Right-size your metrics
  31. 31. Without a closed-loop feedback system, even the best of the metrics are rather useless! Pic: https://instrumentationtools.com/open-loop-and-closed-loop-animation/
  32. 32. “Crawl Walk Run Fly” Metrics Framework
  33. 33. Crawl • Measure inputs or efforts • Encourage participation at the starting point without pre-conditions or judgment • Examples include: • # trainings done • # teams adopting agile • # scrummasters identified • etc.
  34. 34. Walk • Measure internal efficiencies achieved • Encourage behaviors without looking at results • Examples include • (yeah, the favorite) velocity • % test automation • % code coverage • etc.
  35. 35. Run • Measure effectiveness delivered • Look at the business results! • Examples: • Revenue $$$ • Proxy to revenue - engagement ? • Dave McClure’s AARRR Pirate Metrics • etc.
  36. 36. Fly • Measure impact made or the value delivered to the customers or the society at large • Serve the higher purpose beyond business goals • Examples: • People lifted out of poverty since 80s • Net impact on planet earth! • Improve quality of life • etc.
  37. 37. Recap
  38. 38. One last thing… The most important metric is… Pic: https://www.thehindu.com/opinion/open-page/india-is-happy-and-the-rankings-just-dont-match/article18190887.ece