Agile Metrics by Serge Kovaleff http://kiev.agilebasecamp.org/sergej-kovalev/

1. Agile Metrics
facebook.com/ SergeKovaleff @gmail.com
linkedin.com/in/
.blogspot.com

2. What is a Metric?

3. Process Improvement Cycle
Improve
Process
Measure
Define Process Control Process
Process
Execute
Process
PDCA (plan-do-check-act) by Dr. W. Edwards Deming
DMAIC (Define-Measure-Analyze-Improve-Control) in Six Sigma

4. A lot of Metrics different ...
• Business metrics • Process metrics
– RTF (Running Tested Features) – Agile practice maturity
– Impediments cleared per iteration
– Earned Business Value (EBV)
– Impediments carried over the next iteration
– Net Present Value (NPV) – User stories carried over the next iteration
– Internal Rate of Return (IRR) – User stories done per iteration
– Return of Investment (ROI) – Defects carried over the next iteration
• Code Metrics – Defects carried over the next iteration
– Team member loading
– Cyclomatic complexity
– Velocity of development
– Best practices violation – Backlog size
– Coding standards violation • Automation Metrics
– Possible bugs – Code coverage
– Code duplication – Number of builds per day
– Code coverage – Time taken per build
– Dead code – Number of failed/success builds
– Test Quality – Trends in code metrics
• Testing Metrics
• Design Metrics – Acceptance tests per story
– Code dependencies – Defects count per story
• Incoming (Affering Coupling) – Test time to run
• Outgoing (Efferent Coupling) – Tests run frequency
– Abstractness – Manual tests per story
• Number of abstract clases and interfaces – Automation percent
• Number of concrete classes – Time to fix tests
This slide is not for human reading 

6. Thermometer measures
a temperature of
the thermometer

7. There is no spoon THE METRIC

8. The single metric to control
• Quality
• Speed
• Productivity
• Customer satisfaction
• Etc.
… is still being looked for :)

9. Myth
Metrics
improve
productivity?

10. Myth
Myth: bigger velocity is better

11. WILL ACCURACY OF ESTIMATES Myth
INCREASED
IF REQUIRE COMMITTED
= 100% DELIVERED?

12. Choosing a GOOD metric

13. Encourage target behaviour

14. Measure trend

15. Easy to collect and measure

16. Inspire useful discussions (what have learnt)

17. Gives
periodic
feedback

18. Can make conclusions
and fix the process

19. Measurement effect
Tell me how you will measure me and
I’ll tell you how I will behave.
Eli Goldratt “The Goal” (2004)

20. Leading vs. Lagging
Leading Lagging

21. Leading vs. Lagging
Leading Lagging

22. What to measure?
• Productivity metrics
Productivity
• Predictability metrics
Predict
• Quality metrics
Quality
• Value metrics
Value

23. And now ...
The Metrics!

24. Productivity
Productivity metrics

25. Velocity Leading
Productivity
Myth

26. Risk of Story Points inflation

27. Truth: Story points are relative

28. Leading
Work In Progress Productivity

29. Leading
Productivity
Story
Cycle
Time

30. Predictability metrics
Predict

31. Sprint Burn Down Chart Leading
Predict

33. Release Burn Down Chart Leading
Predict

34. Leading
Predict
Burn Up Chart

35. ROI (Return-on-Investment)
• Backlog item - Business-Value
• Value in $

36. Quality metrics

37. Leading
Technical Debt Quality
• Backlog
– Inner vs. External quality
– Testing Automation
– Reengineering
– Maintainability
– Documentation

39. Leading
Running Automated Tests Quality
• Amount of tests
– Functional
– Unit
• Code Coverage
• Must be increasing
• Tests must pass 

40. Lagging
Quality
Post
Sprint
Defect
Arrival

41. Lagging
Post Release Defect Arrival Quality
THE SAME
... after global Release

42. Lagging
The ONLY valid Quality metric Quality

43. Value Metrics

44. Lagging
Customer Satisfaction Survey Value
The best way to find out
whether your customers are
ask them
satisfied is to

45. Basic satisfaction How satisfied are you with
 your purchase?
 the service you received?
 with our company overall?

46. Customer How likely are you to
 buy from us again?
loyalty  recommend our
product/service to others?
 recommend our
company to others?

47. Pr omoters Detractors
Total

48. Lagging
Value
Customer
Satisfaction
Survey

50. How often: Lagging
“So often, that get more information, Value
But not so often that annoy”

51. Key Drivers of satisfaction:
• Conflicting goals
• What does Client feel important
• Scatter diagram (Key Driver Chart)

52. Lagging
Employee Satisfaction Survey Value

53. Lagging
Value

54. Short Summary ∑®
• Goal
– Why do you want to measure?
– What would be the result of the intrusion?
• Classification
– Time
• Leading
• Lagging
– Subject
• Productivity
– Story Cycle Time
• Predictability
– Burn up Chart
• Quality
– Tech Debt
• Value
– Net promoters

55. Questions time 

56. Links
• http://agilebasecamp.org/
• http://blog.scrumtrek.ru/2010/01/wip-story-cycle-time.html
• http://blog.scrumtrek.ru/2010/01/velocity.html
• http://www.scrum.org.za/uploads/2009/09/measuring-for-results-
2-small.pdf
• http://scrumorlando09.pbworks.com/Scrum-Metrics-and-Myths
• http://www.slideshare.net/petebehrens/measuring-agility-top-5-
metrics-and-myths
• http://www.slideshare.net/alimenkou/agile-metrics-2725666
• http://management.about.com/od/competitiveinfo/a/CustomerSatS
urv.htm
• http://en.wikipedia.org/wiki/Net_Promoter