1. Agile Metrics and Quality

2. Metrics -> Improvements
Use metrics to
• Improve quality
• Shorten product cycle time
• Decrease costs
• Decrease risks
• Increase predictability and reliability

3. Control Limits and Variances
We can have limits for
• WIP
• Defects
• Team size
• Velocity
• Time-boxes
• Others?
We can analyze variances to understand
the special causes of variation

4. Focus on Trends
• Trends matter more than values
• Trends and spikes are only the symptoms
• Trends are leading indicators
Images from MountainGoat Software

5. Improve quality
◦ Quality is measured by passed tests and
customer acceptance
◦ Quality is defined by Definition of Done
and Acceptance Criteria
◦ Defect trends to use in Agile projects
 Number of open defects
 Escaped defects
 Defect Cycle Time
 Defect spill-over
◦ Test Automation is critical!

6. Defect Injection and
Resolution
What is the trend of total Active Defects?
Image from MountainGoat Software

7. Analyzing and Preventing
Pareto Chart Defects
Fishbone / 5 Why Technique

8. Escaped Defects
• Used to measure the quality of delivered
code
 Calculation:
9
◦ For all releases, find 8
7
all defects related to
6
the release found
5
after the release date
4
3
◦ Add up all the defects 2
1
◦ Can be captured per
0
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
day / week / month /
… or per sprint / Escaped Defects over Time
release

9. Defect Cycle Time
• Rapid resolution of defects is important
• Average bug-fix time can also be tracked for
different priority bugs
Defect Cycle Time
20
18
16
Desired Threshold
14
12
Hours
10
8
6
4
2
0
1 2 3 4 5 6 7 8 9 10
Sprint No.

10. Defect Spill-over
• Resolution of defects for a sprint story within the sprint is
important
• Definition of Done can include a criteria like “No P1 defects”
• We can also tracked whether spilled-over defects are being
closed in the next sprint
Defect Spill Over
9
8
7
No of defects
6
5
4
3
2
1
0
1 2 3 4 5 6 7 8 9 10
Sprint No

11. Test Automation
 Critical factor for regression testing
 Coverage should be as high as possible

12. Shorten product cycle time
 Cumulative Flow Diagrams
◦ Lead Time
◦ Cycle Time

13. Features CFD – An Elaboration
Time

14. What does this CFD say?
Not started
Started
Completed
Too much Work-In-Progress!
Image from MountainGoat Software

15. Identifying bottlenecks with a
CFD
1 2 3 4 5 6 7 8 8 10 11 12 13
Day
Where is the bottleneck?
In DB Procs – After the widening Analysis phase
Image from http://www.soliantconsulting.com

16. Decrease costs
• Earned Value Management
• Wastage

17. Earned Value Analysis
PV – Planned Value
EV - Earned Value
AC - Actual cost
BAC - Budget @
Completion
EAC - Estimate @
Completion
ETC - Estimate to
CompletionCost Variance (CV) = EV – AC
VAC- Variance @ Completion (SV)
Schedule Variance = EV – PV
Cost Performance Index (CPI) = EV / AC
Schedule Performance Index (SPI) = EV / PV
Estimate at Completion (EAC) = BAC / CPI or
= AC + BAC – EV
Estimate To Complete (ETC) = EAC – AC
Variance at Completion (VAC) = BAC – EAC

18. EVM - Interpretation
PV EV AC Schedule Cost
$1 $1 $1 On schedule On cost
$2 $2 $1 On schedule Under cost
$1 $1 $2 On schedule Over cost
$1 $2 $2 Ahead of schedule On cost
$1 $2 $3 Ahead of schedule Over cost
$1 $2 $1 Ahead of schedule Under cost
$3 $2 $1 Behind schedule Under cost
$2 $1 $3 Behind schedule Over cost
$2 $1 $1 Behind schedule On cost

19. EVM - Progress
EV PV AC
CV
SV
Ref: Earned Value and Agile Reporting
Anthony Cabri, Mike Griffiths

20. Wastage
 Any time not dedicated to working on the
highest priority story / task on the sprint
backlog
 Should be tracked over time and by cause
Wastage (in hrs) Wastage (in hrs)
60 160
140
50 120
100
40 80
60
30 40
20
20 0
10
0
1 2 3 4 5 6 7 8 9 10 11 12

21. Decrease risks
• WIP Limits
• Small Iterations
• More predictability

22. Increase Predictability /
Reliability
• Velocity
• Lead Time
• Cycle Time

23. Metrics to Avoid
• Velocity comparisons
• Productivity e.g. Story Points per day
• Effort Estimation accuracy
• Number of stories completed
• LOC/Function Points
• Code quality

24. Questions?
Contact
Anu.khendry@gmail.com