If you are struggling to forecast project delivery dates and cost, or you want to eliminate the story estimation process because you feel it is waste, or you need to build the business case for hiring more staff, then this session is relevant to you. All estimates have uncertainty, and understanding how multiple uncertain factors compound is the first step to improving project and team predictability. A major benefit of Lean is the low weight capture of cycle time metrics. This session looks at how to use historical cycle time data to answer questions of forecasting and staff skill balancing. This session compares the benefits of using cycle time for analysis over current planning techniques such as velocity, burn-down charts, and cumulative flow diagrams. This session takes you on a journey of what to do after capturing cycle time data or what to do if you have no history to rely upon. Reducing reliance on developer estimation (popularized by the twitter hashtag of #NoEstimates movement) is good general advice, having the tools to plan and manage teams and projects is still important to maintain support at the executive level. This session details the approaches to getting the numbers you need to have whilst minimizing un-necesary overhead and estimating ONLY this factors that matter most.

1. LKCE 2013 – Modern Management Methods
Cycle Time Analytics
Making decisions using Lead Time and Cycle Time to avoid needing estimates for
every story
Troy Magennis
@t_magennis
Slides at bit.ly/agilesim

2. 2
@t_Magennis slides at bit.ly/agilesim

3. Forecasting Myths
1. You need lots of data to forecast
False, if we understand
2.
the behavior of a domain
(cycle time for example)
False. Most uncertainty
Statistical forecasting requires
we can be confident with the
is reduced within
no mathematics
complex initial data samples
first few
3. If
3
False. Everything you
we havesee today uses nothing
no data we can’t
more than the + and /
statistically forecast
operations!
Commercial in confidence

4. Q. What is the chance of the 4th sample
being between the range seen after the
first three samples?
Actual Maximum
(no duplicates, uniform distribution, picked at random)
2
4
3
1
Actual Minimum
@t_Magennis slides at bit.ly/agilesim

5. Q. What is the chance of the 4th sample
being between the range seen after the
first three samples?
Actual Maximum
(no duplicates, uniform distribution, picked at random)
Highest sample
2
?
?
4
3
?
1
?
Lowest sample
Actual Minimum
@t_Magennis slides at bit.ly/agilesim

6. Q. What is the chance of the 4th sample
being between the range seen after the
first three samples?
Actual Maximum
(no duplicates, uniform distribution, picked at random)
Highest sample
25% chance higher
than highest seen
2
25% lower than highest
and higher than second highest
4
3
25% higher than lowest and
lower than second lowest
1
Lowest sample
Actual Minimum
25% lower than
lowest seen
@t_Magennis slides at bit.ly/agilesim
A. 50%
% = (1 - (1 / n – 1)) * 100

7. Q. What is the chance of the 12th sample
being between the range seen after the
first three samples?
Actual Maximum
(no duplicates, uniform distribution, picked at random)
Highest sample
2
9
5
5% chance higher than
highest seen
?
3
12
4
10
6
11
?
7
1
8
Lowest sample
Actual Minimum
5% lower than
lowest seen
@t_Magennis slides at bit.ly/agilesim
A. 90%
% = (1 - (1 / n – 1)) * 100

8. # Prior Samples
3
4
5
6
7
8
9
10
11
12
13
15
17
20
Prediction Next Sample Within Prior Sample Range
50%
67%
75%
80%
83%
86%
88%
89%
90%
91%
92%
93%
94%
95%
@t_Magennis slides at bit.ly/agilesim

9. Four people arrange a
restaurant booking after work
Q. What is the chance they
arrive on-time to be seated?
9
@t_Magennis slides at bit.ly/agilesim

10. 10
15 TIMES more likely at least on person is late
1 in 16 EVERYONE is ON-TIME
Person 1 Person 2 Person 3 Person 4
Commercial in confidence

11. 11

12. Estimating the wrong things
and getting a poor result
doesn’t mean we shouldn’t
estimate at all
We just need to estimate
things that matter most
13
Commercial in confidence

13. 85%
Forecasts are attempts to
Change of
At Least 2
th August
15answer questions about
Teams
2013
future events. They are an
estimate with a stated
Definitely
Greater
uncertainty
than
$1,000,000
17
@t_Magennis slides at bit.ly/agilesim

14. There is NO single
forecast result
Uncertainty In = Uncertainty Out
There will always be many
possible results, some more likely and this is the
key to proper forecasting
@t_Magennis slides at bit.ly/agilesim

15. Likelihood
Probabilistic Forecasting combines many uncertain
inputs to find many possible outcomes, and what
outcomes are more likely than others
50%
Possible
Outcomes
50%
Possible
Outcomes
Time to Complete Backlog
19
@t_Magennis slides at bit.ly/agilesim

16. Likelihood
Did the Obama 2012 Campaign Fund Advertising to
Achieve 50% Chance of Re-election?
85% Possible
Outcomes
15%
Time to Complete Backlog
20
@t_Magennis slides at bit.ly/agilesim

17. Task Uncertainty – Summing Variance
1
2
3
4
Source attribution: Aidan Lyon, Department of Philosophy. University of Maryland, College Park. “Why Normal
Distributions Occur” http://aidanlyon.com/sites/default/files/Lyon-normal_distributions.pdf
21
@t_Magennis slides at bit.ly/agilesim

18. Decision Induced Uncertainty
Every choice we make changes the outcome
Planned / Due Date
July
Cost of Delay
Dev Cost
Staff
Actual Date
August
September
October
Forecast Completion Date
22
@t_Magennis slides at bit.ly/agilesim
November
December

19. What is modelling and how to use cycle time
MODELING AND CYCLE TIME
23

20. A model is a tool used to
mimic a real world process
Models are tools for low-cost
experimentation
@t_Magennis slides at bit.ly/agilesim

21. Simple
Depth of Forecasting models
Linear Projection
System Cycle Time
Diagnostic
Partitioned Cycle Time
26
Simulated process
Commercial in confidence

22. Simple Cycle Time Model
Amount of
Work
(# stories)
Lead Time
or Cycle
Time
Random Chance
/ Risk / Stupidity
27
@t_Magennis slides at bit.ly/agilesim
Parallel
Work in Proc.
(WIP)

23. Capturing Cycle Time and WIP
Story
Start Date
Completed Cycle Time
Date
(days)
1
2
3
1 Jan 2013
5 Jan 2013
5 Jan 2013
15 Jan 2013
12 Jan 2013
4
5
6
7
8
9
28
10
6 Jan 2013
3 Jan 2013
7 Jan 2013
10 Jan 2013
10 Jan 2013
13 Jan 2013
15 Jan 2013
14
Date “Complete” – Date “Started”
7 Jan 2013
18 Feb 2013
22 Jan 2013
18 Jan 2013
26 Jan 2013
Use with attribution

24. Capturing Cycle Time and WIP
Story
Start Date
Completed Cycle Time
Date
(days)
Date
1 Jan
1
2
3
1 Jan 2013
5 Jan 2013
5 Jan 2013
15 Jan 2013
12 Jan 2013
3 Jan
4 Jan
5 Jan
4
5
6
7
8
9
29
10
6 Jan 2013
3 Jan 2013
7 Jan 2013
10 Jan 2013
10 Jan 2013
13 Jan 2013
15 Jan 2013
7 Jan 2013
18 Feb 2013
22 Jan 2013
6 Jan
7 Jan
8 Jan
9 Jan
10 Jan
…
15 Jan
WIP
5
Count of Started, but
26 Jan 2013
Not completed work
18 Jan 2013
Use with attribution

25. Capturing Cycle Time and WIP
Story
Start Date
Completed Cycle Time
Date
(days)
Date
1 Jan
WIP
1
1
2
3
1 Jan 2013
5 Jan 2013
5 Jan 2013
15 Jan 2013
12 Jan 2013
3 Jan
4 Jan
5 Jan
2
2
3
4
5
6
7
8
9
30
10
6 Jan 2013
3 Jan 2013
7 Jan 2013
10 Jan 2013
10 Jan 2013
13 Jan 2013
15 Jan 2013
6 Jan
7 Jan
8 Jan
9 Jan
10 Jan
…
15 Jan
4
5
5
5
7
…
7
14
7
7 Jan 2013
18 Feb 2013
22 Jan 2013
4
42
12
18 Jan 2013
8
26 Jan 2013
13
Use with attribution

26. 31
Trial 1 Trial 2 Trial 100
9
13 13
5
Sum:
51
1
4
7
5
11
28
…
35
19
5
13
11
83
11
Fancy term for turning a small set
of samples into a larger set:
Bootstrapping
Use with attribution
By repetitively sample
we build trial
hypothetical “project”
completions

27. Sum Random Numbers
Historical Story Cycle Time Trend
25
11
29
43
34
26
31
45
22
27
More often
Less often
Sum:
31
43
65
45
8
7
34
73
54
48
295
410
…..
Basic Cycle Time Forecast Monte Carlo Process
1. Gather historical story lead-times
2. Build a set of random numbers based on pattern
3. Sum a random number for each remaining story
to build a potential outcome
4. Repeat many times to find the likelihood (odds)
to build a pattern of likelihood outcomes
Days To Complete
19
12
24
27
21
3
9
20
23
29
187

28. 1. Historical Cycle Time
Monte Carlo Analysis =
Process to Combine
Multiple Uncertain
Measurements /
Estimates
6. Phases
2. Planned Resources/ WIP
4. Historical Scope
Creep Rate
3. The Work (Backlog)
Backlog
Feature 1
Feature 2
Feature 3
(optional)
34
5. Historical Defect Rate and Cycle Times
(optional)
@t_Magennis slides at bit.ly/agilesim

29. 35
Commercial in confidence

30. 36
@t_Magennis slides at bit.ly/agilesim

31. Y-Axis:
Number of
Completed
Stories
Project Complete
Likelihood
Range of complete
stories probability
0 to
50%
X-Axis: Date
37
@t_Magennis slides at bit.ly/agilesim
50 to
75%
>
75%

32. How certain
based on model
forecast
Further
calculations to
make economic
tradeoffs
38
Commercial in confidence

33. What is 10% Cycle Time
Reduction Worth?
Baseline
Staff Cost Cost of Delay Total Cost
€912.000 + €190.000 = €1.102.000
Experiment: 10% Cycle Time Reduction
Staff Cost Cost of Delay Total Cost
€883.200 + €177.419 = €1.060.619
Opportunity: €41.381
39

34. 1. Historical Cycle Time
Design
Develop
Test
Design
Develop
A Process to Combine
Multiple Uncertain
Measurements /
Estimates is Needed
Test
2. Planned Resources/ Effort
4. Historical Scope
Creep Rate
3. The Work (Backlog)
Backlog
Feature 1
Feature 2
Feature 3
(optional)
40
5. Historical Defect Rate & Cycle Times
(optional)

35. What is One Designer Worth?
Baseline
Staff Cost Cost of Delay Total Cost
€912.000 + €190.000 = €1.102.000
Experiment: + 1 Designer
Staff Cost Cost of Delay Total Cost
€610.400 + €5.000
= €615.400
Opportunity: €486.600
41

36. FORECASTING STRATEGIES
42

37. When you have historical data
1. Model Baseline
using historically
known truths
The
Past
2. Test Model
against historically
known truths
3. Forecast
The
Future

38. Compare Model vs Actual Often
Range of complete
probability
Actual results to compare
if model is predictable
45
@t_Magennis slides at bit.ly/agilesim

39. When you have no historical data
The
Future
@t_Magennis slides at bit.ly/agilesim

40. If we understand how cycle time is
statistically distributed, then an
initial guess of maximum allows an
accurate inference to be made
Alternatives • Borrow a similar project’s data
• Borrow industry data
• Fake it until you make it… (AKA guess range)
49
@t_Magennis slides at bit.ly/agilesim

41. Probability Density Function
1997: Industrial Strength Software 2002: Metrics and Models in
by Lawrence H. Software Quality Engineering
(2nd Edition) [Hardcover]
Putnam , IEEE , Ware Myers
Stephen H. Kan (Author)
0.32
0.28
0.24
0.2
0.16
0.12
0.08
0.04
0
-10
0
10
20
30
40
50
60
70
80
x
Histogram
50
Gamma (3P)
Lognormal
Rayleigh
@t_Magennis slides at bit.ly/agilesim
Weibull
90
100
110
120
1

42. Waterfall
Weibull Shape
Parameter = 2
AKA Rayleigh
51
Commercial in confidence

43. Agile / Lean / Kanban
Weibull Shape
Parameter = 1.5
52
Commercial in confidence

44. Typical Operations / Release
Weibull Shape
Parameter = 1
AKA Exponential
53
Commercial in confidence

45. Shape – How Fat the
distribution. 1.5 is a
good starting point.
Probability Density Function
0.28
0.24
f(x)
0.2
Scale – How Wide in
Range. Related to the
Upper Bound. *Rough*
Guess: (High – Low) / 4
Location – The
Lower Bound
0.16
0.12
0.08
0.04
0
0
10
20
30
40
50
60
70
x
Histogram
54
Weibull
@t_Magennis slides at bit.ly/agilesim
80
90
100
110
120

46. What Distribution To Use...
• No Data at All, or Less than < 11 Samples (why 11?)
– Uniform Range with Boundaries Guessed (safest)
– Weibull Range with Boundaries Guessed (likely)
• 11 to 30 Samples
– Uniform Range with Boundaries at 5th and 95th CI
– Weibull Range with Boundaries at 5th and 95th CI
• More than 30 Samples
– Use historical data as bootstrap reference
– Curve Fitting software
55
@t_Magennis slides at bit.ly/agilesim

47. Questions…
• Download the slides (soon) and software at
http://bit.ly/agilesim
• Contact me
– Email: troy.Magennis@focusedobjective.com
– Twitter: @t_Magennis
• Read:
56