1.
Dimitar Bakardzhiev
Managing Director
Taller Technologies Bulgaria
@dimiterbak
#NoEstimates Project
Planning using Monte Carlo
simulation

2.
Clients come to us with an idea for a new
product and they always ask the questions -
how long will it take and how much will it cost
us to deliver? They need a delivery date
and a budget estimate.

3.
Reality is uncertain, yet we
as software developers are
expected to deliver new
products with certainty.

4.
To increase the chances of
project success we need to
incorporate the uncertainty
in our planning and exploit it.

5.
WE CAN’T CONTROL THE WAVES OF
UNCERTAINTY, BUT WE CAN LEARN
HOW TO SURF!

6.
TO ME #NOESTIMATES MEANS
No effort estimates
Effortless
estimates
No estimates
of effort

7.
Deterministic planning used
these days forces certainty
on uncertain situations and
masks the uncertainty
instead of highlighting it.

8.
Project management paradigm
is based on 1st principle of
Scientific Management namely
“In principle it is possible to
know all you need to know to be
able to plan what to do”.

9.
Project management paradigm
believes uncertainty play a role
in project management
uncertainty could be eliminated
by a more detailed planning.

10.
We challenge the project management
paradigm and suggest that for
planning purposes it is better to model
projects as a flow of work items
through a system.

11.
A project is a batch of work items
each one representing
independent customer value that
must be delivered on or before
due date.

12.
We don’t try to estimate the size
of the work items. There are only
two "sizes" - “Small Enough" and
“Too Big". "Too big" should be
split and not allowed to enter the
backlog.

13.
A Project in a Kanban System
Input Queue
DEPLOYED!
Project Backlog
Development Test QA
WIP 5 WIP 4 NO WIPWIP 2

14.
High-level probabilistic planning
• The initial budget and the range of the time frame
• Does not include detailed project plans
• The plan is created with the appropriate buffers
• Schedules are the execution of the high-level plan
• Keep focus on the project intent

15.
Reference class forecasting
Reference class forecasting
promises more accuracy in
forecasts by taking an "outside
view" on the project being
forecasted based on knowledge
about actual performance in a
reference class of comparable
projects.
Daniel Kahneman

16.
Reference class forecasting
• Identification of a relevant reference class of past,
similar projects. The class must be broad enough to be
statistically meaningful but narrow enough to be
comparable with the specific project.
• Establishing a probability distribution for the selected
reference class.
• Comparing the new project with the reference class
distribution, in order to establish the most likely
outcome for the new project.

17.
IDENTIFICATION OF A REFERENCE CLASS OF
SIMILAR PROJECTS

18.
Are the Team structures comparable?

19.
Are the Technologies used
comparable?

20.
Are the Development processes
comparable?

21.
Are the Client types comparable?
http://blog.7geese.com/2013/07/04/7-reasons-why-i-decided-to-work-for-a-startup/

22.
Are the Business domains
comparable?
http://www.mindoceantech.com/

23.
ESTABLISHING A PROBABILITY
DISTRIBUTION FOR THE SELECTED
REFERENCE CLASS

24.
What metric will be used in the
forecast?
The metric should allow us:
• to take an “outside view” on the development
system that worked on the project
• calculating delivery time
• make sense from client’s perspective.

25.
Takt Time!

26.
Takt Time is the average time
between two successive
deliveries

27.
How manufacturing measure Takt
Time?

28.
How knowledge workers measure
Takt Time?

29.
Inter-Departure Time (IDT) is the time
between two successive deliveries
Start 5 days 7 days 2 days 2 days 1 day 5 days Finish
IDT = 0 days
IDT = 0 days
IDT = 5 days
IDT = 7 days
Project delivery time (T) = 5 + 7 + 2 + 2 + 1 + 5 = 22 days

30.
𝑇 =
𝑖=1
𝑁
𝐼𝐷𝑇𝑖
= 5 + 7 + 0 + 2 + 0 + 0 + 2
• T is the time period over which the project was delivered
• IDT is the inter-departure time or the time between two
successive deliveries

31.
Takt Time
𝑇𝑇 =
𝑇
𝑁
=
𝑖=1
𝑁
𝐼𝐷𝑇𝑖
𝑁
• T is the time period over which the project was delivered
• N is the number of items to be delivered in period [0,T]
• 𝑇𝑇 is the Takt Time for period [0,T]

32.
TT calculation
𝑇𝑇 =
𝑇
𝑁
=
22 𝑑𝑎𝑦𝑠
10 𝑠𝑡𝑜𝑟𝑖𝑒𝑠
= 2.2 𝑑𝑎𝑦𝑠/𝑠𝑡𝑜𝑟𝑦

33.
Project Delivery time
𝑇 = 𝑁𝑇𝑇
• T is the time period over which the project will be delivered
N is the number of items to be delivered in period [0,T]
• 𝑇𝑇 is the Takt Time for period [0,T]

34.
Project Delivery time
𝑇 = 𝑁𝑇𝑇 =
45 𝑠𝑡𝑜𝑟𝑖𝑒𝑠
2.2 𝑑𝑎𝑦𝑠
𝑠𝑡𝑜𝑟𝑦
= 99 𝑑𝑎𝑦𝑠

35.
We should NOT use the Takt Time as
a single number but a distribution of
the Takt Time instead!

36.
Bootstrapping
• Introduced by Bradley Efron in 1979
• Based on the assumption that a random sample is a
good representation of the unknown population.
• Does not replace or add to the original data.
• Bootstrap distributions usually approximate the shape,
spread, and bias of the actual sampling distribution.
• Bootstrap is based on the assumption of
independence.

37.
1. Have Inter-Departure Time (IDT) sample of size n
2. Have the number of work items delivered (N)
3. Draw n number of observation 𝑰𝑫𝑻𝒊 with replacement
out of the sample from step 1
4. Calculate Project Delivery time (T) for the sample from
step 2 using 𝑻 = 𝑰𝑫𝑻𝒊
5. Calculate Takt Time (𝑇𝑇) by 𝑻𝑻 = 𝑻/𝑵 using T from
step 4 and N from step 2
6. Repeat many times
7. Prepare distribution for Takt Time (𝑇𝑇)
Bootstrapping the distribution of Takt
Time

38.
Example: Bootstrapping Takt Time (𝑇𝑇)
Sampled IDT data
𝑰𝑫𝑻𝒊=(0,0,1,1,1,2,2,2,5,7)
𝑻 = 𝑰𝑫𝑻𝒊 = 𝟐𝟏 𝒅𝒂𝒚𝒔
𝑻𝑻 = 𝑻/𝑵 = 2.1 days/story
Another 998 draws with replacement
Historical IDT data
𝑰𝑫𝑻𝒊=(0,0,0,0,1,2,2,5,5,7)
𝑻 = 𝑰𝑫𝑻𝒊 = 𝟐𝟐 𝒅𝒂𝒚𝒔
𝑻𝑻 = 𝑻/𝑵 = 2.2 days/story
Sampled IDT data
𝑰𝑫𝑻𝒊=(0,1,1,1,1,2,5,5,5,7)
𝑻 = 𝑰𝑫𝑻𝒊 = 𝟐𝟖 𝒅𝒂𝒚𝒔
𝑻𝑻 = 𝑻/𝑵 = 2.8 days/story
1st draw with
replacement
1000th draw with
replacement

39.
Result: Takt Time (𝑇𝑇) distribution
Median 2,2
STD 0,788833
Average T 2,1943
85 Perc 3
95 Perc 3,5
Mode(s) 2,4
SIP size 1000

40.
Stochastic Information Packet (SIP)
• Comprised of a list of trials of some uncertain
parameter or metric generated from historical data
using Monte Carlo simulation (resampling)
• Represents an uncertainty as an array of possible
outcomes (distribution)
• It is unique per context (business domain, team,
delivery process used etc.)

41.
COMPARING THE NEW PROJECT WITH
THE REFERENCE CLASS
DISTRIBUTION

42.
𝑇 = 𝑁𝑇𝑇 assumes linear delivery rate
Project
Delivery
Time (T)
Project
Delivery
Time (T)
Completed
Work (N)
22 days
10
work
items

43.
Most projects have non-linear
delivery rate

44.
Z-curve

45.
Each leg of the Z-curve is
characterized by:
• Different work type
• Different level of variation
• Different staffing in terms of headcount and level of
expertise

46.
1st leg – Setup time
• climbing the learning curve
• conducting experiments to cover the riskiest work
items
• Innovation!
• setting up environments
• adapting to client’s culture and procedures
• understanding new business domain
• mastering new technology

47.
2nd leg – Productivity period
If the project is scheduled properly the system should be
like a clockwork – sustainable pace, no stress, no
surprises…

48.
3rd leg – Cleaning up
• Clean up the battlefield
• Fix some outstanding defects
• Support the transition of the project deliverable into
operation
https://www.ocoos.com/me/professional-dog-training-in-home/

49.
Project delivery time T
𝑇 = 𝑇𝑧1 + 𝑇𝑧2 + 𝑇𝑧3
Where:
𝑇𝑧1 – is the duration of the 1st leg of the Z-curve
𝑇𝑧2 – is the duration of the 2nd leg of the Z-curve
𝑇𝑧3 – is the duration of the 3rd leg of the Z-curve

50.
Project delivery time T
𝑇 = 𝑁𝑧1 𝑇𝑇𝑧1 + 𝑁𝑧2 𝑇𝑇𝑧2 + 𝑁𝑧3 𝑇𝑇𝑧3
Where:
𝑇𝑇𝑧1 is the Takt Time for the 1st leg of the Z-curve
𝑇𝑇𝑧2 is the Takt Time for the 2nd leg of the Z-curve
𝑇𝑇𝑧3 is the Takt Time for the 3rd leg of the Z-curve
𝑁𝑧1 is the number of items delivered during the 1st leg of the Z-curve
𝑁𝑧2 is the number of items delivered during the 2nd leg of the Z-curve
𝑁𝑧3 is the number of items delivered during the 3rd leg of the Z-curve

51.
Monte Carlo simulation of Project
Delivery Time (T) based on Z-curve
1. Have three Takt Time SIPs (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3) each
one of size n for each of the three legs of the Z-curve
2. Have the number of work items to be delivered for
each of the three legs of the Z-curve (𝑁𝑧1, 𝑁𝑧2, 𝑁𝑧3)
3. Draw one observation out of the n, with replacement
(bootstrap) from each of (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3)
4. Calculate Project Delivery time (T) for the sample from
step 3 using 𝑇 = 𝑁𝑧1 𝑇𝑇𝑧1 + 𝑁𝑧2 𝑇𝑇𝑧2 + 𝑁𝑧3 𝑇𝑇𝑧3
5. Repeat many times
6. Prepare Delivery time (T) probability distribution

52.
EXAMPLE: MONTE CARLO SIMULATION OF
PROJECT DELIVERY TIME (T)

53.
The New Project to be delivered
• THE SAME Fortune 500 Staffing company
• THE SAME development organization
• THE SAME technology – Java; Spring; Oracle;
• Delivery time TO BE PREDICTED

54.
Takt Time distributions for each of the
three legs of Z-curve for the reference
class

55.
Project scope
After some analysis the team have broken down the
requirements into user stories, accounting for Cost of
Delay, added work items for Dark matter and Failure
load and decided that:
• 12 stories TO BE delivered in the 1st leg of Z-curve
• 70 stories TO BE delivered in the 2nd leg of Z-curve
• 18 stories TO BE delivered in the 3rd leg of Z-curve

56.
Monte Carlo simulated summation of…
…will give us the time needed to deliver the project!
12 work items 70 work items 18 work items

57.
Monte Carlo simulation of Project
Delivery Time (T)
Simulated one Project Delivery
Time value
𝑻 = 𝑁𝑧1 𝑇𝑇𝑧1 + 𝑁𝑧2 𝑇𝑇𝑧2 + 𝑁𝑧3 𝑇𝑇𝑧3
= 12 × 1.43 + 70 × 0.3 + 18 × 1.11
= 58.14 𝑑𝑎𝑦𝑠
49998 draws with replacement from
each of (𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3)
Takt Time SIPs:
𝑇𝑇𝑧1, 𝑇𝑇𝑧2, 𝑇𝑇𝑧3
Work items:
𝑁𝑧1, 𝑁𝑧2, 𝑁𝑧3
1st draw with
replacement from
each of (𝑇𝑇𝑧1, 𝑇𝑇𝑧2,
𝑇𝑇𝑧3)
50000th draw with
replacement from
each of (𝑇𝑇𝑧1, 𝑇𝑇𝑧2,
𝑇𝑇𝑧3)
Simulated one Project Delivery
Time value
𝑻 = 𝑁𝑧1 𝑇𝑇𝑧1 + 𝑁𝑧2 𝑇𝑇𝑧2 + 𝑁𝑧3 𝑇𝑇𝑧3
= 12 × 1.81 + 70 × 0.54 + 18 × 0.64
= 71.04 𝑑𝑎𝑦𝑠

58.
Mode = 76 days; Median = 77 days; Mean = 78 days; 85th perc = 90 days

59.
By taking an outside view when
forecasting a new project we will
produce more accurate results faster
than using the deterministic inside
view.

60.
References
Here are the distributions for the baseline project
SIPs_MonteCarlo_FVR.xlsx
Here is the planning simulation in Excel
High_Level_Project_Planning.xlsx
What is SIP?

61.
Dimitar Bakardzhiev is the Managing Director of
Taller Technologies Bulgaria and an expert in driving
successful and cost-effective technology
development. As a Lean-Kanban University (LKU)-
Accredited Kanban Trainer (AKT) and avid, expert
Kanban practitioner, Dimitar puts lean principles to
work every day when managing complex software
projects with a special focus on building innovative,
powerful mobile CRM solutions. Dimitar has been one
of the leading proponents and evangelists of Kanban
in his native Bulgaria and has published David
Anderson’s Kanban book as well as books by Eli
Goldratt and W. Edwards Deming in the local
language.
@dimiterbak