High-level project planning using Monte Carlo simulation
•Transferir como PPTX, PDF•
15 gostaram•3,375 visualizações
High-level planning is planning the initial budget and also the range of the time frame for that project. It does not include detailed project plans – these will be developed later. Management has to plan in order to make sure they have the financial means and the resources for all the projects the company has. We don’t plan in detail what is not absolutely necessary to plan. So, the plan itself is created with the appropriate buffers protecting the decisions that have to be included in the plan from the "common and expected uncertainty" aka common cause variation. The short-term details, like the scheduling, are done based on the immediate needs and capabilities - and we look on these schedules as the execution of the higher level plan. We propose a probabilistic approach for preparing high-level plans. Based on existing historical statistical data recorded about a software development organization/team, predictions can be made that could be used for planning new projects. Here are the SIPs for the baseline project http://modernmanagement.bg/data/SIPs_MonteCarlo_FVR.xlsx Here is the planing simulation in Excel http://modernmanagement.bg/data/High_Level_Project_Planning.xlsx The video http://youtu.be/GE9vrJ741WY on how to use the Excel files
Recomendados
Recomendados
Mais conteúdo relacionado
Destaque
Destaque (16)
Mais de Dimitar Bakardzhiev
Mais de Dimitar Bakardzhiev (8)
Último
Último (20)
High-level project planning using Monte Carlo simulation
- 1. Dimitar Bakardzhiev Managing Partner Taller Technologies Bulgaria @dimiterbak High-Level Project Planning using Monte Carlo simulation
- 2. Management is prediction. Theory of knowledge helps us to understand that management in any form is prediction. W. Edwards Deming
- 3. High-level planning • The initial budget and the range of the time frame • Does not include detailed project plans • The plan is created with the appropriate buffers • Schedules are the execution of the high-level plan
- 4. We challenge the project management paradigm and suggest it is better to model projects as a flow of work items through a system
- 5. A project is a batch of work items each one representing independent customer value that must be delivered on or before due date. The batch can contain even only one work item.
- 6. A Project in a Kanban System Input Queue DEPLOYED! Project Backlog Development Test QA WIP 5 WIP 4 NO WIPWIP 2
- 8. Z-curve
- 9. 1st leg – Setup time • climbing the learning curve • conducting experiments (spikes) to cover the riskiest work items • setting up environments • adapting to client’s culture and procedures • understanding new business domain • mastering new technology
- 10. 2nd leg – Productivity period If the project is scheduled properly the system should be like a clockwork – sustainable pace, no stress, no surprises…
- 11. 3rd leg – Cleaning up • Clean up the battlefield • Fix some outstanding defects • Support the transition of the project deliverable into operation
- 12. Project delivery time T 𝑇 = 𝑇𝑧1 + 𝑇𝑧2 + 𝑇𝑧3 Where: 𝑇𝑧1 – is the duration of the 1st leg of the Z-curve 𝑇𝑧2 – is the duration of the 2nd leg of the Z-curve 𝑇𝑧3 – is the duration of the 3rd leg of the Z-curve
- 13. Takt Time is the time between two successive deliveries Start 5 days 7 days 2 days 2 days 1 day 5 days Finish 0 days 0 days 5 days 7 days Project delivery time = 5 + 7 + 2 + 1 + 5 = 20 days
- 14. How manufacturing measure Takt Time?
- 15. How knowledge workers measure Takt Time?
- 16. Average Takt Time 𝑇𝑇 = 𝑇 𝑁 • T is the time period over which the project will be delivered (delivery time) • N is the number of items to be delivered in period [0,T] • 𝑇𝑇 is the average Takt Time for period [0,T] at the Dev System level
- 17. Calculating Delivery time for a project : 𝑇 = 𝑁𝑇𝑇 • T is the average time period over which the project will be delivered (delivery time) • N is the number of items to be delivered in period [0,T] • 𝑇𝑇 is the average Takt Time for period [0,T] at the Dev System level
- 18. Project delivery time 𝑇 𝑇 = 𝑁𝑧1 𝑇𝑇𝑧1 + 𝑁𝑧2 𝑇𝑇𝑧2 + 𝑁𝑧3 𝑇𝑇𝑧3 Where: 𝑇𝑇𝑧1 is the average Takt Time for the 1st leg of the Z-curve 𝑇𝑇𝑧2 is the average Takt Time for the 2nd leg of the Z-curve 𝑇𝑇𝑧3 is the average Takt Time for the 3rd leg of the Z-curve 𝑁𝑧1 is the number of items delivered during the 1st leg of the Z-curve 𝑁𝑧2 is the number of items delivered during the 2nd leg of the Z-curve 𝑁𝑧3 is the number of items delivered during the 3rd leg of the Z-curve
- 19. We should NOT use the Average Takt Time as a single number but a distribution of the average Takt Time instead!
- 20. Stochastic Information Packet (SIP) • Comprised of a list of trials of some uncertain parameter or metric generated from historical data using Monte Carlo simulation (resampling) • Represents an uncertainty as an array of possible outcomes (distribution) • It is unique per context (business domain, team, delivery process used etc.)
- 21. How to predict project delivery time? 1. Collect Takt Time data for each of the Z-curve legs of a Baseline Project 2. Using Monte Carlo simulation generate Average Takt Time (AvgTT) distribution (SIP) for each of the Z-curve legs and store them for future use Then when a new project needs to be planned: 1. Using Monte Carlo simulation generate Delivery time (T) probability distribution based on the baseline AvgTT SIPs 2. For predicted delivery time use the 85 percentile of the Delivery time (T) probability distribution
- 22. 1. COLLECT TAKT TIME DATA FROM A PROJECT TO BE USED AS A BASELINE How to predict project delivery time?
- 23. Data for the baseline project • Fortune 500 Staffing company • Technology – Java; Spring; Oracle; • Delivery time of 199 days • Project scope of 238 user stories • 42 stories delivered in the 1st leg of Z-curve • 161 stories delivered in the 2nd leg of Z-curve • 28 stories delivered in the 3rd leg of Z-curve
- 24. 2. GENERATE SIP FOR AVERAGE TAKT TIME (AVGTT) USING MONTE CARLO SIMULATION How to predict project delivery time?
- 25. Takt Time SIPs for each of the three legs of Z-curve
- 26. 3. GENERATE DELIVERY TIME (T) PROBABILITY DISTRIBUTION BASED ON THE AVG TAKT TIME DISTRIBUTIONS (SIP) USING MONTE CARLO SIMULATION How to predict project delivery time?
- 27. New project to be delivered • THE SAME Fortune 500 Staffing company • THE SAME development organization • THE SAME technology – Java; Spring; Oracle; • Delivery time TO BE PREDICTED
- 28. Project scope The team would break down the requirements, account for Cost of Delay, add work items for Dark matter and Failure load and decide that: • 12 stories TO BE delivered in the 1st leg of Z-curve • 70 stories TO BE delivered in the 2nd leg of Z-curve • 18 stories TO BE delivered in the 3rd leg of Z-curve
- 29. Monte Carlo summation of… …will give us the time needed to deliver the project! 12 work items 70 work items 18 work items
- 30. Mode = 76 days; Median = 77 days; Mean = 78 days; 85th perc = 90 days
- 31. References Here are the SIPs for the baseline project SIPs_MonteCarlo_FVR.xlsx Here is the planing simulation in Excel High_Level_Project_Planning.xlsx Content is inspired and based on ideas from David J. Anderson, Troy Magennis, Alexei Zheglov What is SIP?
Notas do Editor
- Decision making using Takt Time to avoid bottom-up estimating by story, use case etc.
- High-level planning is planning the initial budget and also the range of the time frame for that project. It does not include detailed project plans – these will be developed later. Management has to plan in order to make sure they have the financial means and the resources for all the projects the company has. We don’t plan in detail what is not absolutely necessary to plan. So, the plan itself is created with the appropriate buffers protecting the decisions that have to be included in the plan from the "common and expected uncertainty" aka common cause variation. The short-term details, like the scheduling, are done based on the immediate needs and capabilities - and we look on these schedules as the execution of the higher level plan. We propose a probabilistic approach for preparing high-level plans. Based on existing historical statistical data recorded about a software development organization/team, predictions can be made that could be used for planning new projects.
- Clients come to us with an idea for a product and they always ask the questions - how long will it take and how much will it cost us to deliver? They need a delivery date and a budget estimate. Does the customer actually ask for a "project“? Or is a project just a delivery vehicle or a container for managing risk? We challenge the project management paradigm and suggest it is better to model projects as a flow of work items through a system. A project is just a batch taken out from the flow of work. Say the flow processes thousands of work items. We can name a batch of some 100 work items “a project” but we have statistical data for the flow itself and we use that data!
- Hence the definition - A project is a batch of work items each one representing independent customer value that must be delivered on or before due date. The batch can contain even only one work item. The size of the work item does not matter. There are only two "sizes" - "small enough" and "too big". "Too big" should be naturally split by a mature development organization (team) and not allowed to enter the backlog.
- Here is an animation how a batch of work items or a project flows through a Kanban board. Initially all work items are in the Backlog, and then they flow through the board and end up in Completed column.
- This is a Cumulative Flow Diagram from a real project. It visualizes the rate at which the work items were delivered. On the X axis we have the project time in days. On the Y axis we have the number of work items delivered each day. We will be using the data behind this diagram in order to predict delivery date for future projects. It turns out that the delivery rate (TH) follows a Z-curve pattern as visualized by the red line.
- Z-curve can be divided in three parts or we can say it has three legs. There is empirical evidence here that 20% of the time the delivery rate will be slow. Then for 60% of the time we’ll going to go fast or “the hyper productivity” period. And the 20% till the end we’ll go slowly. Of course numbers may vary depending on the context but the basic principle about the three sections is correct. Only the second Z-curve leg is representative for the Dev System capability. It shows the common cause variation specific to each Dev System process. First and third Z-curve legs are project specific and are affected by special cause variation.
- The 1st leg of the Z-curve is the time when the developers climb the learning curve and setup their minds for the new project. But this leg of the Z-curve could also be used for: conducting experiments (spikes) to cover the riskiest work items setting up environments adapting to client’s culture understanding new business domain mastering new technology accepting client’s procedures All above are examples of special causes of variation specific to a project…
- The second leg of the Z-curve is the productivity period. If the project is scheduled properly the system should be like clockwork – sustainable pace, no stress, no surprises…
- The third leg of the Z-curve is when the team will clean up the battlefield, fix some outstanding defects and support the transition of the project deliverable into operation.
- Project delivery time can be represented as a sum of the duration of each one of the three legs of the Z-curve. Or in other words it equals the duration of the 1st leg plus the duration of the 2nd leg plus the duration of the 3rd leg of the Z-curve.
- http://www.lean.org/Common/LexiconTerm.aspx?termid=337 http://en.wikipedia.org/wiki/Takt_time So what is Takt Time and how we can measure it? Takt Time is the time between two successive deliveries. Here is a simple diagram presenting the delivery rate for a fictitious project. Each delivered work item is represented by a yellow box. On the left we have the start date for the project and on the right we have the end date. We can see that five day after the project started the first work item was delivered. Its Takt Time is 5 days. Seven days after that two new work items were delivered. Now what is their Takt Time? The first work item has a Takt Time of seven days, but the second one has a Takt Time of zero days. That is because the time between the two work items is zero days. It is not zero minutes but since we measure Takt Time in days it is zero days. Two days after that three new work items were delivered. According to the definition of Takt Time one of them has Takt Time of two days but the other two work items both have Takt Tame of zero days. And as we see how it went – eventually all of the work items were delivered. Important thing to note here is that the total sum of all Takt Time values equals the delivery time of the project.
- In what units of time we measure Takt Time? In manufacturing they measure Takt Time in hours, minutes even in seconds for the mass production. In knowledge work we are measuring Takt Time in days.
- In knowledge work we measure Lead time and Takt Time in days.
- Average Takt Time is calculated by dividing over which the project is or will be delivered by the number of work items delivered. For instance if we have 20 days delivery time and we have 20 work items delivered the average Takt Time equals 1 day. That means that on average the time between two successive deliveries is 1 day. 𝑇𝑇 = 𝑇 𝑁 T is the time period over which the project will be delivered (lead time) N is the number of items to be delivered or the total arrivals in [0,T] 𝑇𝑇 is the average Takt Time at the development organization level
- If we have the average Takt Time for the dev system and we have a number of work items to be delivered we can calculate how much time ON AVERAGE will take the system to deliver all 20 work items. The formula is average Takt Time times the number of work items to be delivered.
- Here comes the important point - we should NOT use the Average Takt Time as a single number but a distribution of the average Takt Time instead!
- The Stochastic Information Packet or SIP represents an uncertainty as an array of possible outcomes. The concept was formalized in 2006 in an article in OR/MS Today http://probabilitymanagement.org/library/Probability_Management_Part1s.pdf What is this thing called Stochastic Information Packet? Every SIP is unique. We cannot compare one dev system’s historical delivery data with another team. If we do have historical data about average Takt Time it will be invalidated if any of the following happens: • team structure is changed • development process is changed • technology being used is changed • development process being used is changed - say if introduce pair programming If anything of the above is changed all historical data will be invalidated including the SIP itself.
- How can we predict delivery date for a new project? By following the algorithm presented here. First we will have to collect Takt Time data for each of the Z-curve legs for the Dev System. Then using Monte Carlo simulation we will generate Average Takt Time distribution or the so called Stochastic Information Packet (SIP) for each of the Z-curve legs. This SIP should be stored for future usage. The third step starts when we have a new project to plan. Then we will generate Delivery time (T) probability distribution based on the AvgTT SIP using Monte Carlo simulation from the second step. Then we will use 85th percentile as delivery time. The 85th percentile means that for every six out of seven projects we should be able to meet the predicted delivery time.
- We will take a real project as an example how to use Monte Carlo simulation for predicting project delivery time. This is the Excel file we will be using to run the Monte Carlo Simulation. The link to the file can be found in the description section for this video.
- Dark matter The number of work items will grow through natural expansion as we investigate them. It is possible some features will be broken into 2 or more once you get started on them. That is why we add work items to compensate for the unknown work or dark matter. My experience is that I would buffer at least 20% for this and with novice teams on a new product (especially if it was a highly innovative new product where we had no prior knowledge or experience) then I might go as high as 100%. Failure load We add work items to compensate for the expected failure load in terms of defects, rework, and technical debt. Failure load tracks how many work items the Kanban system processes due to poor quality. That includes production defects (bugs) in software and new features requested by the users because of a poor usability or a failure to anticipate properly user needs. Defects represent opportunity cost and affect the lead time and throughput of the Kanban system. The count of defects is a good indicator if the organization is improving or not. Add work items to compensate for the unknown work or dark matter. The number of work items will grow through natural expansion as we investigate them. It is possible some features will be broken into 2 or more once you get started on them. My experience is that I would buffer at least 20% for this and with novice teams on a new product (especially if it was a highly innovative new product where we had no prior knowledge or experience) then I might go as high as 100%. Add work items to compensate for the expected failure load in terms of defects, rework, technical debt.
- http://modernmanagement.bg/data/SIPs_MonteCarlo_FVR.xlsx http://modernmanagement.bg/data/High_Level_Project_Planning.xlsx