Damn... we missed the date again
•Transferir como PPTX, PDF•
4 gostaram•886 visualizações
This document discusses improving project estimation and predicting delivery dates. It begins by noting that detailed estimation does not strongly influence lead times due to typically low flow efficiencies in systems. Lead time histograms are proposed as a better approach to predict completion dates with confidence levels, as they incorporate all system factors without personal bias. Class of service and work item characteristics can identify the most accurate histogram. Considering cost of delay functions alongside histograms helps determine when work should start to balance timely delivery and optimization of resources. The document advocates predicting customer expectations rather than effort or duration and focuses on statistical prediction over detailed planning.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Damn... we missed the date again
Semelhante a Damn... we missed the date again (20)
Mais de Sudipta Lahiri
Mais de Sudipta Lahiri (18)
Último
Último (20)
Damn... we missed the date again
- 1. #LKIN17 @sudiptal Damn… we missed the date, again! SUDIPTA LAHIRI HEAD OF PRODUCTS, DIGITÉ 1
- 2. #LKIN17 @sudiptal Agile/Lean Practitioner and Student Head of Products @ Digité ex-Head of Engineering and Professional Services @ Digité Development of SwiftKanban, SwiftALM products Organize the LimitedWIP Societies in India Sudipta Lahiri (Sudi) 2.5+ decades in the industry 2
- 3. #LKIN17 @sudiptal Let us do a quick poll… All those who have not missed a project deadline, please sit All those have not missed the same project deadlines more than once… please sit All those have not missed the same project deadlines more than twice... please sit All those have not missed the same project deadlines more than thrice… please sit 3
- 4. #LKIN17 @sudiptal Why does it happen? Estimates => Deadlines! Requirement clarity Often, needs many more iterations Turnaround time to sign-offs! Estimation Complexity Experience of person estimating of person developing Hidden work/invisible work Test Cycles/# of test iteration 90-10% Syndrome Why does this happen? Work getting pushed downstream before it was really ready! Other reasons… 4
- 5. #LKIN17 @sudiptal What do we estimate? Customers want to know by when something will get done We estimate how long its going to take… … and then we match the two… … but they don’t match! 5
- 6. #LKIN17 @sudiptal What’s our typical reaction to this? Wish we had planned more! Wish we had spent more time in estimation! Wish we had more detailed requirements OR we had designed in greater detail Wish we had… 6
- 7. #LKIN17 @sudiptal … and that is where the problem starts! 7
- 8. #LKIN17 @sudiptal Why? The time that a work item takes to get completed is only “LITTLE” influenced by its real effort! SURPRISED…? You Bet! For the last 50-60 years, we have spent countless hours in trying to estimate how much effort the work item takes… It’s a classic waste/non-value add (in lean language)! 8
- 9. #LKIN17 @sudiptal Let understand this more… Any value delivery is a combination of value added steps, mixed with non-value steps in between 9
- 11. #LKIN17 @sudiptal F F FF F F F Defining Customer Lead Time UAT E I Dev Ready Delivery Ready G D 5 ∞ Pull Ongoing Development Testing Done3 3 Test Ready 5 ∞ Customer Lead Time Discarded J Pool of Ideas Done ∞ Frequency of batch transfers needs to be calculated and added to kanban system lead time to calculate customer lead time Frequency of batch transfers needs to be calculated and added to kanban system lead time to calculate customer lead time The clock still starts ticking when we accept the customers order, not when it is placed! Slide from LKU Training Deck 11
- 12. #LKIN17 @sudiptal Test Ready F E I G D GY PB DE MN P1 AB Customer Lead Time Waiting Waiting WaitingWorking Ideas Dev Ready 5 Ongoing Development Testing Done 3 35 UAT Release Ready ∞ ∞ Working WaitingWorking Slide from LKU Training Deck Flow Efficiency 12
- 13. #LKIN17 @sudiptal Test Ready Flow Efficiency F E I G D GY PB DE MN P1 AB Customer Lead Time Waiting Waiting WaitingWorking * Zsolt Fabok, Lean Agile Scotland, Sep 2012, Lean Kanban France, Oct 2012 ** Hakan Forss, Lean Kanban France, Oct 2013 Ideas Dev Ready 5 Ongoing Development Testing Done 3 35 UAT Release Ready ∞ ∞ Flow efficiency measures the percentage of total lead time is spent actually adding value (or knowledge) versus waiting Flow efficiency % = Work Time x 100% Lead Time Working WaitingWorking Multitasking means time spent in working columns is often waiting time Slide from LKU Training Deck Flow efficiencies of 1-5% are commonly reported. *, ** > 40% is good! 13
- 14. #LKIN17 @sudiptal Even in a Kanban team like SwiftKanban Our flow efficiency over a 7 year period < 50% 14
- 15. #LKIN17 @sudiptal Waiting Waiting WaitingWorking Working WaitingWorking Test Ready Implications of low Flow Efficiency F E I G D GY PB DE MN P1 AB Ideas Dev Ready 5 Ongoing Development Testing Done 3 35 UAT Release Ready ∞ ∞ Low flow efficiency means that most of lead time is influenced by environmental factors that are unlikely to change soon Customer Lead Time In a low flow efficiency environment, Class of service is much more likely to influence lead time than any other factor As a result, lead time is not very sensitive to the size or complexity of a single work item, or to the specific people involved or their individual capabilities Slide from LKU Training Deck 15
- 16. #LKIN17 @sudiptal This profoundly changes our thinking of the last 5 decades! The estimate that we made with a lot effort and diligence isn’t a real factor! Further… how many assumptions to get to the estimate? Requirement assumptions Design assumptions People skill assumptions No business interruption/dedicated resource assumptions Contingency assumptions Management acceptance of that contingency assumptions… Estimation bias of the person doing the estimation! In short, there are simply TOO MANY SYSTEM variables in a system that are NOT in our control… yet, we estimate with confidence (with buffers) 16
- 17. #LKIN17 @sudiptal The odds were always stacked against you to win this game If you did manage to deliver on time, has someone ever told you: You must have buffered your estimates like crazy!” You got lucky! … and… you did a good job! 17
- 18. #LKIN17 @sudiptal Let’s improve our win(ability)! LET’S PREDICT WITH GREATER CONFIDENCE… WITH A MATHEMATICAL FOUNDATION 18
- 19. #LKIN17 @sudiptal Understanding Distributions 19 Most of us think of distributions as a “Bell Curve” Extremely common in natural domain A legacy of our appraisal discussions… ? In a Normal Bell Curve, also called Gaussian Distribution, Mean(or Average) = Modal CLT applies (from Wikipedia): When independent random variables are added, their properly normalized sum tends toward a normal distribution (a bell curve) even if the original variables themselves are not normally distributed.
- 20. #LKIN17 @sudiptal Lead Time & Weibull Distributions Lead time histograms observed to be Weibull distributions typically with shape parameter 1.0 < k < 2.0 This example is a Weibull distribution with a scale parameter equal to 65 and shape parameter equal to 1.4 Outliers with known special causes at 87 & 105 are omitted from the “best fit” curve Slide from LKU Training Deck 20
- 21. #LKIN17 @sudiptal Lead Time Distribution 0 0.5 1 1.5 2 2.5 3 3.5 1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141 148 Days CRs&Bugs SLA expectation of 44 days with 85% on-time Mean of 31 days SLA expectation of 105 days with 98 % on-time Lead Time Histogram Slide from LKU Training Deck 21
- 22. #LKIN17 @sudiptal Lead Time Distribution 0 0.5 1 1.5 2 2.5 3 3.5 1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141 148 Days CRs&Bugs SLA expectation of 44 days with 85% on-time Mean of 31 days SLA expectation of 105 days with 98 % on-time This is multi-modal data! The work is of two types: Change Requests (new features); and Production Defects This is multi-modal data! The work is of two types: Change Requests (new features); and Production Defects Lead Time Histogram Slide from LKU Training Deck 22
- 23. #LKIN17 @sudiptal 85% at 10 days Mean 5 days 98% at 25 days ProductionDefects ChangeRequests 85% at 60 days Mean 50 days 98% at 150 days Mode Median 45 days Filter by Type/Class to get Single Modal Data Slide from LKU Training Deck 23
- 24. #LKIN17 @sudiptal ChangeRequests 60 days, SLE (customer expectation) Use Lead Time Distribution to Evaluate Service Delivery Effectiveness 22-150 day spread of variation 85% on-time 15% late Due Date Performance (DDP) Predictability Slide from LKU Training Deck 24
- 25. #LKIN17 @sudiptal Let’s recap Given the flow efficiency that most systems experience (in low teens or single digits)… … to somehow think that more planning and more detailed estimation will help us predict better when we can deliver our work item to your customer/end user is “mathematically” flawed! 25
- 26. #LKIN17 @sudiptal So… how do stop making wrong commitments? 1. Don’t answer the wrong question… How long will this get done? When will this get done? How much time this will take? How many resources do you need? 2. The correct question is… When do you need it? What is the Cost of Delay? 26
- 27. #LKIN17 @sudiptal So… how do make accurate predictions? We use mathematical basis – Lead Time Histograms Lead Time Histograms guide us to predict a completion date WITH ASSOCIATED LEVEL OF CONFIDENCE 27
- 28. #LKIN17 @sudiptal Let’s improve our hit rate… Step 1 Class of Services If you are not following Kanban, you might commonly call it as “Priority” David calls CoS as the “Sonic Screwdriver” for a Kanban System Use CoS to better meet Due Dates If a work item gets closer to its Due Date, escalate its CoS to Expedite The getKanban games gives a good example of how this could work! 28
- 29. #LKIN17 @sudiptal Let’s improve our hit rate… Step 2 Classify your work items with greater clarity/refinement , for e.g.: For our SwiftKanban product: T-shirt sizing: quick and effective A level of accuracy enough for most cases! Modules Nature of work: Defect vs Enhancements Unit level defect vs Core Scenario Defect Version No(s): Older versions inherently take more time for us! 29
- 30. #LKIN17 @sudiptal Let’s improve our hit rate… Step 3 Can we reduce the long tail of this distribution….? Reduce the gap between Modal and Mean Yes: By pushing for greater flow (reducing WIP) Keeping an eye on the age of the work items Keep some visual indicators when the work items is close the modal value (Amber) OR when it is closer to the mean value (Red) 30
- 31. #LKIN17 @sudiptal Let’s improve our hit rate… Step 4 Risk Management trims the tail 31 Identify risks, their likelihood & impact (delay that extends lead time). Eliminating risks or reducing their impact trims the tail on the distribution. Trimming the tail moves the mean to the left, increasing delivery rate! Slide from LKU Training Deck 85th percentile mean Risks often cause long lead times
- 32. #LKIN17 @sudiptal Let’s improve our hit rate… Step 5 Let start the work at the right time… not too early and not too late! Not too early… because in the normal environment where Demand > Supply, you, you could work on something else now, knowing that a delay in starting will not risk your Completion Date Not too late… 32
- 34. #LKIN17 @sudiptal CoS mapped to Delay Cost Functions Slide from LKU Training Deck Class of service and its policiesColor Func Expedite – white; critical and immediate cost of delay; can exceed other kanban limit (bumps other work); limit 1 Fixed date – orange; cost of delay goes up significantly after deadline Standard - yellow; increasing urgency, cost of delay is shallow but accelerates before leveling out Intangible – blue; cost of delay may be significant but is not incurred until significantly later, if at all timeimpact time impact time impact time impact 34
- 35. #LKIN17 @sudiptal How easy it is to define the CoD function? You know what’s an Expedite work items! Product Management should be able to help you define the “Fixed Date” or “Intangible” work items You might struggle to define the curve for Standards but relative scaling might be easy 35
- 36. #LKIN17 @sudiptal (Finishing) Cost of Delay Function Lead Time Distribution When to start your work (Starting Cost of Delay) 36
- 37. #LKIN17 @sudiptal Fixed Date CoS From SwiftESP Tool 37
- 38. #LKIN17 @sudiptal Standard CoS (Fixed Rate of value gained) From SwiftESP Tool 38
- 39. #LKIN17 @sudiptal Let us recap now… 39
- 40. #LKIN17 @sudiptal We explained why detailed estimation or planning is unproductive in low Flow Efficiency systems End dates predictions aren’t deterministic They should always be accompanied by a % of confidence A practical, statistical, fast approach using Lead Time histograms Lead Time histograms assimilate all system characteristics and eliminates personal bias Use CoS and work item characteristics to identify histogram that reflects your work item most accurately Use CoD functions, in combination with Lead Time histograms, to understand when you should start your work 40
- 41. #LKIN17 @sudiptal Thank you…. Reach me at: @sudiptal slahiri@digite.com sudiptalahiri.wordpress.com “Absorb what is useful, discard what is useless and add what is specifically your own” Bruce Lee
Notas do Editor
- “Customer Lead Time” is not the same as “System Lead Time”. Customer Lead Time starts when we pull the work (commit to do the work) and includes batch transfers on the delivery end.
- Lead time is also affected by Flow Efficiency (see next slides)
- Flow efficiency tells us how much of the time the work is moving (flowing) rather than waiting in some sort of delay.
- When there is low flow efficiency, class of service can help improve lead time.
- We can recognize Lead Time distribution in a curve. Learning to read and interpret the distribution curve is one way to understand opportunities for improvement. This shape is like saying, “Most of the time we deliver fairly quickly but sometimes it takes us much longer, and then gives us our range of predictability”. Lead time histograms typically exhibit a Weibull distribution curve. Weibull is not a single distribution shape rather it is a family of distribution curves shaped by a parameter referred to as “little k”. A typical value for k is 1.5. The example shown here is from real data for software engineering of telecom-grade equipment and exhibits a k approximately equal to 1.4. Knowing that lead time typically follows a Weibull distribution is a convenient shorthand for risk management and planning decisions. It is not important to obsess over specific shapes or values of k.
- Knowing Lead Time allows for conversations about different delivery expectations and capabilities, based on actual data.
- A lead time histogram provides us insight into service delivery capability. For a single service processing a single type of work, we would expect a single mode in the data regardless of how many classes of service are offered. This example has 2 clusters and hence appears multi-modal. The explanation for this is that 2 distinct types of work were being processed through the same workflow.
- Our management challenge is that the risk is in the tail of the distribution. Median is always less than the mean and lies between the mode and the mean. Median is less sensitive to the tail on distribution and hence less variable in the presence of assignable/special cause variation causing a long tail. However, it is the mean that is used in Little’s Law and therefore we do care about risks that affect the tail in the distribution when using Little’s Law to forecast.
- SLE (Service Delivery Effectiveness)
- [duplicate of previously shown slide 200]