SlideShare uma empresa Scribd logo
1 de 24
Baixar para ler offline
Psychological Impacts on Judgment
        in Cost Estimation

              Jordan Garner
            UC Davis (JPL Summer Hire)
           Art B. Chmielewski
               Jet Propulsion Laboratory
           California Institute of Technology

               September 12, 2011




                                                1
Special Thanks to
• Dr. David Ullman of Robust Decisions and Oregon State
  University for his assistance with the web experiment and
  continued support of this novel research.

• Prof. Don Forsyth of the University of Richmond for his expert
  consultation on socio-psychological effects in decision making.




                                                                2
258%       275%                           1100%




100%

                                                  1500%
               Bad Cost Estimates
                    Happen

                                  1400%
                                                  600%

                                          2200%




                                                    3
       220%               1400%
Overruns Start with
         Bad Initial Cost Estimates
• Bad cost estimates are in every sector of business
  world: construction projects, movie
  business, transportation projects, military
  programs, aerospace, etc.
• Bad cost estimates know no borders, race, sex or
  century.




                                                       4
Causes of Overruns
• Overruns start with flawed initial cost estimates and
  inadequate reserves.
• However, the post mortem analyses give less blame to the
  estimating than to failures in execution such as:
   –   Changes in scope and requirements
   –   Inadequate communication
   –   Government and contractor intervention
   –   Unforeseen technical issues
   –   New technology
   –   Acts of god
• Specific technical reasons for overruns seem to be more palatable
  than poor cost estimates.



                                                                      5
Are Estimates Getting Better?
“For the past 70 years, for which data on cost estimation is
observable, no significant improvements in
forecasting, estimating or prediction a project’s cost have ever
been made. This is despite the increase in awareness of past
estimation inaccuracy, new strategies of estimation, the hiring of
more experts to help the estimation process, inventions solving
past technical and communication issues.”

– Prof. Bent Flyvbjerg, at Oxford University's Saïd Business School




                                                                  6
Unaccounted Psychological Effects?
• Thesis: Could humans be prone to psychological
  factors that make them truly and honestly believe in
  poor estimates?
• We conducted a simple experiment to test and
  quantitatively measure the power of psychological
  fallacies on people’s ability to make estimates.




                                                         7
Overheard During Cost Estimating:
•   “I have a bogey of $400k. Please give me your own estimate.”
•   “We will hold 30% reserve for you.”
•   “I sent you a WBS cost table. Can you fill it in?”
•   “We need your best estimate by Friday.”
•   You have an allocation of $1.3M, can you give me an
    estimate?

Our simple experiment proved that the above common costing
phrases guarantee overruns!



                                                               8
Dishwashing Experiment

• Participants in the on-line experiment were asked in
  different ways to estimate the time needed to
  perform a simple task – washing the dishes shown on
  the next chart.




                                                     9
10
Psychological Effects Tested
• 5 psychological effects were tested :
   1.   Anchoring
   2.   Q&A Mismatch
   3.   Decomposition
   4.   Reserve Comfort
   5.   Planning Fallacy
• Every respondent to the survey was randomly asked one
  of several questions testing different psychological
  heuristics or fallacies.
• 507 volunteers participated: 142 JPLers, 305 college
  students and 60 other adults. ~2300 data points were
  collected.


                                                          11
All answers were graphed and
     analyzed to establish conclusions
90

80

70

60

50

40                                       upper Standard Deviation
                                         estimate
30
                                         lower Standard Deviation
20

10

0




                Psychological category
                                                          12
Effect #1: Anchoring

The objective was to test how easily influenced
people may be by a wrong answer – “the anchor.”
The anchor set asked:
Estimate how many minutes it will take you to clean the
kitchen. One respondent estimated that it will take
about 10 minutes to finish cleaning up. He may be
wrong of course.




                                                      13
1. Anchoring Results
• The nominal value was 30 min, the anchored case 25 min.

• The “best case scenario” estimate (described later) was 27
  min which was 2 min LONGER than the anchored result.

• The result points out that it is very easy to dramatically skew
  the estimates by asking anchored questions, such as: “We
  would like you to come in around $6M”, “I have a bogey of
  $400k for you”, “the last robot arm we built cost $7M”…



                                                                    14
Effect #2: Q&A Mismatch
The purpose was to test if there is a mismatch between the type
of estimate expected and provided.

Different participants were asked:
• Estimate how many minutes it will take you to clean the whole
   kitchen.
• There is a 50% chance that you will finish this task within __ min
• There is a 75% chance that you will finish this task within __ min
• There is a 99% chance that you will finish this task within __ min



                                                                  15
2. Q&A Mismatch Results
• The 50% confidence estimate was 31 min. The nominal
  estimate was 30 min. People unconsciously interpret the
  nominal as the 50% case, meaning that you will exceed your
  estimate in half the cases!

• However, when a manager asks for an estimate he/she
  expects a much more reliable result, possibly in the 80%-90%
  confidence range. This points out that there is mismatch
  between the expectation and the answer.




                                                                 16
Effect #3: Decomposition
The objective was to test if decomposing the project into
smaller pieces and deeper levels of a WBS improved accuracy
of the estimate.
Estimate decomposition was simulated by asking:
1. How many minutes will it take to clean all the plates and the
     sets of silver?
2. How long will it take to clean the sets of coffee cups and
     saucers?
3. How long will it take to clean the bowls?
4. Etc.


                                                               17
3. Decomposition Results
• Decomposition average was 31 minutes, just one
  minute longer than the nominal average (30 min).
  The attempt at becoming more accurate by cutting
  up the project was not accomplished.
• Decomposition, at least in this case, was more time
  consuming than helpful.
• Deep decompositions provide more detail but
  compound psychological effects.



                                                        18
Effect #4: Reserve Comfort
This question tested the realism of “a comfortable” reserve.

Respondents were asked:
1. I am 90% sure that the time it will actually take to clean the
    kitchen is within plus or minus __ minutes from my estimate.




                                                               19
4. Reserve Comfort Results
• The reserve for 90% confidence was 8 min or 28%. The 25-30% seems to be
  the magical intuitional comfort level that is used by many industries.

• When a manager asks for a reserve he/she means “I want to be very sure
  that I will not exceed this reserve. I want my reserve to cover almost the
  worst case.”

• However, that is not how it is interpreted by the employee.
   – The worst case estimate was 51 min and required 70% reserve.
   – The 99% confidence case averaged 45 min. and needed 50% reserve.
     Both of these cases are significantly higher than the popular 30%.



                                                                               20
Large Projects Reserve Comparison
                 100%

                 90%
                                                                   The realistic amount of
Needed Reserve




                 80%                                               budget reserve required
                 70%                                               for 18 large projects
                 60%                                               studied is 52%.
                 50%

                 40%

                 30%

                 20%


                 10%
                                                                    Recent aerospace projects
                        10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

                                   Planned Reserve                                         21
Effect #5: Planning Fallacy
The planning fallacy, as defined by Daniel Kahneman and Amos
Tversk is a tendency to be overly optimistic in planning.
To asses the extent of optimism we asked:
1. In the best case scenario (if everything went as well as
    possible), how many minutes would it take you to clean the
    whole kitchen?
2. In the worst case scenario (if everything went as poorly as
    possible), how many minutes it would take you to clean the
    whole kitchen?



                                                             22
5. Planning Fallacy Results
• The following results were obtained:
   –   51 min worst case
   –   45 min 99% confidence
   –   30 min nominal
   –   27 min best case




• These results show how skewed people are toward optimism. The
  nominal estimate was 10% longer than the best case but 70%
  shorter than the worst case.
• People are so optimistic that it was easy to anchor them down but
  anchoring up failed.

                                                                      23
Conclusions
• To improve the quality of cost estimates it is recommended to
  diminish the effects of psychological impact on judgment:

    Train the managers not to anchor.
    Establish proper Estimation Language which makes the
     questions compatible with common interpretation.
    Deep decompositions do not improve accuracy.
    Calculate the reserve based on risk.
    Account for optimism by including in the baseline
     likely, historical and common risks.




                                                                  24

Mais conteúdo relacionado

Destaque

Graybill.michael
Graybill.michaelGraybill.michael
Graybill.michaelNASAPMC
 
Carmichael.kevin
Carmichael.kevinCarmichael.kevin
Carmichael.kevinNASAPMC
 
Chen.tim
Chen.timChen.tim
Chen.timNASAPMC
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski johnNASAPMC
 
Inter mitchell
Inter mitchellInter mitchell
Inter mitchellNASAPMC
 
Ed mangopanelpm challengefinal
Ed mangopanelpm challengefinalEd mangopanelpm challengefinal
Ed mangopanelpm challengefinalNASAPMC
 
Vince.bilardo
Vince.bilardoVince.bilardo
Vince.bilardoNASAPMC
 
Newman lengyel dartpm-chal_case
Newman lengyel dartpm-chal_caseNewman lengyel dartpm-chal_case
Newman lengyel dartpm-chal_caseNASAPMC
 
Claunch.cathy
Claunch.cathyClaunch.cathy
Claunch.cathyNASAPMC
 
Newman.steve
Newman.steveNewman.steve
Newman.steveNASAPMC
 
Sharyl butler
Sharyl butlerSharyl butler
Sharyl butlerNASAPMC
 
Bingham.alph
Bingham.alphBingham.alph
Bingham.alphNASAPMC
 
Freaner.claude
Freaner.claudeFreaner.claude
Freaner.claudeNASAPMC
 
Kerry.mushkin
Kerry.mushkinKerry.mushkin
Kerry.mushkinNASAPMC
 
Oberhettinger
OberhettingerOberhettinger
OberhettingerNASAPMC
 
Greene.stacie
Greene.stacieGreene.stacie
Greene.stacieNASAPMC
 
Lane.william
Lane.williamLane.william
Lane.williamNASAPMC
 
Walter.bowman
Walter.bowmanWalter.bowman
Walter.bowmanNASAPMC
 
Hatfieldskip
HatfieldskipHatfieldskip
HatfieldskipNASAPMC
 
Sand.steven
Sand.stevenSand.steven
Sand.stevenNASAPMC
 

Destaque (20)

Graybill.michael
Graybill.michaelGraybill.michael
Graybill.michael
 
Carmichael.kevin
Carmichael.kevinCarmichael.kevin
Carmichael.kevin
 
Chen.tim
Chen.timChen.tim
Chen.tim
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski john
 
Inter mitchell
Inter mitchellInter mitchell
Inter mitchell
 
Ed mangopanelpm challengefinal
Ed mangopanelpm challengefinalEd mangopanelpm challengefinal
Ed mangopanelpm challengefinal
 
Vince.bilardo
Vince.bilardoVince.bilardo
Vince.bilardo
 
Newman lengyel dartpm-chal_case
Newman lengyel dartpm-chal_caseNewman lengyel dartpm-chal_case
Newman lengyel dartpm-chal_case
 
Claunch.cathy
Claunch.cathyClaunch.cathy
Claunch.cathy
 
Newman.steve
Newman.steveNewman.steve
Newman.steve
 
Sharyl butler
Sharyl butlerSharyl butler
Sharyl butler
 
Bingham.alph
Bingham.alphBingham.alph
Bingham.alph
 
Freaner.claude
Freaner.claudeFreaner.claude
Freaner.claude
 
Kerry.mushkin
Kerry.mushkinKerry.mushkin
Kerry.mushkin
 
Oberhettinger
OberhettingerOberhettinger
Oberhettinger
 
Greene.stacie
Greene.stacieGreene.stacie
Greene.stacie
 
Lane.william
Lane.williamLane.william
Lane.william
 
Walter.bowman
Walter.bowmanWalter.bowman
Walter.bowman
 
Hatfieldskip
HatfieldskipHatfieldskip
Hatfieldskip
 
Sand.steven
Sand.stevenSand.steven
Sand.steven
 

Semelhante a Arthur.chmielewski

Software estimation is crap
Software estimation is crapSoftware estimation is crap
Software estimation is crapIan Garrison
 
Estimates in Project Management
Estimates in Project ManagementEstimates in Project Management
Estimates in Project ManagementIntaver Insititute
 
Estimations in Project Management
Estimations in Project ManagementEstimations in Project Management
Estimations in Project ManagementIntaver Insititute
 
How Quick Are We to Judge? A Case Study of Trust and Web Site Design
How Quick Are We to Judge? A Case Study of Trust and Web Site DesignHow Quick Are We to Judge? A Case Study of Trust and Web Site Design
How Quick Are We to Judge? A Case Study of Trust and Web Site DesignNew York Technology Council
 
How to design powerful experiments - Ying Zhang
How to design powerful experiments - Ying ZhangHow to design powerful experiments - Ying Zhang
How to design powerful experiments - Ying ZhangProduct Anonymous
 
anchoring-heuristic Decision Making
anchoring-heuristic Decision Makinganchoring-heuristic Decision Making
anchoring-heuristic Decision MakingÖzkan Özer
 
Mmig talk jan 245 2011
Mmig talk jan 245 2011Mmig talk jan 245 2011
Mmig talk jan 245 2011Brock Dubbels
 
PMI Global Congress North America 2013 - Improving Focus and Predictability o...
PMI Global Congress North America 2013 - Improving Focus and Predictability o...PMI Global Congress North America 2013 - Improving Focus and Predictability o...
PMI Global Congress North America 2013 - Improving Focus and Predictability o...Joe Cooper
 
An Implementation of Preregistration
An Implementation of PreregistrationAn Implementation of Preregistration
An Implementation of Preregistrationmboehme
 
Introduction to Project Decision Analysis
Introduction to Project Decision AnalysisIntroduction to Project Decision Analysis
Introduction to Project Decision AnalysisIntaver Insititute
 
Estimating the Business Value of UX Research
Estimating the Business Value of UX ResearchEstimating the Business Value of UX Research
Estimating the Business Value of UX ResearchEmily Danielson
 
Understanding Uncertainty.pdf
Understanding Uncertainty.pdfUnderstanding Uncertainty.pdf
Understanding Uncertainty.pdfMohanadHussien2
 
Week12 slides
Week12 slidesWeek12 slides
Week12 slideshenry KKK
 
Monte Carlo on a Blackpool Budget - Paul Brierly - HOlink2016
Monte Carlo on a Blackpool Budget - Paul Brierly - HOlink2016Monte Carlo on a Blackpool Budget - Paul Brierly - HOlink2016
Monte Carlo on a Blackpool Budget - Paul Brierly - HOlink2016HOlink
 
Project Management Control Systems
Project Management Control SystemsProject Management Control Systems
Project Management Control Systemsskillern
 
Crowdsourcing using MTurk for HCI research
Crowdsourcing using MTurk for HCI researchCrowdsourcing using MTurk for HCI research
Crowdsourcing using MTurk for HCI researchEd Chi
 
Leveraging the Latest in Brain Science to Deliver the Next Generation of E-Le...
Leveraging the Latest in Brain Science to Deliver the Next Generation of E-Le...Leveraging the Latest in Brain Science to Deliver the Next Generation of E-Le...
Leveraging the Latest in Brain Science to Deliver the Next Generation of E-Le...Human Capital Media
 

Semelhante a Arthur.chmielewski (20)

Art c
Art cArt c
Art c
 
Art c
Art cArt c
Art c
 
Software estimation is crap
Software estimation is crapSoftware estimation is crap
Software estimation is crap
 
Estimates in Project Management
Estimates in Project ManagementEstimates in Project Management
Estimates in Project Management
 
Estimations in Project Management
Estimations in Project ManagementEstimations in Project Management
Estimations in Project Management
 
How Quick Are We to Judge? A Case Study of Trust and Web Site Design
How Quick Are We to Judge? A Case Study of Trust and Web Site DesignHow Quick Are We to Judge? A Case Study of Trust and Web Site Design
How Quick Are We to Judge? A Case Study of Trust and Web Site Design
 
How to design powerful experiments - Ying Zhang
How to design powerful experiments - Ying ZhangHow to design powerful experiments - Ying Zhang
How to design powerful experiments - Ying Zhang
 
anchoring-heuristic Decision Making
anchoring-heuristic Decision Makinganchoring-heuristic Decision Making
anchoring-heuristic Decision Making
 
Mmig talk jan 245 2011
Mmig talk jan 245 2011Mmig talk jan 245 2011
Mmig talk jan 245 2011
 
PMI Global Congress North America 2013 - Improving Focus and Predictability o...
PMI Global Congress North America 2013 - Improving Focus and Predictability o...PMI Global Congress North America 2013 - Improving Focus and Predictability o...
PMI Global Congress North America 2013 - Improving Focus and Predictability o...
 
An Implementation of Preregistration
An Implementation of PreregistrationAn Implementation of Preregistration
An Implementation of Preregistration
 
Introduction to Project Decision Analysis
Introduction to Project Decision AnalysisIntroduction to Project Decision Analysis
Introduction to Project Decision Analysis
 
Estimating the Business Value of UX Research
Estimating the Business Value of UX ResearchEstimating the Business Value of UX Research
Estimating the Business Value of UX Research
 
Understanding Uncertainty.pdf
Understanding Uncertainty.pdfUnderstanding Uncertainty.pdf
Understanding Uncertainty.pdf
 
Week12 slides
Week12 slidesWeek12 slides
Week12 slides
 
Debra Lerner's Presentation at the WWCMA April Meeting
Debra Lerner's Presentation at the WWCMA April MeetingDebra Lerner's Presentation at the WWCMA April Meeting
Debra Lerner's Presentation at the WWCMA April Meeting
 
Monte Carlo on a Blackpool Budget - Paul Brierly - HOlink2016
Monte Carlo on a Blackpool Budget - Paul Brierly - HOlink2016Monte Carlo on a Blackpool Budget - Paul Brierly - HOlink2016
Monte Carlo on a Blackpool Budget - Paul Brierly - HOlink2016
 
Project Management Control Systems
Project Management Control SystemsProject Management Control Systems
Project Management Control Systems
 
Crowdsourcing using MTurk for HCI research
Crowdsourcing using MTurk for HCI researchCrowdsourcing using MTurk for HCI research
Crowdsourcing using MTurk for HCI research
 
Leveraging the Latest in Brain Science to Deliver the Next Generation of E-Le...
Leveraging the Latest in Brain Science to Deliver the Next Generation of E-Le...Leveraging the Latest in Brain Science to Deliver the Next Generation of E-Le...
Leveraging the Latest in Brain Science to Deliver the Next Generation of E-Le...
 

Mais de NASAPMC

Bejmuk bo
Bejmuk boBejmuk bo
Bejmuk boNASAPMC
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski johnNASAPMC
 
Yew manson
Yew mansonYew manson
Yew mansonNASAPMC
 
Wood frank
Wood frankWood frank
Wood frankNASAPMC
 
Wood frank
Wood frankWood frank
Wood frankNASAPMC
 
Wessen randi (cd)
Wessen randi (cd)Wessen randi (cd)
Wessen randi (cd)NASAPMC
 
Vellinga joe
Vellinga joeVellinga joe
Vellinga joeNASAPMC
 
Trahan stuart
Trahan stuartTrahan stuart
Trahan stuartNASAPMC
 
Stock gahm
Stock gahmStock gahm
Stock gahmNASAPMC
 
Snow lee
Snow leeSnow lee
Snow leeNASAPMC
 
Smalley sandra
Smalley sandraSmalley sandra
Smalley sandraNASAPMC
 
Seftas krage
Seftas krageSeftas krage
Seftas krageNASAPMC
 
Sampietro marco
Sampietro marcoSampietro marco
Sampietro marcoNASAPMC
 
Rudolphi mike
Rudolphi mikeRudolphi mike
Rudolphi mikeNASAPMC
 
Roberts karlene
Roberts karleneRoberts karlene
Roberts karleneNASAPMC
 
Rackley mike
Rackley mikeRackley mike
Rackley mikeNASAPMC
 
Paradis william
Paradis williamParadis william
Paradis williamNASAPMC
 
Osterkamp jeff
Osterkamp jeffOsterkamp jeff
Osterkamp jeffNASAPMC
 
O'keefe william
O'keefe williamO'keefe william
O'keefe williamNASAPMC
 
Muller ralf
Muller ralfMuller ralf
Muller ralfNASAPMC
 

Mais de NASAPMC (20)

Bejmuk bo
Bejmuk boBejmuk bo
Bejmuk bo
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski john
 
Yew manson
Yew mansonYew manson
Yew manson
 
Wood frank
Wood frankWood frank
Wood frank
 
Wood frank
Wood frankWood frank
Wood frank
 
Wessen randi (cd)
Wessen randi (cd)Wessen randi (cd)
Wessen randi (cd)
 
Vellinga joe
Vellinga joeVellinga joe
Vellinga joe
 
Trahan stuart
Trahan stuartTrahan stuart
Trahan stuart
 
Stock gahm
Stock gahmStock gahm
Stock gahm
 
Snow lee
Snow leeSnow lee
Snow lee
 
Smalley sandra
Smalley sandraSmalley sandra
Smalley sandra
 
Seftas krage
Seftas krageSeftas krage
Seftas krage
 
Sampietro marco
Sampietro marcoSampietro marco
Sampietro marco
 
Rudolphi mike
Rudolphi mikeRudolphi mike
Rudolphi mike
 
Roberts karlene
Roberts karleneRoberts karlene
Roberts karlene
 
Rackley mike
Rackley mikeRackley mike
Rackley mike
 
Paradis william
Paradis williamParadis william
Paradis william
 
Osterkamp jeff
Osterkamp jeffOsterkamp jeff
Osterkamp jeff
 
O'keefe william
O'keefe williamO'keefe william
O'keefe william
 
Muller ralf
Muller ralfMuller ralf
Muller ralf
 

Último

2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdfThe Good Food Institute
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applicationsnooralam814309
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNeo4j
 
Extra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfExtra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfInfopole1
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingFrancesco Corti
 
Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxSatishbabu Gunukula
 
The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)codyslingerland1
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3DianaGray10
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
 
UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2DianaGray10
 
The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)IES VE
 
UiPath Studio Web workshop series - Day 1
UiPath Studio Web workshop series  - Day 1UiPath Studio Web workshop series  - Day 1
UiPath Studio Web workshop series - Day 1DianaGray10
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0DanBrown980551
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FESTBillieHyde
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTopCSSGallery
 
AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024Brian Pichman
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and businessFrancesco Corti
 
March Patch Tuesday
March Patch TuesdayMarch Patch Tuesday
March Patch TuesdayIvanti
 
Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl
 

Último (20)

2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applications
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4j
 
Extra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfExtra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdf
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is going
 
Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptx
 
The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2
 
SheDev 2024
SheDev 2024SheDev 2024
SheDev 2024
 
The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)
 
UiPath Studio Web workshop series - Day 1
UiPath Studio Web workshop series  - Day 1UiPath Studio Web workshop series  - Day 1
UiPath Studio Web workshop series - Day 1
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FEST
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development Companies
 
AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and business
 
March Patch Tuesday
March Patch TuesdayMarch Patch Tuesday
March Patch Tuesday
 
Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile Brochure
 

Arthur.chmielewski

  • 1. Psychological Impacts on Judgment in Cost Estimation Jordan Garner UC Davis (JPL Summer Hire) Art B. Chmielewski Jet Propulsion Laboratory California Institute of Technology September 12, 2011 1
  • 2. Special Thanks to • Dr. David Ullman of Robust Decisions and Oregon State University for his assistance with the web experiment and continued support of this novel research. • Prof. Don Forsyth of the University of Richmond for his expert consultation on socio-psychological effects in decision making. 2
  • 3. 258% 275% 1100% 100% 1500% Bad Cost Estimates Happen 1400% 600% 2200% 3 220% 1400%
  • 4. Overruns Start with Bad Initial Cost Estimates • Bad cost estimates are in every sector of business world: construction projects, movie business, transportation projects, military programs, aerospace, etc. • Bad cost estimates know no borders, race, sex or century. 4
  • 5. Causes of Overruns • Overruns start with flawed initial cost estimates and inadequate reserves. • However, the post mortem analyses give less blame to the estimating than to failures in execution such as: – Changes in scope and requirements – Inadequate communication – Government and contractor intervention – Unforeseen technical issues – New technology – Acts of god • Specific technical reasons for overruns seem to be more palatable than poor cost estimates. 5
  • 6. Are Estimates Getting Better? “For the past 70 years, for which data on cost estimation is observable, no significant improvements in forecasting, estimating or prediction a project’s cost have ever been made. This is despite the increase in awareness of past estimation inaccuracy, new strategies of estimation, the hiring of more experts to help the estimation process, inventions solving past technical and communication issues.” – Prof. Bent Flyvbjerg, at Oxford University's Saïd Business School 6
  • 7. Unaccounted Psychological Effects? • Thesis: Could humans be prone to psychological factors that make them truly and honestly believe in poor estimates? • We conducted a simple experiment to test and quantitatively measure the power of psychological fallacies on people’s ability to make estimates. 7
  • 8. Overheard During Cost Estimating: • “I have a bogey of $400k. Please give me your own estimate.” • “We will hold 30% reserve for you.” • “I sent you a WBS cost table. Can you fill it in?” • “We need your best estimate by Friday.” • You have an allocation of $1.3M, can you give me an estimate? Our simple experiment proved that the above common costing phrases guarantee overruns! 8
  • 9. Dishwashing Experiment • Participants in the on-line experiment were asked in different ways to estimate the time needed to perform a simple task – washing the dishes shown on the next chart. 9
  • 10. 10
  • 11. Psychological Effects Tested • 5 psychological effects were tested : 1. Anchoring 2. Q&A Mismatch 3. Decomposition 4. Reserve Comfort 5. Planning Fallacy • Every respondent to the survey was randomly asked one of several questions testing different psychological heuristics or fallacies. • 507 volunteers participated: 142 JPLers, 305 college students and 60 other adults. ~2300 data points were collected. 11
  • 12. All answers were graphed and analyzed to establish conclusions 90 80 70 60 50 40 upper Standard Deviation estimate 30 lower Standard Deviation 20 10 0 Psychological category 12
  • 13. Effect #1: Anchoring The objective was to test how easily influenced people may be by a wrong answer – “the anchor.” The anchor set asked: Estimate how many minutes it will take you to clean the kitchen. One respondent estimated that it will take about 10 minutes to finish cleaning up. He may be wrong of course. 13
  • 14. 1. Anchoring Results • The nominal value was 30 min, the anchored case 25 min. • The “best case scenario” estimate (described later) was 27 min which was 2 min LONGER than the anchored result. • The result points out that it is very easy to dramatically skew the estimates by asking anchored questions, such as: “We would like you to come in around $6M”, “I have a bogey of $400k for you”, “the last robot arm we built cost $7M”… 14
  • 15. Effect #2: Q&A Mismatch The purpose was to test if there is a mismatch between the type of estimate expected and provided. Different participants were asked: • Estimate how many minutes it will take you to clean the whole kitchen. • There is a 50% chance that you will finish this task within __ min • There is a 75% chance that you will finish this task within __ min • There is a 99% chance that you will finish this task within __ min 15
  • 16. 2. Q&A Mismatch Results • The 50% confidence estimate was 31 min. The nominal estimate was 30 min. People unconsciously interpret the nominal as the 50% case, meaning that you will exceed your estimate in half the cases! • However, when a manager asks for an estimate he/she expects a much more reliable result, possibly in the 80%-90% confidence range. This points out that there is mismatch between the expectation and the answer. 16
  • 17. Effect #3: Decomposition The objective was to test if decomposing the project into smaller pieces and deeper levels of a WBS improved accuracy of the estimate. Estimate decomposition was simulated by asking: 1. How many minutes will it take to clean all the plates and the sets of silver? 2. How long will it take to clean the sets of coffee cups and saucers? 3. How long will it take to clean the bowls? 4. Etc. 17
  • 18. 3. Decomposition Results • Decomposition average was 31 minutes, just one minute longer than the nominal average (30 min). The attempt at becoming more accurate by cutting up the project was not accomplished. • Decomposition, at least in this case, was more time consuming than helpful. • Deep decompositions provide more detail but compound psychological effects. 18
  • 19. Effect #4: Reserve Comfort This question tested the realism of “a comfortable” reserve. Respondents were asked: 1. I am 90% sure that the time it will actually take to clean the kitchen is within plus or minus __ minutes from my estimate. 19
  • 20. 4. Reserve Comfort Results • The reserve for 90% confidence was 8 min or 28%. The 25-30% seems to be the magical intuitional comfort level that is used by many industries. • When a manager asks for a reserve he/she means “I want to be very sure that I will not exceed this reserve. I want my reserve to cover almost the worst case.” • However, that is not how it is interpreted by the employee. – The worst case estimate was 51 min and required 70% reserve. – The 99% confidence case averaged 45 min. and needed 50% reserve. Both of these cases are significantly higher than the popular 30%. 20
  • 21. Large Projects Reserve Comparison 100% 90% The realistic amount of Needed Reserve 80% budget reserve required 70% for 18 large projects 60% studied is 52%. 50% 40% 30% 20% 10% Recent aerospace projects 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Planned Reserve 21
  • 22. Effect #5: Planning Fallacy The planning fallacy, as defined by Daniel Kahneman and Amos Tversk is a tendency to be overly optimistic in planning. To asses the extent of optimism we asked: 1. In the best case scenario (if everything went as well as possible), how many minutes would it take you to clean the whole kitchen? 2. In the worst case scenario (if everything went as poorly as possible), how many minutes it would take you to clean the whole kitchen? 22
  • 23. 5. Planning Fallacy Results • The following results were obtained: – 51 min worst case – 45 min 99% confidence – 30 min nominal – 27 min best case • These results show how skewed people are toward optimism. The nominal estimate was 10% longer than the best case but 70% shorter than the worst case. • People are so optimistic that it was easy to anchor them down but anchoring up failed. 23
  • 24. Conclusions • To improve the quality of cost estimates it is recommended to diminish the effects of psychological impact on judgment:  Train the managers not to anchor.  Establish proper Estimation Language which makes the questions compatible with common interpretation.  Deep decompositions do not improve accuracy.  Calculate the reserve based on risk.  Account for optimism by including in the baseline likely, historical and common risks. 24

Notas do Editor

  1. Quote this, his authority is important