7 Signs of Bad Project Estimates - And How You Can Fix It

7 Signs You Have a Bad Project Estimate
(And what to do about it)

January 20th, 2010 Kolinger Associates, LLC 2010 All Rights Reserved

Today’s Presentation
Estimating project cost and schedule is a serious challenge to
our companies.

Common symptoms include projects that refuse to end, projects
cancelled, people overworked, missing a window of opportunity
for a new product, and overruns causing an inability to fund
other important initiatives.

Tools may help - but this presentation is primarily about the
ideas and concepts necessary for improvement.

Kolinger Associates, LLC 2010 All Rights Reserved

Overview
• This presentation considers factors that are common to most but some
specific references to software development estimation.
• Me: 25+ years, mostly in large corporations. Developed software,
consulted, trained, managed – and recovered - scores of projects.
• Founded OfficeWork Software in 2005.


Bad Estimates Very Common

A “Bad Estimate” working definition: Money got spent and the required
result was not achieved in time.

A few sundry findings:

• 67% of all projects under-planned by 290%
• 1% of projects meet time, budget, and scope goals
• 60% of all projects over 50k Lines of Code cancelled before
completion.
• Common gripe: “<<your organization>> does not deliver”

(82.3% of all statistics are made up on the spot. Just kidding.)


The Saga
1. Sufficient assets are not allocated, and there is not enough time.

2. The schedule (if there is one) starts to slip, one day at a time.

3. The project manager suddenly realizes the slippage and seeks a culprit.

4. People from various departments involved accuse people from other departments of
delaying the project. More time is wasted in finger-pointing.

5. To make up time, the project manager decides to “crash” the project by applying
more assets/resources to all activities that are currently being performed.

6. Everyone scrambles to crash his job, and people are infuriated to find out that they
are either further behind or finished and their part is not yet needed. Interest wanes
as people chafe under the new delays. The project is over-budget due to all the
crashing, and the whole thing is either:

– a. on-time but shoddy,
– b. well-done but late,
– c. both shoddy and late, or
– d. abandoned.

Dr. Robert Graham – Project Management as if People Mattered

Question
Think of a recent project, that went over
budget, was late, or fell short of the original
scope.

– Why did that happen?
– What could you have done about it?

(Write down your answer)


Test Your Estimation IQ
Question: Why do IT Project Managers
estimate at the drop of a hat?
Answer: Genetic Mutation!
• Have Project
• Will Estimate
• No Requirements Needed!

Actual Project Manager DNA !

Warning: Do not ride in the elevator with your
project sponsor! (Hint: You might rattle off an estimate!)

Gopal Kapur, Center for Project Management


Question?

1. How many of you have managers who
pressure for cost and date estimates before
requirements are clear?
2. How many of you are those managers?
3. What are you doing to encourage better
estimating?
4. How is it going?


Let’s Estimate!
The Project (Input):
Web-based information system. (Legacy system
already exists; let’s just make it web-based)

The
Estimation
(Magic wand)
Process

The Estimate (Output):

– 12 months
– 4.4 FTE
– 5,600 hours
– $400,000

“We will have it for you in 12 months”


What to look for
So, how can you recognize the bad estimate…
before you get bit?

12 months
4.4 FTE
5,600
hours
$400,000


1. When You Hear ‘Done’ More Than 4 Times.
So, what the heck does ‘done’ look like?
Look for a clear, written definition of the end-result.

•Project ‘success’ is meeting expectations. Period.
• Common finding:
– Everyone ‘knows’
– Nobody agrees
•Define the project – in writing - one-pager:
– Description: (What, where, when, how much)
– Desired Results: (Benefits that will be achieved)
– In Scope / Out of Scope
– Assumptions
•De-babelize –Use language everyone understands
•Sign-off from the sponsor and each stakeholder is critical.
(Harder than it sounds)


2. When you Hear, “What is the Plan?”
The estimate is based on the plan. Look for the plan with
evidence of traceability to the project definition and make
sure everyone is aware.
If you ever hear people saying, “I don’t know what the plan is”, take action!!

Your job as project manager is to keep everyone honest to the project’s
schedule, scope, and budget. Even if the estimate were ‘perfect’ you
can render it useless through a poor plan.
• The plan includes, but is more than the project schedule
• The plan includes the project definition, the people, a WBS (Work Breakdown
by deliverable), estimates, resource assignments
• Milestones and activities are connected (or linked) so that a critical path can be
identified and managed.
• Connections are critical as they indicate hand-offs from one person to another.
• Each deliverable has an owner
• Each person on the team is associated with a deliverable
• Have an independent group review the plan. (Find bug on paper vs.
production).


Clue: Missing Dependencies

In this case, ‘date’ was the primary goal


View resource graph and Gantt


Review usage in excel


3. When You Have a Single Number Estimate
Look for a schedule based on appropriate estimate ranges

Two critical lessons:
1. Projects are bigger than they appear.
2. The road to project hell is paved with single number estimates.

Gopal Kapur, Center for Project Management

Best Case Most Likely Worst Case

Software is “different”


Estimating Confidence in Software Delivery
Effort Duration
Project cost Observed Limits of Estimate Accuracy Project
(effort and size) schedule
+ 400 %
4x 1.6x
Overestimated

2x 1.25x

Completion
1.5x 1.15x
1.25x 1.1x
Exact 0% 1.0x 1.0x
Fair 0.8x 0.9x
0.67x 0.85x

0.5x 0.8x
Rough
Underestimated
0.25x 0.6x
Sizing - 400%
Initial Approved Requirements Product Detailed Product
product product specification design design complete
definition definition specification specification

Best Most Likely Worst

10% 50% 90%
Chance Chance Chance
B. Boehm – SW Engineering Economics


4. When It Has Not Been Done Before
Look for an estimate that factors your capability
Project management is all about making promises and keeping
promises.
Promises are more likely to be kept if:
• Made voluntarily (you agree)
• It’s your commitment (your data)
• it is according to capability (i.e., you have proven ability)
Include Risk Management (Project Management for Adults)
To maintain a viable estimate constantly manage project adversity
and team capability

Stan Rifkin – Master Systems
Tom DeMarco - ASG
Robert Charette - ITHABI


5. When You Can’t See Evidence of
Historical Data
Look for an estimate that uses historical performance data as a reference

• The estimate should consider a reasonable work week for resources based
upon historical data for individuals and the organization.
• Examples:
– 60% productivity or 24 hours/week/person
– Individual characteristics
• Technical skill factor
• Business knowledge factor
– Careful use of overtime
– Defect and rework rates

Only lunch breaks are 100% productive


Efficiency


Question
• What percent of a programmer’s time is
spent coding?
• How is this factored into the estimate and
schedule?


What do Programmers Do?


Look for a Repeatable Estimating Process

Historical data is only valuable (normalized) in the context of a
repeatable process


Estimation Methods
• Formula
• Activity-based
• Analogy


How to Estimate
(Kapur “Flinch” Method)

$850B …. For PHASE 1 !


Balancing the Estimate

Your Goal: Commitment according to capability!


Project Difficulty vs. Management Capability

Project: Web-Enabled Business
Information System


Historical Data: Example of Estimating Maturity
Effort Duration
Project cost Observed Limits of Estimate Accuracy Project
(effort and size) schedule
+ 400 %
4x 1.6x
Overestimated

6. Completion
2x 1.25x

1.5x 1.15x
1.25x 1.1x
Exact 0% 1.0x 1.0x
0.8x 0.9x
0.67x 0.85x

0.5x 0.8x

Underestimated
0.25x 0.6x
- 400%
Initial Approved Requirements Product Detailed Product
product product specification design design complete
definition definition specification specification

Legend
Idea / OP H/L Requirements Most Code Complete Project Complete
- Effort / Budget
10/1/02 9/1/03 3/1/04 8/2/04
- Schedule ~ 392% Variance ~ 317% Variance Requirements ~ 33% Variance
FTE 4.4 FTE 4.4 Complete *1 FTE 13.4 FTE 17.9
2/1/04
~ 80%Variance Footnote: * 1. Independent Estimate at point
FTE 10.0 of most requirements complete


An Example of Estimating [Im]Maturity
Estimation Accuracy
Commercial Web System

Project OP / Idea H/L Req Most Code Project
Milestone (Forecast) Done Complete Complete
(Forecast) (Forecast) (Actual)
Date of Estimate 10/1/02 9/1/03 3/1/04 8/1/04 Final
Effort Hours 5,600 unknown ~ 16,000 25,000
Cost $ 402,000 $ 475,000 $ 1,484,429 $ 1,979,238
Cost Variance 392% 317% 33% 0%
FTE 4.4 4.4 13.4 17.9
FTE Variance 311% 311% 33% 0%


Where did the money go?

Pay to put bugs in.

$280k

$100k
$20k

$560k $620k

Pay to take bugs out.

How much waste? ~60% Re-work!


Quality Results
Defects Found Pre and Post Implementation

1. Defects Discovered in Testing (after “Code Freeze”):

Code defects: 977 (70%)
Missing / Incorrect Requirements: 394 (30%)
Total: 1,371

2. Defects Discovered by the User in Production:

First 90 days: 136


Better Estimation via Better Control of Quality
Comparing Defect Removal Results of Two Projects

Case A Case B

Activity (Dollars) (Person Hours) Defects (Dollars) (Person Hours) Defects

Top-Level
$2,875 115 50 $0 0 0
Design Review

Detailed
Cheap $17,500 700 140 $0 0 0
Design Review

Code
$42,500 1,700 110 $0 0 0
Inspection
Unit Test $19,250 770 50 $38,000 1,500 125
Expensive
Integration
$62,250 2,490 100 $107,205 4,290 190
Test

Total $128,625 5,775 450 (90%) $145,250 5,810 315

Maintenance $325,000 13,000 50 $797,500 31,900 185

Lifecycle Total $453,625 18,755 500 $942,750 37,710 500

Reference: Capers Jones, SPR


Question – Crash Schedule?
• What happens when we compress the
schedule?
• Good to know when devising a plan that
will work – and some exec asks for it
sooner than you can deliver.
• You need to know the minimum
development time.


Crashing the Schedule


Effects of Schedule Compression - 1
What’s the effect of a 15% schedule reduction?
Before
Optimized Compressed
Project Size: 100 KLOC
Team Size: 11
Cost $1.5 MM
Duration 23 months
MTTD: 2.7 days

Can we go faster with more people?
QSM


What’s the effect of a 15% schedule reduction?
Before After

Optimized Compressed
Project Size: 100 KLOC Same
Team Size: 11 24
Cost $1.5 MM $2.7 MM
Duration 23 months 20 months
MTTD: 2.7 days 1 day

QSM Worth it? Really?


Quality Considerations
A B

20 Months 23 Months
Defects / Mistakes

Time
• An individual’s learning curve must be accounted for in a shorter schedule.
• There is a minimum development time!


Software Development Effort - 9 Projects
Intense Schedule Pressure

Schedule
Cost


Deadline Effect


Improving Process Reduces Costs

For a typical software system, consisting of 500K Lines of Code:

SEI Effort Defects Total Cost
Level (P. Mos.) (Number) (Mill. $)
I 16,362 25,069 163.3
II 6,468 9,909 64.7
III 1,876 2,874 18.8
IV 866 1,326 8.7
V 342 524 3.4

Significant savings as a result of achieving Level 2

Most of these improvements come from better project management!
Stan Rifkin, Master Systems
Larry Putnam, QSM
Ray Dion, Raytheon


Right-sizing ... or Dumb-sizing?
Two Competitors, Same Products, Same Sales Revenues, Different Cultures

Bureaucratic Robust
Core
Management Core Fixers
Management
24% 29%
31%

Process Fixers
Improvement 63%
13%
Process Improvement
40%

Total Employees = 152 Total Employees = 80

Mark Blaxill, The Fallacy of the Quick Overhead Fix, July 1991 Harvard Business Review

Main Build – Estimated Duration


Main Build Time - 9 Projects
Historical Data

Months Duration
Industry Average

ESLOC (Thousands)


Main Build Effort - 9 Projects

Historical Data

Person Months
ESLOC (Thousands)


Project Staffing Comparisons
35.0
61% Company Peak Staff
Projects from XYZ
Division Benchmark Peak Staff

30.0

108%

25.0 Percent by which this
project exceeds the typical
77%
business systems average
staffing

20.0
64% -11%

Projects from ABC Division
15.0

0%
10.0
0%
180%
-8.3%
-25% 0%
5.0

0.0
A B C D E F G H I J K


A Business Case for Process
Improvement at Company X
50 KLOC, Productivity Index (PI)=15, MBI=0.2

SEI Calendar Time Effort Defects Total Cost (1,000s)

Level (Mos) (PMos) Discovered Shipped Median Lowest Highest

I 18 67 297 11 614 202 20901

II 11 16 71 3 148 109 196
III 9 9 40 1 82 58 105

IV 7.5 5 22 1 44 32 57

V 5 2.5 11 0 23 5 31


Benchmark Power

One company could save $7 million in a portfolio of 14
projects by making the right improvements in their
management approach. In addition, this portfolio of
projects would finish 3.5 months sooner and with a much
higher reliability/quality. This remarkable analysis is made
possible by using the high-performance benchmark
database and SLIM toolset.


Know your capability !!!
Commitments must be tempered with known
capability!

Project Difficulty Management Capability

Question:
• For your current project, where would you look for a mismatch of
commitment and capability?
• How should you respond?


6. When You Have to Micro-Manage
People - Place them where strengths are leveraged, and their
weaknesses really don’t matter.

– People aren’t fungible
– Consider:
• 85% feel they are in the wrong job
• 10% are in their dream job
• 95% confess they lie on the job
– Misplaced people will disrupt “It’s time for performance review!”
– Fear repels learning and improvement
– Focus on developing strengths not fixing weaknesses
– Company slogans and performance reviews demoralize

“Malicious Compliance” – be careful what you
ask for! You might get it!


7. When You Hear Any Answer That Starts
With, “Well …”
Look for effective project reviews and the signs of being off-track

• A ‘perfect’ estimate is valid as long as you manage to the plan.
• Avoid use of ‘percent complete.’
– Remember the “90:90 Rule of Project Tracking.”
• The first 90% of the project takes 90% of the work.
• The last 10% takes the other 90%!
• Do not be bamboozled in project reviews! Verify:
– 1. Do we have the right people?
– 2. Do we have a good plan?
– 3. Are we going to make it?


“Relax – We’re 90 Percent Complete!”

Traditional software progress measures do not provide visibility
Software Engineering Economics, Boehm – pp 608


7 Signs – A Review
1. When You Hear the Word ‘Done’ 4 Times.
So, what the heck does ‘done’ look like?
2. When You Hear, “What is the Plan?”
Look for the plan with evidence of traceability to the project definition and make sure
everyone is aware.
3. When You Have a Single Number Estimate
Look for a schedule based on appropriate estimate ranges
4. When It Has Not Been Done Before
Look for an estimate that factors in data of your organization’s capability
5. When You Can’t See Evidence of Historical Data
Look for an estimate that uses historical performance data as a reference
6. When You Have to Micro-Manage
Get the right people in the right place
7. When You Hear Any Answer That Starts With, “Well
…” Look for effective project reviews and the signs of being off-track


Conclusion
You must protect your company from a bad estimate

• Radical improvement is possible
• Information is not enough
• Needed:
1. Invest in knowing your capability
2. Learn to make commitments according to capability
3. An outside perspective may help


Acknowledgements
My thanks to the following individuals who have made important
contributions to the body of knowledge presented in this document.

• Gopal Kapur – Center for Project Management
• Dr. Stan Rifkin – Master Systems
• Dr. Robert N. Charette – ITABHI Corporation
• Dr. Barry Boehm - USC Center for Software Engineering
• Dr. Gerald Chester – Strategies@Work
• Dr. Michael Deutch – Hughes Aerospace
• Dr. Robert Graham - SMC
• Tom DeMarco – Atlantic Systems Guild
• Larry Putnam, Sr. – Quantitative Software Management
• Dan Swaigen – PM Connect
• Gordon Landies – GL Ventures
• John Maher, CPA – Maher Accountancy
• Dennis Peacocke – SCS
• Alan Lashbrook – AT&T
• Ron Linski – Alliance Business Consulting
• Betsy Guthrie - Autodesk


About Us
Kolinger Associates provides solutions and advice for better estimating and
managing large projects. With our help you will …

Spend your money a little differently … and get a much better result.

• Process assessment and development
• Planning and Estimating Training
• People Assessment
• Rescue Off-Track Projects
• Tools Training and Implementation
• Project Reviews

Contact 415-246-7264 – joe@kolinger.net - www.kolinger.net


7 Signs of Bad Project Estimates - And How You Can Fix It

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a 7 Signs of Bad Project Estimates - And How You Can Fix It

Semelhante a 7 Signs of Bad Project Estimates - And How You Can Fix It (20)

Mais de Kolinger & Associates, LLC

Mais de Kolinger & Associates, LLC (6)

Último

Último (20)

7 Signs of Bad Project Estimates - And How You Can Fix It