Recent College of Performance Management Webinar on using Technical Performance to inform Earned Value Management. Six steps to building a credible Performance Measurement Baseline to connect the dots between all the elements of the program

1. We
want
to
thank
all
of
you
for
joining
us
in
todays
Webinar.
We’ll
be
presen9ng
a
quick
overview
of
the
material
published
in
Measurement
News
around
building
a
credible
Performance
Measurement
Baseline
(PMB)
and
assessing
program
performance
in
ways
beyond
just
the
Earned
Value
numbers.
We’ll
show
the
steps
needed
to
development
that
PMB
and
assess
program
progress
using
the
measures
contained
in
the
IMP
and
IMS.
These
measures
start
with
Measures
of
Effec9veness
and
Measures
of
Performance
in
the
Integrated
Master
Plan.
The
outcomes
of
Work
Packages
in
the
Integrated
Master
Schedule,
can
inform
the
Technical
Performance
Measures
at
the
subsystem
and
system
level
in
the
IMP.
At
all
level
reducible
and
irreducible
risks
play
a
cri9cal
role
in
defining
a
credible
PMB.
The
buy
down
of
reducible
risks
and
the
margin
needed
to
handle
the
irreducible
risk
must
be
represented
in
the
IMP
and
the
IMS.
1
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

2. 2
The
use
of
Earned
Value
Management
is
guided
by
EAI-­‐748-­‐C,
where
32
Guidelines
define
the
processes
needed
to
have
a
compliant
Earned
Value
Management
System
(EVMS).
Early
in
748-­‐C,
it
states
we
should
objec9vely
assess
accomplishments
at
the
work
performance
level.
This
means
we
need
to
measure
the
work
performance
where
the
work
is
performed.
This
is
logical.
If
we
plan
the
work
at
a
specific
level
–
meaning
the
WBS
level
–
then
we
should
assess
the
performance
of
the
work
at
the
same
level.
748-­‐C
also
tells
us
that
quality
and
technical
content
of
the
work
are
outside
of
the
processes
of
Earned
Value.
This
sounds
a
bit
odd,
assessing
the
accomplishments
of
the
work
surely
must
involve
assessing
the
technical
and
quality
aspects
of
that
work.
In
this
webinar,
we’ll
focus
on
connec9ng
the
dots
between
Earned
Value
Management
and
Technical
Performance
assessment.
But
we’re
going
to
go
into
more
detail;
around
risk,
because
it’s
a
topic
that
inconsistently
dealt
with
on
many
programs.
Our
Measureable
News
ar9cle,
Building
A
Credible
Performance
Measurement
Baseline
Establishing
a
credible
Performance
Measurement
Baseline:
with
a
risk
adjusted
Integrated
Master
Plan
and
Integrated
Master
Schedule,
starts
with
the
WBS
and
connects
Technical
Measures
of
progress
to
Earned
Value,
addressed
all
measures
of
performance
in
more
detail.
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

3. Using
Earned
Value
Management
processes
is
a
cri9cal
success
factor
for
projects.
EVM
shows
the
Efficacy
of
your
dollar.
§ For
the
dollar
spent,
did
you
get
a
dollar
in
value?
§ Did
you
get
that
dollar’s
value
on
the
day
you
planned
to
get
that
value?
§ If
you
didn’t
get
a
dollar’s
value
for
a
dollar
spent,
when
will
the
dollar
of
value
be
earned?
The
answers
to
these
ques9ons
provide
visibility
into
the
performance
of
the
project.
But
this
assumes
that
the
assessment
of
the
value
earned
in
exchange
for
the
cost,
has
some
connec9on
to
other
abributes
of
the
project.
§ Effec9veness
of
the
project’s
outcomes
in
accomplishing
the
mission.
§ Performance
of
the
project’s
outcomes
in
enabling
the
effec9veness.
If
the
outcomes
perform
as
needed
(fit
for
purpose)
or
aren’t
effec9ve
in
accomplishing
the
mission
(fit
for
use),
then
the
earned
value
and
it’s
indices
–
Cost
Performance
Index
and
Schedule
Performance
Index
–
don’t
match
the
technical
performance
of
the
project
3
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

4. 4
Using
DI-­‐MGMT-­‐81861
as
our
star9ng
point,
we
need
to
show
how
to
connect
“objec9ve
technical
measures”
with
the
Earned
Value
Performance
metrics
of
the
program.
Some
of
you
may
not
be
subject
to
81861.
It
describes
how
to
submit
the
monthly
Integrated
Program
Management
Report
on
a
monthly
basis,
star9ng
with
the
WBS
cumula9ve
performance
to
date
and
the
current
period
performance.
These
two
measures
should
match.
If
not
we
need
to
find
out
why,
which
one
is
in
error,
and
what
the
correc9ve
ac9ons
are
to
get
them
back
in
syn.
When
they
do
match
we
then
have
visibility
to
all
three
variables
of
all
projects.
Cost,
Schedule,
and
Technical
Performance.
If
they
don’t
match,
we
not
have
a
place
to
look
for
the
mismatch.
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

5. Here’s
one
look
at
the
element
of
a
project.
§ WBS
–
we
all
know
about
breaking
down
the
work
into
the
individual
deliverables.
§ Cost
–
the
budget
needed
to
produce
these
deliverables.
§ Statement
of
Work
–
describing
what
needs
to
be
delivered.
§ Technical
Performance
Measures
–
the
assessment
of
the
technical
performance
if
each
deliverable.
There
are
other
measures
than
just
Technical
Performance
Measures.
• Measures
of
Effec9veness
• Measures
of
Performance
• Key
Performance
Parameters
• Other
…ili8es
of
the
project
–
maintainability,
serviceability,
…
§ Integrated
Master
Plan
and
Integrated
Master
Schedule
–
showing
the
planned
increasing
maturity
of
the
deliverables
and
the
planned
work
to
make
this
increasing
maturity
appear
on
the
needed
date
§ Risk
–
both
reducible
and
irreducible
risk
to
the
success
of
the
project
5
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

6. The
elements
of
the
Performance
Measurement
Baseline
begin
with
the
Statement
of
Work
(SOW)
which
may
include
a
Concept
of
Opera9ons.
These
document
say
what
the
system
must
be
able
to
do.
The
government
frequently
defines
a
standard
WBS
per
881C,
but
it
is
up
to
the
contractors
to
refine
the
WBS
to
deliver
products
that
sa9sfy
the
requirements
enumerated
in
the
SOW
and
CONOPS.
The
Contractors’
WBS
contains
all
the
products
that
must
be
delivered
and
the
CWBS
Dic9onary
defines
the
scope
of
each
product
and
defines
the
criteria
that
says
when
the
product
meets
the
Technical
and
Opera9onal
Requirements
.
The
Integrated
Master
Plan
(IMP)
is
an
Event-­‐
Based
“plan”
describing
the
maturity
of
each
Major
Deliverable.
These
deliverable
are
comprised
of
systems
and
subsystems
to
meet
the
system’s
Measures
of
Effec9veness
(MOEs)
that
are
ar9culated
in
the
Capability
Development
Document
(CDD)
by
way
of
Key
Performance
Parameters.
Work
performed
in
the
Integrated
Master
Schedule
and
it’s
Work
Packages
produces
the
systems
and
subsystems
the
meet
the
Accomplishment
Criteria.
Risk
Management
is
shown
in
yellow
to
reflect
the
fact
that
the
planning
and
implementa9on
ac9vi9es
must
adjust
for
and
manage
the
risks
that
can
jeopardize
delivery
the
required
capabili9es.
The
chart
also
shows
that
Technical
Performance
Measures
informing
earned
value.
TPMs
show
the
appropriate
technical
progress
at
the
work
package
level
of
detail
in
the
IMS.
TPMs
must
be
met
in
order
for
the
subsystems
and
systems
to
meet
the
acceptance
criteria.
Use
of
the
TPM
permits
the
Program
Manager
to
keep
the
program
on
track
toward
accomplishing
the
required
system
capabili9es.
Cost
and
schedule
progress
depend
on
this
informa9on
to
provide
a
complete
and
consistent
progress
picture.
6
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

7. This
chart
and
the
next
one
describe
6
steps
needed
to
create
the
credible
Performance
Measurement
Baseline.
Step
1:
As
started
earlier,
all
planning
begins
with
a
thorough
analysis
of
the
SOW,
SOO
and
CONOPS
and
this
results
in
a
comprehensive
CWBS
and
Dic9onary
that
defines
the
criteria
of
what
cons9tutes
acceptable
deliverables.
The
WBS
is
paramount
in
defining
the
products.
Step
2:
The
IMP
shows
the
progression
of
maturity
of
those
deliverables
by
assuring
that
the
systems
and
subsystems
meet
the
MOEs
and
MOPs
as
defined
by
the
KPPs.
Step
3:
Major
“reducible
risks”
that
threaten
successful
deliverables
must
be
addressed
in
the
top-­‐level
plan.
The
PM
needs
to
decide
whether
the
risk
will
be
mi9gated
and
his
plans
for
doing
so
will
be
reflected
later
as
he
develops
the
IMS.
The
PM
may
need
to
change
logic,
add
resources
or
9me
or
all
three
within
the
IMS
For
those
risks
that
are
not
mi9gated,
the
PM
must
set
aside
Management
Reserves.
We
describe
how
this
should
be
done
on
the
next
slide.
7
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

8. Step
4:
Aler
the
IMP
is
created
and
major
discrete
risks
plans
are
established,
the
contractor
must
develop
the
IMS
that
contains
work
packages
and
task
ac9vi9es
needed
to
create
the
subsystems
and
system
deliverables
that
meets
the
Accomplish
Criteria.
This
results
in
a
cost-­‐phased
plan
from
contract
start
to
contract
comple9on.
Step
5:
The
cost-­‐phase
plan
must
next
be
adjusted
for
irreducible
risks.
These
are
risks
that
occur
in
the
normal
process
of
developing
such
systems
where
the
PM
can
not
take
specific
ac9on
to
mi9gate
the
risks.
Rather,
the
PM
can
add
a
schedule
cushion
or
Margin
within
the
phased
plan
to
accommodate
these
risks.
This
is
done
by
performing
a
Monte
Carlo
simula9on
on
the
plan
with
the
known
dura9on
distribu9ons
within
the
IMS
ac9vi9es.
More
on
this
later.
When
completed,
this
results
in
a
Performance
Management
Baseline
Step
6:
Aler
the
schedule
margin
has
been
added
to
the
IMS,
the
PM
must
calculate
the
Management
Reserve
using
the
PMB
and
the
specific
irreducible
risks
that
were
not
mi9gated,
to
determine
the
Management
Reserves.
To
do
this,
the
PM
performs
a
Monte
Carlo
Simula9on
on
the
PMB
to
determine
the
confidence
level
of
the
plan
and
calculates
the
amount
of
management
reserve
needed
to
bring
the
contract
budgeted
baseline
(CBB)
to
an
acceptable
confidence
level.
8
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

9. As
state
earlier,
TPMs
show
the
appropriate
technical
progress
at
the
work
package
level
of
detail
in
the
IMS
and
they
are
the
objec9ve
measures
of
progress.
They
must
be
met
in
order
for
the
subsystems
and
systems
to
meet
the
acceptance
criteria.
Cost
and
schedule
progress
depend
on
this
informa9on
to
provide
a
complete
and
consistent
status
picture
at
any
9me.
9
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

10. The
WBS
is
the
star9ng
point
of
our
efforts.
The
WBS
tells
us
what
is
to
be
delivered
on
the
program
and
with
the
WBS
Dic9onary,
SOW,
SOO,
ConOps
and
other
documents,
what
the
objec9ve
technical
measures
are
for
these
deliverables.
In
some
sense
the
WBS
appears
to
be
simple.
In
fact
it’s
hard
work
to
assemble
the
WBS
in
the
proper
manner.
MIL-­‐STD-­‐881C
is
the
star9ng
place.
But
reading
881C
needs
to
take
into
considera9on
the
specific
of
the
program
under
management.
The
WBS
is
the
architectural
structure
of
the
program.
It
shows
the
rela9onship
between
the
skeleton
parts
of
the
program.
10
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

11. The
WBS
as
a
document
needs
to
have
integrity
is
what
is
says
and
what
it
contains.
This
integrity
starts
with
the
MECE
principle.
As
an
informa9on
organizing
processes
–
and
that’s
what
the
WBS
is,
MECE
1. Mutually
exclusive
-­‐
Informa9on
should
be
grouped
into
categories
so
that
each
category
is
separate
and
dis9nct
without
any
overlap
2. Collec9vely
exhaus9ve
-­‐
All
of
the
categories
taken
together
should
deal
with
all
possible
op9ons
without
leaving
any
gaps.
As
an
accoun9ng
system
–
McKinsey
is
where
MECE
started
–
it
makes
sense.
No
double
coun9ng
and
no
overlooking
informa9on
For
the
WBS
it
means
each
deliverable
is
properly
decomposed
from
it’s
parent.
No
cross
connec9ons
between
deliverables.
No
missing
elements
for
the
deliverables
11
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

12. The
“goodness”
of
the
WBS
is
assessed
by
the
visibility
to
the
deliverables,
their
rela9onships
to
each
other,
their
parents
and
children
in
the
MECE
manner.
No
widows,
nor
orphans,
no
rela9onships
between
cousins,
only
parent
childe.
And
the
parent
is
a
signal
parent
.
12
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

13. Risk
management
starts
with
the
WBS.
The
WBS
Dic9onary
contains
a
descrip9on
of
the
outcomes
of
the
work
performed
in
the
IMS
to
produce
the
outcomes
in
the
WBS.
Risks
are
assigned
to
describe
the
impediments
to
producing
the
outcomes
that
fulfill
the
MOEs,
MOPs,
and
TPMs.
Each
of
the
risks
is
entered
into
the
risks
register.
The
probability
of
occurrence,
the
probability
of
the
impact,
the
cost
of
the
impact,
the
cost
to
mi9gate
the
risk,
are
all
in
the
Risk
Register.
With
these
probabili9es,
and
the
period
over
which
they
are
in
effect
is
then
the
basis
of
the
Monte
Carlo
Simula9on
of
the
risk
exposure
to
the
program
for
the
reducible
risks
from
the
WBS.
13
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

14. The
Integrated
Master
Plan
is
a
unique
approach
to
defining
what
Done
looks
like
for
a
program.
It
takes
the
elements
of
the
program
defined
in
the
WBS
through
increasing
maturity
levels
to
produce
the
end
item
deliverables
of
your
contract.
In
a
tradi9onal
approach
the
work
is
planned
with
ac9vi9es
or
collec9ons
of
ac9vi9es.
These
are
linked
in
a
sequence
to
produce
outcomes
from
the
work
efforts.
What
is
missing
is
a
structured
assessment
of
the
increasing
maturity
of
the
products
produced
by
this
work
in
units
of
measure
that
address
the
effec9veness
and
performance
of
the
deliverables.
The
IMP
approach
is
especially
useful
for
development
efforts
where
increasing
maturity
of
system
design
&
integra9on
efforts
leads
to
procurement,
fabrica9on,
assembly
and
test
of
a
system
that
meets
stakeholder
needs.
This
is
the
basic
premise
of
Systems
Engineering.
14
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

15. Our
first
step
here
is
to
recognize
the
difference
between
a
Plan
and
a
Schedule.
The
PLAN
is
a
procedure
used
to
achieve
an
objec9ve.
It
is
a
set
of
intended
ac9ons,
through
which
one
expects
to
achieve
a
goal.
The
SCHEDULE
is
the
sequence
of
these
intended
ac9ons
placed
against
9me,
needed
to
implement
the
PLAN.
But
we
don’t
what
to
mix
the
PLAN
with
the
SCHEDULE.
The
PLAN
is
the
strategy
for
the
successful
comple9on
of
the
project.
It
describes
“What”
needs
to
be
accomplished
to
meet
the
technical
requirements
and
goals
of
the
program.
In
the
strategic
planning
domain,
a
PLAN
is
a
hypothesis
that
needs
to
be
tested
along
the
way
to
confirm
where
we’re
headed.
The
SCHEDULE
builds
off
the
plan,
providing
the
order
of
the
work,
the
expected
9me
to
do
the
work,
and
an
appropriate
level
of
detail
needed
to
execute
the
PLAN.
It
provides
status
of
work
done
and
the
forecast
of
work
to
be
done.
We
need
both.
PLANS
without
SCHEDULES
are
not
executable.
SCHEDULES
without
PLANS
have
no
stated
mission,
vision,
or
descrip9on
of
success
other
than
the
execu9on
of
the
work.
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

16. The
hierarchical
structure
of
an
IMP
depicted
here
helps
understand
the
rela9onship
of
the
elements
of
the
IMP
and
the
rela9onship
of
schedule
to
the
Plan.
The
Events
are
the
top
level
of
the
hierarchy
and
represent
key
assessment
points
for
the
increasing
maturity
of
the
system.
They
are
typically
sequen9al.
For
most
DoD
programs,
events
may
include
a
System
Requirements
Review
(SRR),
a
System
Design
Review
(SDR),
a
Preliminary
Design
Review
(PDR),
a
Cri9cal
Design
Review
(CDR),
a
Test
Readiness
Review
(TRR),
and
End
Item
Delivery.
These
may
not
apply
directly
to
your
program
but
I
would
encourage
you
to
assess
what
the
key
maturity
assessment
events
would
be
in
your
domain.
Each
event
is
then
broken
down
into
accomplishments
against
each
product
or
process
defined
in
the
WBS
that
represent
the
required
maturity
level
established
by
the
event.
Criteria
are
then
developed
for
each
accomplishment
that
provide
objec9ve
measurements
defining
what’s
required
to
assess
an
accomplishment
as
done.
The
schedule
(the
IMS)
then
takes
all
of
the
IMP
elements
and
defines
work
packages
and
tasks
at
an
appropriate
level
of
detail
needed
to
reflect
the
needed
integra9on
between
efforts,
enabling
accurate
forecas9ng
and
16
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

17. Here
are
the
Measures
of
Effec9veness
(MOEs),
Measures
of
Performance
(MOPs),
and
specific
Key
Performance
Parameters
(KPPs)
for
the
turn-­‐of-­‐the
century
Wright
Brothers
Heavier
than
Air
Military
Flyer.
This
informa9on
was
extracted
from
the
actual
Army
Signal
Corps’
Request
for
Proposal
in
December
1907.
The
MOEs
describe
capabili9es
in
the
targeted
opera9ng
environment
and
the
MOPs
get
more
specific
about
how
the
plaoorm
must
perform,
e.g.,
it
must
fly
at
an
average
of
40
miles
per
hour
in
s9ll
air
and
travel
at
least
125
miles.
The
two
KPPs
deal
with
training
and
sustainment.
The
Wright
Brothers
had
six
months
to
design
and
build
the
aircral
and
promised
to
do
so
for
$25,000.
It
was
a
fairly
ambi9ous
schedule
and
geqng
the
aircral
to
meet
the
performance
requirements
meant
designing
and
building
aircral
to
maximize
thrust
and
minimize
drag
and
maximize
lil.
Weight
obviously
had
to
be
controlled
and
kept
within
limits
so
that
MOE
and
MOPs
could
be
met.
In
subsequent
slides,
we
show
how
weight
could
have
been
used
as
a
Technical
Performance
Measure
to
inform
cost
and
schedule
progress
as
the
plane
was
designed
and
built.
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

18. Risks
that
come
for
probability
distribu9on,
that
is
there
is
a
probability
that
something
can
happen,
are
reducible.
We
can
spend
money
to
reduce
the
probability
of
occurrence.
18
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

19. There
are
two
types
types
of
uncertainty
inherent
in
any
type
of
development
endeavor:
those
the
are
specific
and
reducible,
i.e.,
one
can
do
something
about
them
to
reduce
or
eliminate
them
and
those
that
are
irreducible,
ones
that
naturally
occur
is
such
ac9vi9es
the
specifics
of
which
can
not
be
iden9fied
or
mi9gated.
These
two
uncertain9es
are
also
categorized
as
Epistemic
(for
specific
events)
and
Aleatory
(naturally
occurring).
During
step
3,
the
PM
iden9fies
the
specific
risk
events
that
could
jeopardize
the
system
in
delivering
the
requirements.
The
informa9on
about
these
risk
events
are
recorded
in
the
risk
register.
At
a
minimum,
they
include
a
descrip9on
of
the
risk,
the
WBS
elements
which
could
be
affected,
the
probability
of
occurrence
and
the
technical,
cost
and
schedule
consequences
if
they
occur.
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

20. Once
the
specific
reducible
risk
are
iden9fied,
the
contractor
must
decide
which
risks
will
be
mi9gated,
the
cost
of
doing
so,
and
the
remaining
risk
exposure
aler
the
mi9ga9on
ac9ons
have
been
taken.
Risk
mi9ga9on
ac9ons
must
be
included
in
the
IMS.
With
budget
and
defined
outcomes.
These
outcomes
are
the
reduc9on
of
the
risk.
20
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

21. This
is
an
example
of
a
risk
mi9ga9on
plan
“buy
down”
over
9me.
Each
numbered
step
represents
a
specific
ac9on
defined
to
reduce
either
the
probability
or
impact
of
the
risk.
These
ac9ons
are
described
in
the
Risk
Management
Plan,
represent
contract
scope
and
are
budgeted,
scheduled,
and
tracked
in
the
IMS
and
PMB
just
like
any
other
contract
scope.
Measuring
performance
against
a
risk
reduc9on
plan
can
be
a
form
of
technical
performance
measurement.
21
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

22. With
the
Integrated
Master,
the
reducible
risks
from
the
WBS,
let’s
build
the
Integrated
Master
Schedule.
The
IMS
tells
us
what
work
must
be
performed
in
what
order
to
produce
the
outcomes
for
the
Criteria
that
complete
the
Accomplishments
to
increase
the
maturity
of
the
deliverables
assessed
at
each
Event.
22
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

23. The
Integrated
Master
Schedule
is
the
road
map
to
producing
the
outcomes
of
the
project.
This
picture
is
likely
no9onal,
but
illustrates
the
core
concept
of
a
Master
Schedule.
No9ce
the
longest
work
ac9vity
–
training
the
astronauts.
The
other
work
was
technical
and
based
on
standard
engineering
processes.
New
materials
had
to
be
developed,
processes
to
assemble
the
materials,
and
tes9ng
them.
But
the
human
factors
were
unprecedented.
The
IMS
is
the
mechanism
of
delivering
the
Criteria
needed
to
sa9sfy
the
Accomplishments
back
in
the
pyramid
chart
where
the
ver9cal
integra9on
was
shown.
The
founda9on
of
the
program's
success
is
a
credible,
risk
adjusted
IMS.
We’ll
see
how
this
is
done
in
this
sec9on.
23
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

24. The
term
IMS
is
known
in
many
domains.
But
the
Integrated
Master
Plan
(IMP)
is
just
now
coming
to
domains
outside
the
US
Department
of
Defense.
The
IMP
is
an
event–based,
plan
of
a
hierarchy
of
Program
Events,
with
each
Event
supported
by
Accomplishments,
and
each
Accomplishment
supported
by
a
specific
Criteria
to
be
sa9sfied
for
its
comple9on.
The
three
elements
of
the
IMP
are:
1. Event
–
an
assessment
point
at
conclusion
of
significant
program
ac9vi9es;
2. Accomplishment
–
is
the
planned
result(s)
prior
to
or
at
comple9on
of
an
Event
that
defines
a
level
of
the
program's
progress;
3. Criteria
–
provides
a
defini9ve
evidence
that
a
specific
Accomplishment
has
been
completed
at
the
conclusion
of
the
Work
Package.
The
Integrated
Master
Plan
(IMP)
is
expanded
to
a
9me–based
Integrated
Master
Schedule
(IMS)
to
produce
a
networked,
mul9–layered
schedule
showing
all
the
detailed
tasks
required
to
accomplish
the
work
effort
contained
in
the
IMP.
The
IMS
flows
directly
from
the
IMP
and
supplements
it
with
addi9onal
levels
of
detail.
It
incorporates
all
of
the
IMP
events,
accomplishments,
and
criteria;
to
these
ac9vi9es
it
adds
the
detailed
tasks
necessary
to
support
the
IMP
criteria
along
with
each
task’s
dura9on
and
its
rela9onships
with
other
tasks.
This
network
of
integrated
tasks,
when
9ed
to
the
start
date
(e.g.,
contract
award),
creates
the
task
and
calendar–based
schedule
that
is
the
IMS.
24
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

25. Tom
spoke
about
the
reducible
risks
the
result
from
the
reducible
uncertainty.
These
risks
–
the
reducible
ones
–
can
be
bought
down
by
spending
9me
and
money.
Irreducible
uncertainty
and
the
resul9ng
risk
can’t
be
bought
down.
We
can’t
pay
money
to
make
the
irreducible
uncertainty
go
away.
By
defini9on
it
is
always
there.
It’s
part
of
program.
We
can
only
handle
irreducible
risk
with
margin.
Cost
margin,
technical
margin,
or
schedule
margin.
25
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

26. Irreducible
uncertainty
starts
with
the
natural
variability
of
the
underlying
processes.
The
natural
randomness
of
the
work
ac9vi9es,
the
technical
performance
of
a
product
or
process.
This
is
the
Deming
variance.
Spending
money
in
an
abempt
to
reduce
it
is
a
waste
–
Muda
in
Japanese
As
well
ambiguity
about
the
variances
in
a
processes
is
irreducible.
This
is
a
natural
ambiguity
that
more
money
and
9me
can’t
address.
So
margin
is
needed
to
protect
the
project
from
these
naturally
occurring
variances
and
ambigui9es.
The
last
class
of
irreducible
risk
is
know
risk,
but
a
risk
we
can’t
do
anything
about.
That
is
it
is
un-­‐
mi9gable.
There
is
a
possibility
that
the
comet
will
hit
the
building
we
work
in.
It
is
possible
but
not
likely
probable.
But
seemingly
other
low
probability
events
have
been
in
the
news.
The
probability
that
an
earthquake
will
trigger
a
tsunami
that
will
breach
a
seawall
built
to
a
specific
height
and
flood
the
basement
of
the
nuclear
power
sta9on
where
the
auxiliary
power
–
the
site
power
–
generators
were
places
because
it
was
too
expensive
to
place
them
on
the
3rd
floor
of
the
equipment
room.
The
result
was
the
lose
of
site
power
and
the
inability
to
restart
the
reactor
when
it
scrammed
and
was
cooling
too
fast
and
was
going
the
warp
the
fuel
rods.
All
known
but
un-­‐mi9gateable
–
at
least
at
the
9me
of
the
design
basis.
Aler
the
fact
of
course,
it
was
mi9gable.
So
tes9ng
the
“mi9gatability”
of
the
risk
is
cri9cal.
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

27. One
star9ng
point
for
irreducible
risks
is
Reference
Class
Forecas8ng.
RFC
can
be
applied
to
reducible
risks
as
well.
Bent
Flybjerg’s
picture
here
are
the
steps
to
developing
a
reference
class
forecast.
Reference
class
forecas9ng
was
originally
developed
to
compensate
for
the
type
of
cogni9ve
bias
iden9fied
in
pioneering
work
on
planning
and
decision
making
carried
out
by
Kahneman
and
Tversky
(1979a)
and
Kahneman
1994.
Kahneman
and
Tversky
found
that
errors
of
judgment
are
olen
systema9c
and
predictable
rather
than
random,
manifes9ng
bias
rather
than
confusion,
and
that
any
correc9ve
prescrip9on
should
reflect
this.
They
also
found
that
many
errors
of
judgment
are
shared
by
experts
and
lay
people
alike.
Finally
Kahneman
and
Tversky
found
that
errors
remain
compelling
even
when
one
is
fully
aware
of
their
nature.
Thus
awareness
of
a
perceptual
or
cogni9ve
illusion
does
not
by
itself
produce
a
more
accurate
percep9on
of
reality,
according
to
Kahneman
and
Tversky
(1979b:
314).
Awareness
may,
however,
enable
one
to
iden9fy
situa9ons
in
which
the
normal
faith
in
one's
impressions
must
be
suspended
and
in
which
judgment
should
be
controlled
by
a
more
cri9cal
evalua9on
of
the
evidence.
Reference
class
forecas9ng
is
a
method
for
such
cri9cal
evalua9on.
Human
judgment,
including
forecasts,
are
biased.
Reference
class
forecas9ng
is
a
method
for
unbiasing
forecasts.
27
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

28. The
cri9cal
concept
in
irreducible
uncertainty
and
the
resul9ng
risk
is
that
every
node
in
the
IMS
is
a
probabilis9c
process.
The
fancy
term
is
a
stochas9c
process.
That
is
it
behaves
sta9s9cally
and
resul9ng
in
a
naturally
occurring
variability
that
cannot
be
reduced.
When
these
stochas9c
processes
are
connected
in
a
network,
their
individual
behavior
produces
other
–
many
9mes
in
predictable
–
behaviors.
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

29. For
an
example
of
schedule
margin
needed
to
protect
from
the
natural
varia9ons,
let’s
look
at
a
case
study
developed
for
a
training
course.
Here
the
contract
due
date
was
August
31
1908.
For
the
Wright
Brother’s
delivery
of
their
Flyer
The
determinis9c
schedule
shows
a
date
of
August
4
1908.
The
80%
confidence
date
for
the
deliver,
given
the
naturally
occurring
variances
in
the
work
dura9on
was
August
14
1908.
The
result
is
a
10
day
schedule
margin
between
the
determinis9c
date
and
the
80%
confidence
date
for
the
probabilis9c
comple9on.
The
steps
needed
to
produce
the
IMS,
model
the
probabilis9c
distribu9ons
and
run
the
Monte
Carlo
are
beyond
our
short
9me
here.
So
we’ll
use
this
as
a
no8onal
example
for
now.
29
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

30. Once
you’ve
determined
an
acceptable
confidence
level
for
your
schedule
and
the
amount
of
schedule
margin
needed
to
achieve
that
confidence,
then
you
need
to
decide
where
to
place
it
in
your
schedule
and
how
to
manage
it.
Some
prefer
to
hold
it
all
at
the
end
item
delivery
milestone.
That
can
be
effec9ve
but
it
can
also
be
hard
to
communicate
rules
for
acceptable
consump9on.
An
alternate
approach
is
shown
here
with
margin
baselined
at
key
points
in
the
process.
Good
candidates
for
placement
of
margin
are
high
convergence
points-­‐where
many
thing
come
together
(integra9on
points)
or
where
many
things
get
tested
together;
these
are
olen
“points
of
discovery”
in
complex
systems
where
things
don’t
go
as
planned;
these
represent
irreducible
uncertainty
that
likely
can
only
be
managed
with
margin.
How
you
manage
these
is
just
as
important
as
where
you
place
them.
In
the
graphic,
no9ce
the
dividing
line
between
green
and
yellow
is
not
a
straight
line
(9me-­‐based)
degrada9on.
It’s
stair-­‐stepped
with
each
step
9ed
to
re9rement
of
irreducible
risk
associated
with
comple9on
of
key
tests.
With
this
approach,
all
team
members
knew
if
we
fell
behind
(fell
into
the
yellow)
we
had
to
regain
the
9me
back
(work
a
Saturday).
We
measured
performance
to
this
plan
and
implemented
correc9ve
ac9on
twice
to
“Stay
Green”.
Forcing
correc9ve
ac9on
to
stay
green
enabled
on-­‐9me
delivery
of
this
planetary
spacecral
in
a
9ghtly
constrained
schedule.
30
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

31. CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

32. With
each
of
the
elements
in
Steps
1
through
5,
let’s
assemble
them
into
our
Performance
Measurement
Baseline.
The
Performance
Measurement
Baseline
(PMB)
is
a
9me-­‐phased
budget
plan
for
accomplishing
work
against
which
contract
performance
is
measured.
It
includes
the
budgets
assigned
to
scheduled
control
accounts
and
the
applicable
indirect
budgets.
For
future
effort,
not
planned
to
the
control
account
level,
the
PMB
also
includes
budgets
assigned
to
higher
level
Contractor
Work
Breakdown
Structure
(CWBS)
elements,
and
to
Undistributed
Budgets
(UB).
It
does
not
include
Management
Reserve
(MR).
32
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

33. When
we
started
this
Webinar
we
men9oned
connec8ng
the
dots.
Well
here’s
the
dots
and
this
is
how
they
are
connected.
This
chart
speaks
for
itself,
with
the
dots
connected
to
the
lines.
We’ve
included
the
no9on
of
con9ngency
here.
The
Department
of
Energy
uses
con9ngency
at
the
agency
or
field
office
level.
The
Department
of
Defense
doesn’t
formally
do
this.
33
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

34. The
Wright
Brothers
proposed
to
build
a
Heavier
than
Air
Flying
machine
that
met
specific
performance
and
measures
of
effec9veness.
In
order
deliver
the
system
that
met
these
requirements,
weight
had
to
be
controlled.
On
the
next
slides,
we
use
weight
as
the
TPM
to
inform
cost
and
schedule
progress
during
this
six
month
design/build
schedule.
34
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

35. Let’s
re-­‐emphases
that
point
that
technical
progress
needs
to
drive
the
reported
cost
and
schedule
progress
so
that
stakeholders
have
a
clear
and
consistent
status
picture
and
provide
leading
indicators
so
that
management
can
take
correc9ve
ac9on.
Keeping
the
program
green
is
the
day-­‐to-­‐day
focus
and
that
ensure
that
the
system
is
delivered
on
9me,
within
the
resources
and
accomplishes
the
required
capabili9es.
Our
Measureable
News
white
speaks
to
these
execu9on
process
in
more
detail
by
having
Earned
Value
measures
be
informed
by
objec9ve
Technical
Performance
35
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

36. Here
is
how
technical
performance
informs
the
cost
and
schedule
status
and
in
turn,
how
the
reported
cost
and
schedule
informa9on
informs
the
technical
progress.
In
this
picture,
weight
is
above
the
upper
design
threshold
for
the
Wright
Brothers’
Flyer
and
the
report
performance
is
consistent
because
it
falls
below
the
planned
Budgeted
Cost
of
Work
Scheduled.
36
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

37. Here’s
the
summary
of
the
steps
needed
to
create
a
credible
plan
–
one
that
has
been
risk-­‐adjusted
for
both
reducible
and
irreducible
risks,
and
quan9fiably
protected
with
Management
Reserves
to
cover
the
known
risks
that
were
not
mi9gated.
The
chart
also
reminds
us
that
cost
and
schedule
progress
must
be
based
on
objec9ve
technical
measures
of
performance
and
these
are
typically
manifested
in
Technical
Performance
Measures
(TPMs).
37
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n

38. Here
are
some
references
that
have
guided
our
efforts
in
connec9ng
the
elements
of
a
credible
Performance
Measurement
Baseline.
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n
38

39. CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n
39

40. 40
CPM
Webinar,
3
October
2014
Using
Technical
Progress
to
Inform
Earned
Value
Performance
Glen
B.
Alleman,
Niwotridge,
L.L.C.
Tom
Coonce,
Ins9tute
for
Defense
Analyses
Rick
Price,
Lockheed
Mar9n