Technical Performance Measures provide assessment of Physical Percent Complete in Earned Value Management Systems in ways not possible with simple measures of progress to plan stated by Control Account Managers

1. Integrated Master Plan
The Foundation for Program Success
Starting with the Integrated Master Plan provides clear and
concise descriptions of what “done” looks in units of
measure meaningful to the decision makers.
Measures of Effectiveness (MoE), Measures of
Performance (MoP), Technical Performance Measures
(TPM), Key Performance Parameters (KPP)

2. Agenda for Our Workshop
• Explain the importance of using an IMP to
create a picture of what done looks like
• There is a study of the 3 options we are
exploring to better connect technical
performance to cost and schedule
• Provide hands on example to create one
of those options
2

3. The Situation
• There is evidence that BCWP and actual
program technical performance are not
connected.
• There are several possible reasons:
– Connecting TPM to EV starts with the IMP, and
the IMP is not mandated for ACAT1’s
– The Systems Engineering community is not
always connected with the EV community
– The DI-MGMT-81861 IPMR DID states the IMP is
“optional.”
– The guides do not clearly state how the SE
community and EV community should be working
together to get the TPMs into the PMB
3

4. 4 Root Causes of ACAT1 Program
Performance Shortfalls
Unrealis)c
Performance
Expecta1ons,
missing
Measures
of
Effec1veness
(MoP),
and
Measures
of
Performance
(MoE)
Unrealis)c
Cost
and
Schedule
es1mates
based
on
inadequate
risk
adjusted
growth
models.
Inadequate
assessment
of
risk
and
unmi1gated
exposure
to
these
risks
without
proper
handling
plans.
Unan)cipated
technical
issues
without
alterna1ve
plans
and
solu1ons
to
maintain
effec1veness.
Unan1cipated
Cost
and
Schedule
Growth
The
Lens
of
the
Performance
Assessment
“Borrowed”
with
permission
from
Gary
Bliss,
Director,
Performance
Assessments
and
Root
Cause
Analyses
(PARCA),
Office
of
Assistant
Secretary
of
Defense
for
Acquisi1on.
Originally
Observa1ons
from
AT&L/PARCA's
Root
Cause
Analyses
February
16th,
2011
4
IPMR
is
our
lens
to
reveal
early
unan1cipated
growth
in
Cost
and
Schedule
through
early
assessment
of
physical
percent
complete,
.e.g.
Technical
Performance
Measures

5. Elements of Program Success
5
Objec1ve
Status
and
Essen1al
Views
to
support
the
proac1ve
management
processes
needed
to
keep
the
program
GREEN
Risk
Management
SOW
Techncial
and
Opera1onal
Requirements
CWBS
&
CWBS
Dic1onary
Integrated
Master
Plan
(IMP)
Integrated
Master
Schedule
(IMS)
Measures
of
Effec1veness
Measures
of
Performance
Technical
Performance
Measures
JROC
Key
Performance
Parameters
Program
Specific
Key
Performance
Parameters
Technical
Performance
Measures
Status
Earned
Value
Management
System
Performance
Measurement
Baseline
SOO
ConOps
WBS
5

6. Value of the IMP/IMS to the PM†
• Systematic approach to program planning,
scheduling, and execution.
• Provides a tool for improved day-to-day
program execution and for on going insight
into program status by both government
program office personnel and contractor
personnel.
• Helps in the development and support “what-
if” exercises and identifies and assesses
candidate problem workarounds.
• Focuses and strengthens the government-
contractor team.
†Integrated
Master
Plan
and
Master
Schedule
Implementa4on
and
Use
Guide,
V0.9,
21
Oct
2005
6

7. Drivers of Technical Performance Measures
Start with Mission Need
MoE
KPP
MoP
TPM
Mission
Need
Acquirer
Defines
the
Needs
and
Capabili1es
in
terms
of
Opera1onal
Scenarios
Contractor
Defines
Physical
Solu1ons
that
meet
the
needs
of
the
Government
Opera)onal
measures
of
success
related
to
the
achievement
of
the
mission
or
opera)onal
objec)ve
being
evaluated.
Measures
that
characterize
physical
or
func)onal
aCributes
rela)ng
to
the
system
opera)on.
Measures
used
to
assess
design
progress,
compliance
to
performance
requirements,
and
technical
risks.
Government
Contractor
7
The
CPM-­‐200C
Presenta1on
has
the
background
on
MoE’s,
MoP’s,
and
KPP’s

8. Key Performance Parameters (KPP)
• KPP’s are performance attributes of a system
considered critical to the development of an
effective military capability.
• JROC mandatory KPPs – JCIDS Manual,
§B-A-2, 19 Jan 2012
8
1. Force
Protec)on
(FP-­‐KPP)
–
protec1on
of
personnel
by
preven1ng
or
mi1ga1ng
hos1le
ac1ons
against
friendly
personnel,
military
and
civilian.
2. Survivability
–
speed,
maneuverability,
detectability,
and
countermeasures
that
reduce
a
system’s
likelihood
of
being
engaged
by
hos1le
fire
3. Sustainment
–
provide
a
system
with
op1mal
Availability,
Reliability,
and
Ownership
Cost
to
the
warfighter
4. Net-­‐Ready
(NR-­‐KPP)
–
verifiable
performance
measures
and
associated
metrics
required
to
evaluate
the
1mely,
accurate,
and
complete
exchange
and
use
of
informa1on
to
sa1sfy
informa1on
needs
for
a
given
capability.
5. Training
–
ensure
training
requirements
are
properly
addressed
6. Energy
–
op1mize
fuel
and
electric
power

9. PUTTING THE PRINCIPLES
INTO PRACTICE
How do we demonstrate that using TPMs can provide better status
information to the Program Manager
9

10. Tactical Situation
Analysis System
10
Command
and
Data
Center
Mobile
Sensors
UAV
with
Airborne
Sensors
Ground
Based
Sensors

11. TSAS MoE, MoP, and KPPs
• TSASMOE.4 Transport Radius
– Performance Requirement Statement:
– The TSAS shall be transportable within a 3,000
mile radius via a C-17/C-141/C-5 package.
• Refines Higher-Level Requirement:
– JROCKPP.3 SUST Sustainment
– JROCKPP.6 ENGY Energy
• Refined By Lower-Level Requirements:
– TSASMOP.10 Transport Volume
– TSASMOP.11 Transport Weight
11

12. EO/IR Weight TPM
AvTPM.7 EO/IR Weight
• Performance Requirement Statement:
– The UAV EO/IR weight shall be less than or
equal to 100 pounds.
• Refines Higher-Level Requirement:
– UAVMOP.10 Sensor Weight Capacity
• Specifies:
– Component: UAV.1.4.14 EO/IR Electro-
Optical / Infrared
12

13. Tier 0 IMP/IMS for TSAS Avionics
13

14. Tier 1 IMS for TSAS Avionics
The
AVTPM.7
TPM
starts
with
the
avionics
components.
The
WP
is
Done
when
the
Systems
Engineers
complete
their
weight
alloca1on
to
each
sensor
subsystem.
14
Other
TPMs
are
used
to
assess
the
comple1on
of
the
WP.
Weight
alone
is
used
for
this
workshop.

15. WORKSHOP EXERCISE
15

16. The Principles of the Workshop Exercise
• Show the traditional ways of calculating
BCWP without being informed by
Technical Performance Measures (TPM).
– Option A compares traditional CAM
performance level with the TPM plan
• Using EO/IR weight as a TPM, calculate
earned value from TPM data
– Option B should drive BCWP from the actual
technical performance
16

17. Using Weight as a TPM for the EO/IR
• Weight is our TPM for our EO/IR sensor
suite
– Threshold (Requirement)
– Objective (Goal)
– Upper and Lower Control Limits
– Plan
– Current Value
17

18. Option A: Comparing traditional CAM EVM
assessment without TPM/QBD
Monthly
BCWPs
are
Determined
by
CAM
EVTs
Month
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sept
Oct
Nov
Dec
Upper
Control
Limit
125
125
125
125
125
125
125
125
125
125
125
125
Requirement
Weight
100
100
100
100
100
100
100
100
100
100
100
100
Planned
Weight
120
120
115
115
110
110
105
105
100
100
100
100
Actual
Weight
120
123
128
130
128
120
110
103
103
103
103
103
Percent
Actual/
Planned
Wt
100.0%
97.6%
89.8%
88.5%
85.9%
91.7%
95.5%
101.9%
97.1%
97.1%
97.1%
97.1%
BCWS
$71.2
$63.0
$65.5
$73.3
$68.6
$65.5
$76.4
$65.5
$80.6
$71.8
$67.0
$68.6
BCWScum
$71.2
$134.2
$199.7
$273.0
$341.6
$407.1
$483.5
$549.0
$629.7
$701.4
$768.5
$837.1
ACWP
$71.2
$63.0
$65.5
$78.0
$78.0
$78.0
$76.4
$65.5
$80.6
$71.8
$67.0
$68.6
ACWPcum
$71.2
$134.2
$199.7
$277.7
$355.7
$433.7
$510.1
$575.6
$656.2
$728.0
$795.0
$863.7
BCWP
$71.2
$63.0
$65.5
$73.3
$68.6
$65.5
$76.4
$65.5
$80.6
$71.8
$67.0
$68.6
BCWPcum
$71.2
$134.2
$199.7
$273.0
$341.6
$407.1
$483.5
$549.0
$629.7
$701.4
$768.5
$837.1
SV
$.00
$.00
$.00
$.00
$.00
$.00
$.00
$.00
$.00
$.00
$.00
$.00
SVcum
$.00
$.00
$.00
$.00
$.00
$.00
$.00
$.00
$.00
$.00
$.00
$.00
CV
$.00
$.00
$.00
($4.72)
($9.36)
($12.48)
$.00
$.00
$.00
$.00
$.00
$.00
CVcum
$.00
$.00
$.00
($4.72)
($14.08)
($26.56)
($26.56)
($26.56)
($26.56)
($26.56)
($26.56)
($26.56)
CPIi
(BCWP/ACWP)
1.00
1.00
1.00
0.94
0.88
0.84
1.00
1.00
1.00
1.00
1.00
1.00
CPIcum
1.00
1.00
1.00
0.98
0.96
0.94
0.95
0.95
0.96
0.96
0.97
0.97
SPIi
(BCWP/BCWS)
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
SPIcum
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
18

19. Option A: EVM Performance
0.40
0.50
0.60
0.70
0.80
0.90
1.00
1.10
1.20
$.0
$100.0
$200.0
$300.0
$400.0
$500.0
$600.0
$700.0
$800.0
$900.0
$1000.0
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sept
Oct
Nov
Dec
CPI/SPI
Thousands
BCWScum
ACWPcum
BCWPcum
CPIcum
SPIcum
19

20. Option A: TPM Performance
120
120
115
115
110
110
105
105
100
100
100
100
120
123
128
130
128
120
110
103
103
103
103
103
0.40
0.50
0.60
0.70
0.80
0.90
1.00
1.10
1.20
90
95
100
105
110
115
120
125
130
135
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sept
Oct
Nov
Dec
CPI/SPI
Pounds
Upper
Control
Limit
Planned
Weight
Actual
Weight
CPIcum
SPIcum
20

21. Workshop Exercise
• Using BCWS, ACWP, upper control weight
or percent planned to actual weight,
calculate
– BCWP,
– BCWPcum
– CPIcum and SPIcum
21

22. Option B: Use TPMs to Determine Performance
Month
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sept
Oct
Nov
Dec
Upper
Control
Limit
125
125
125
125
125
125
125
125
125
125
125
125
Requirement
Weight
100
100
100
100
100
100
100
100
100
100
100
100
Planned
Weight
120
120
115
115
110
110
105
105
100
100
100
100
Actual
Weight
120
123
128
130
128
120
110
103
103
103
103
103
Percent
Actual/
Planned
Wt
100.0%
102.5%
111.3%
113.0%
116.4%
109.1%
104.8%
98.1%
103.0%
103.0%
103.0%
103.0%
BCWS
$71.2
$63.0
$65.5
$73.3
$68.6
$65.5
$76.4
$65.5
$80.6
$71.8
$67.0
$68.6
BCWScum
$71.2
$134.2
$199.7
$273.0
$341.6
$407.1
$483.5
$549.0
$629.7
$701.4
$768.5
$837.1
ACWP
$71.2
$63.0
$65.5
$78.0
$78.0
$78.0
$76.4
$65.5
$80.6
$71.8
$67.0
$68.6
ACWPcum
$71.2
$134.2
$199.7
$277.7
$355.7
$433.7
$510.1
$575.6
$656.2
$728.0
$795.0
$863.7
BCWP
BCWPcum
SV
SVcum
CV
CVcum
CPIi
(BCWP/ACWP)
CPIcum
SPIi
(BCWP/BCWS)
SPIcum
22

23. Let’s Go To Work
23

24. Some Ideas To Better Connect TPMs to
Cost and Schedule
• Strengthen the PARCA’s IPMR Guide
• Provide input to the NDIA guides to
include TPMs as performance measures
• Change DI-MGMT-81861 to mandate an
IMP for ACAT1 programs at next update
cycle
• Other ideas?
24

25. Summary of Our Results
• Showed the importance of the IMP in
determining what Done looks like
– Provides the full picture of required effectiveness,
performance, KPPs and eventually to TPMs
– Allows one to see how technical progress will be
achieved over time
• Described and demonstrated one possible
way to better connect technical performance
to cost and schedule performance.
• Had participants calculate BCWP from the
TPM.
25

26. Tom
Coonce
Glen
Alleman
Rick
Price
26

27. 27

28. Appendix
28

29. Option B: Calculate EV using TPM
(This is one way to do this; there are others)
• Final EO/IR weight is 100 lbs
• TPM Physical % Complete measured
each period of performance for Plan
versus Actual weight
• Once we reach 100% of planned weight
for a given month
– Then we can take all the BCWS for the month
– Plus BCWP deficits from prior months where
planned weight was less than actual weight
29

30. One Way to Determine Performance from TPMs
• PCi = 1 – (WtPi – WtAi) / WtPi
• IF PCi < 1
– THEN BCWPi = PCi X BCWSi
– ELSE BCWPi = BCWSi + (SVcum i-1)
ENDIF
• CPIi = BCWPi /ACWPi
• SPIi = BCWPi / BCWSi
These
calcula1on
need
to
be
stated
in
the
Work
Instruc1ons
and
documented
in
the
Quan1fiable
Backup
Data
30

31. Option B: Use Plan vs. Actual Weight
Percent to Determine Performance
Month
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sept
Oct
Nov
Dec
Upper
Control
Limit
125
125
125
125
125
125
125
125
125
125
125
125
Requirement
Weight
100
100
100
100
100
100
100
100
100
100
100
100
Planned
Weight
120
120
115
115
110
110
105
105
100
100
100
100
Actual
Weight
120
123
128
130
128
120
110
103
103
103
103
103
Percent
Actual/
Planned
Wt
100.0%
102.5%
111.3%
113.0%
116.4%
109.1%
104.8%
98.1%
103.0%
103.0%
103.0%
103.0%
BCWS
$71.2
$63.0
$65.5
$73.3
$68.6
$65.5
$76.4
$65.5
$80.6
$71.8
$67.0
$68.6
BCWScum
$71.2
$134.2
$199.7
$273.0
$341.6
$407.1
$483.5
$549.0
$629.7
$701.4
$768.5
$837.1
ACWP
$71.2
$63.0
$65.5
$78.0
$78.0
$78.0
$76.4
$65.5
$80.6
$71.8
$67.0
$68.6
ACWPcum
$71.2
$134.2
$199.7
$277.7
$355.7
$433.7
$510.1
$575.6
$656.2
$728.0
$795.0
$863.7
BCWP
$71.2
$61.5
$58.9
$64.8
$59.0
$60.1
$72.9
$100.8
$78.3
$69.7
$65.1
$66.6
BCWPcum
$71.2
$132.6
$191.5
$256.3
$315.3
$375.4
$448.3
$549.0
$627.3
$697.0
$762.1
$828.7
SV
$.00
($1.54)
($6.65)
($8.46)
($9.65)
($5.46)
($3.47)
$35.23
($2.35)
($2.09)
($1.95)
($2.00)
SVcum
$.00
($1.54)
($8.19)
($16.65)
($26.30)
($31.76)
($35.23)
$.00
($2.35)
($4.44)
($6.39)
($8.39)
CV
$.00
($1.54)
($6.65)
($13.18)
($19.01)
($17.94)
($3.47)
$35.23
($2.35)
($2.09)
($1.95)
($2.00)
CVcum
$.00
($1.54)
($8.19)
($21.37)
($40.38)
($58.32)
($61.79)
($26.56)
($28.91)
($31.00)
($32.95)
($34.95)
CPIi
(BCWP/ACWP)
1.00
0.98
0.90
0.83
0.76
0.77
0.95
1.54
0.97
0.97
0.97
0.97
CPIcum
1.00
0.99
0.96
0.92
0.89
0.87
0.88
0.95
0.96
0.96
0.96
0.96
SPIi
(BCWP/BCWS)
1.00
0.98
0.90
0.88
0.86
0.92
0.95
1.54
0.97
0.97
0.97
0.97
SPIcum
1.00
0.99
0.96
0.94
0.92
0.92
0.93
1.00
1.00
0.99
0.99
0.99
31

32. Option B: EVM Performance
0.40
0.50
0.60
0.70
0.80
0.90
1.00
1.10
1.20
$.0
$100.0
$200.0
$300.0
$400.0
$500.0
$600.0
$700.0
$800.0
$900.0
$1000.0
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sept
Oct
Nov
Dec
CPI/SPI
Thousands
BCWScum
ACWPcum
BCWPcum
CPIcum
SPIcum
32

33. Option B: TPM Performance
0.40
0.50
0.60
0.70
0.80
0.90
1.00
1.10
1.20
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
110.0%
120.0%
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sept
Oct
Nov
Dec
CPI/SPI
TPM
%
of
Plan
Percent
Actual/Planned
Wt
CPIcum
SPIcum
33

34. References
• Office of the Undersecretary of Defense Acquisition Technology and
Logistics “Interoperability/Systems Engineering and Acquisition
Resource & Analysis / Acquisition Management,” Version 0.9,
October 21, 2005.
• “Integrated Master Plan (IMP) & Integrated Master Schedule (IMS)
Elementary Textbook,” Presented By Integrated Systems, AGS &
BMS Program Cost Scheduling & Control R. Scordino / F. Fontanet,
IPM 2002 Fall Conference, Nov 2002.
• The Integrated Project Management Handbook, Dayton Aerospace,
Inc., 8 February 2002, prepared for Office of the Undersecretary of
Defense Acquisition Technology and Logistics Interoperability/
Systems Engineering and Acquisition Resource & Analysis/
Acquisition Management.
34

35. References
• “Integrated Life Cycle Management,” Air Force Instruction
63-101/20-101, 07 March 2013.
• “DoD Integrated Master Plan and Integrated Master Schedule
Preparation and Use Guide,” 21 Oct 2005.
• “Air Force Integrated Master Schedule (IMS) Assessment Guide,
Version 3,” 7 June 2012.
• Planning and Scheduling Excellence Guide, 22 June 2012.
• AFMC (Air Force Materiel Command). “Integrated Master Plan and
Schedule Guide,” AFMC Pamphlet 63-5. Wright-Patterson AFB,
Ohio: 2005.
• PARCA-EVM Assessment Study, Final Report Contract #
HQ0034-12A0011-0001, Paul Solomon and David D. Walden,
August 13, 2012.
• “Basing Earned Value on Technical Performance,” Paul Solomon,
Cross Talk, January/February, 2003, pp. 25-28.
35