SlideShare uma empresa Scribd logo
1 de 63
Baixar para ler offline
CPM–500 EVM/SE/TPM Integration




   CPM–500–B/C/F : Integrating Systems
Engineering with Earned Value Management
   Lesson 3 CPM–500F: Technical Performance Measures


                        Glen B. Alleman
                              Lewis & Fowler
                          galleman@lewisandfowler.com
                                 (303) 241 9633




                                   June 2010
                                 Naples, Florida
          Professional Education Program (Training Track) presented by
                PMI–College of Performance Management faculty

                                Rights Reserved                                                    1/63
CPM–500 EVM/SE/TPM Integration




                               The Purpose Of This Series

 This is the last “connected” lesson in a 3–part series that explores the
  integration of System Engineering (SE) and Earned Value Management
  (EVM).
 You don’t need to attend all three lessons – but it helps in Connecting
  The Dots
 Why have this series?
    – Often, project control specialists see the planning of the cost and
      schedule baselines as an isolated activity disconnected from the technical
      or engineering aspects.
    – This type of thinking significantly impacts development of the baseline
      which results in poor maintenance and eventual irrelevance of the
      baseline
    – We claim “victory” if after any one of these lessons, you can recognize the
      importance of integrating these disciplines such that he/she will advocate
      greater cooperation between the engineering and project control
      elements of his/her respective organization


                                  Rights Reserved                                       2/63
CPM–500 EVM/SE/TPM Integration




                                  Lesson 3 Objectives

In this lesson we will:
 Understand how to establish credibility for
    Technical Performance Measures by using Earned
    Value and Systems Engineering to measure
    progress
 Establish the processes of defining and measuring
    technical performance
 Establish the role of the technical baseline in
    program management
 Learn how TPM’s are integrated into planning and
    execution at the control account level through
    examples and a hands on exercise
                      Rights Reserved                                3/63
CPM–500 EVM/SE/TPM Integration


                    Can Earned Value Alone Get Us
                              To Our Destination?
 How do we increase visibility into the program’s performance?
 How do we reduce cycle time to deliver the product?
 How do we foster accountability?
 How do we reduce risk?
 How do we start this journey to success?




         Increasing the Probability of Success means we have to
               Connect The Dots to Reach Our Destination
                               Rights Reserved                                   4/63
CPM–500 EVM/SE/TPM Integration




                                                  To Achieve Success …
We Need to …




©gapingvoid ltd www.gapingvoidgallery.com   Rights Reserved                             5/63
CPM–500 EVM/SE/TPM Integration




Learnings from Lesson 1 and 2


   How cost and schedule (EV) are
   connected to the Performance
      Measurement Baseline


              Rights Reserved                             6/63
CPM–500 EVM/SE/TPM Integration


                  Increasing the Probability of
                    Program Success Means …
Building A Credible Performance Measurement Baseline


                  Risk                   Cost

          IMP/IMS          PMB              SOW

                  WBS                    TPM


         This is actually harder than it looks!
                       Rights Reserved                                    7/63
CPM–500 EVM/SE/TPM Integration




    Doing This Starts With Some Guidance
Systems engineering uses technical performance
measurements to balance cost, schedule, and
performance throughout the life cycle. Technical
performance measurements compare actual versus
planned technical development and design. They
also report the degree to which system requirements
are met in terms of performance, cost, schedule, and
progress in implementing risk handling. Performance
metrics are traceable to user–defined capabilities.
― Defense Acquisition Guide
   (https://dag.dau.mil/Pages/Default.aspx)
  In The End ― It’s All About Systems Engineering
                     Rights Reserved                               8/63
CPM–500 EVM/SE/TPM Integration

                      Guidance for Measures of
               Effectiveness, Performance, and
                        Technical Performance
Our starting point is not EVM, it’s Systems Engineering
 MOE’s are an essential part of Systems Engineering,
  guided by IEEE 1220 and EIA 632.
 System’s Engineers drive the content of all
  measurement items, customer or supplier.




                         Rights Reserved                               9/63
CPM–500 EVM/SE/TPM Integration


                Just A Reminder Of The …
         Primary Elements of Earned Value

                                   Cost


Funding margin                                         Schedule margin for
   for under        Over cost or      Over cost or     over target baseline
 performance           under             over                 (OTB)
                    performance        schedule


                                Over
        Technical            schedule or
      Performance               under                Schedule
                             performing


                         Schedule margin for
                        underperformance or
                         schedule extension
                            Rights Reserved                                            10/63
CPM–500 EVM/SE/TPM Integration

               Previous Approaches Using EV
                  Are Mostly Unsuccessful In
                           Connecting These
 Traditional approaches to program management
  are retrospective
  – Cost and schedule of Earned Value
  – Risk Management
  – Systems Engineering
 Reporting past performance
  – Sometimes 30 to 60 days old
  – Variances are reporting beyond the widow of
    opportunity for correction
                       Rights Reserved                            11/63
CPM–500 EVM/SE/TPM Integration


                      It’s All Been Said Before.
                    We Just Weren’t Listening…
… the basic tenets of the process are the need for
seamless management tools, that support an integrated
approach … and “proactive identification and
management of risk” for critical cost, schedule, and
technical performance parameters.
― Secretary of Defense, Perry memo, May 1995
            Why Is This Hard To Understand?
 We seem to be focused on EV reporting, not the use
  of EV to manage the program.
 Getting the CPR out the door is the end of Program
  Planning and Control’s efforts, not the beginning.
                       Rights Reserved                             12/63
CPM–500 EVM/SE/TPM Integration


                 The Gap Seems To Start With A
                             Common Problem
 Many Times, The Information from Cost, Schedule, Techncial
Performance, and Risk Management Gets Mixed Up When We
                 Try to Put Them Together




                         Rights Reserved                              13/63
CPM–500 EVM/SE/TPM Integration


                When We Put The Cart Before
                  The Horse, We Discover …
 EVM really doesn’t do its job effectively
 Most of the time EV has no measure of quality
  or compliance with technical requirements.
 EV measures progress to plan in units of
  “money,” not tangible value to the customer
 Most EV System Descriptions fail to connect the
  dots between cost, schedule, and technical
  performance – even though instructed to do so
  in the official guidance
                     Rights Reserved                            14/63
CPM–500 EVM/SE/TPM Integration




            The NDIA EVM Intent Guide Says




        Notice the inclusion of Technical along with
                     Cost and Schedule
That’s the next step is generating Value from Earned Value
  EV MUST include the Technical Performance Measures
                        Rights Reserved
CPM–500 EVM/SE/TPM Integration




   Back To Our Technical
   Performance Measures

Technical Performance Measures do what
                they say,
   Measure the Technical Performance
of the product or service produced by the
                program
                Rights Reserved                             16/63
CPM–500 EVM/SE/TPM Integration


                                  What’s Our Motivation for
                                   “Connecting the Dots?”
TPMs are a set of measures that provide the supplier and
acquirer with insight into progress to plan of the technical
solution, the associated risks, and emerging issues.
Technical Performance Measures …
 Provide program management with information
  to make better decisions
 Increase the probability of delivering a solution
  that meets both the requirements and mission
  need
We’ve been talking about this since as early as 1984, in Technical Performance
Measurement Handbook, Defense Systems Management College, Fort Belvoir, VA 22060
                                  Rights Reserved                                      17/63
CPM–500 EVM/SE/TPM Integration




                                Measure of Effectiveness (MoE)

    The operational measures of success that are closely
    related to the achievements of the mission or operational
    objectives evaluated in the operational environment,
    under a specific set of condition

    Measures of Effectiveness …
     Are stated by the buyer in units meaningful to
      the buyer
     Focus on capabilities independent of any
      technical implementation
                              MoE Belong to the End User
“Technical Measurement,” INCOSE–TP–2003–020–01 Rights Reserved                            18/63
CPM–500 EVM/SE/TPM Integration




                                  Measure of Performance (MoP)

    Measures that characterize physical or functional
    attributes relating to the system operation, measured or
    estimated under specific conditions
    Measures of Performance are …
     Attributes that assure the system has the
      capability to perform
     Assessment of system to assure it meets design
      requirements necessary to satisfy the MOE
    MoP’s belong to the Program – Developed by the Systems
     Engineer, Measured By CAMs, and Analyzed by PP&C
“Technical Measurement,” INCOSE–TP–2003–020–01 Rights Reserved                            19/63
CPM–500 EVM/SE/TPM Integration




                       Key Performance Parameters (KPP)

    Represent the capabilities and characteristics so
    significant that failure to meet them can be cause for
    reevaluation, reassessing, or termination of the program
    Key Performance Parameters …
     have a threshold or objective value
     Characterize the major drivers of performance
     Are considered Critical to Customer (CTC)

        The acquired defines the KPPs during the operational
       concept development – KPPs say what DONE looks like
“Technical Measurement,” INCOSE–TP–2003–020–01 Rights Reserved                            20/63
CPM–500 EVM/SE/TPM Integration




            Technical Performance Measures (TPM)

    Attributes that determine how well a system or system
    element is satisfying or expected to satisfy a technical
    requirement or goal
    Technical Performance Measures …
     Assess design progress
     Define compliance to performance requirements
     Indentify technical risk
     Are limited to critical thresholds
     Include projected performance
“Technical Measurement,” INCOSE–TP–2003–020–01 Rights Reserved                            21/63
CPM–500 EVM/SE/TPM Integration




                                  Dependencies Between Measures
        Stakeholders Define Needs and Capabilities                          Suppliers Define Physical Solutions that
                 in terms of Operational Scenarios                          meet the needs of the Stakeholders


                                                                                      KPP



            Mission
                                             MoE                                     MoP                            TPM
             Need
                                       Operational                              Measures that                 Measures used to
                                       measures of                              characterize                  assess design
                                       success related to                       physical or                   progress,
                                       the achievement                          functional                    compliance to
                                       of the mission or                        attributes relating           performance
                                       operational                              to the system                 requirements, and
                                       objective being                          operation                     technical risks
                                       evaluated.


“Coming to Grips with Measures of Effectiveness,” N. Sproles,
Systems Engineering, Volume 3, Number 1, pp. 50–58
                                                              Rights Reserved                                                  22/63
CPM–500 EVM/SE/TPM Integration




                       “Candidates” for Technical Measures
                                      Concept Description
                                 Useful Life
     Physical Size and Stability Weight
                                 Volumetric capacity
                                              Accuracy
         Functional Correctness
                                              Power performance
                                           Supportability
                                           Maintainability
                         All the “ilities”
                                           Dependability
                                           Reliability
                                           Reliability = Mean Time Failure
                                             Utilization
                                  Efficiency Response time
                                             Throughput
          Suitability for Purpose Readiness
                                    Rights Reserved                                         23/63
INCOSE Systems Engineering Handbook
CPM–500 EVM/SE/TPM Integration




                               “Measures” of Technical Measures
              Attribute Description
                         Measured technical progress or estimate of
      Achieved to Date
                         progress
                         Value of a technical parameter that is predicted to
      Current Estimate
                         be achieved
                         Point in time when an evaluation of a measure is
             Milestone
                         accomplished
         Planned Value Predicted value of the technical parameter
  Planned Performance Profile representing the project time phased
                 Profile demonstration of a technical parameter
        Tolerance Band Management alert limits
                       Threshold Limiting acceptable value of a technical parameter
                                       Demonstrated technical variance
                        Variances
INCOSE Systems Engineering Handbook    Predicted technical variance
                                              Rights Reserved                                24/63
CPM–500 EVM/SE/TPM Integration




                                                    A Familiar Graphic of TPMs
                          TPM
                                                                     Upper Limit
                                     Planned Profile
                                                                                          Current Estimate

                                       Planned Value
Mean To Between Failure




                                                                                             Threshold


                                        Variance                         Lower Limit



                                      Achieved to Date


                                                                                       Milestones
                          Time = Program Maturity

                                                       Rights Reserved                                           25/63
CPM–500 EVM/SE/TPM Integration




A Simple Method of Assembling the TPMs

   MOE / MOP            KPP / TPM             Risks

                         Define the         Assess the
 Select Technical
                          planned         impact on Risk
  Performance
                        progress for        from this
   Parameters
                         each TPM            progress




   Parameters             Progress             Risk

  Weight        XXXX
  Speed         XXXX
  MTBF          XXXX
  Loiter Time   XXXX



                        Rights Reserved                               26/63
CPM–500 EVM/SE/TPM Integration


 TPMs from an Actual Program
James Webb Space Telescope




    Rights Reserved                            27/63
CPM–500 EVM/SE/TPM Integration


TPMs from an Actual Program
   Chandra X–Ray Telescope




    Rights Reserved                            28/63
CPM–500 EVM/SE/TPM Integration




  What Does A Real Technical
Performance Measure Look Like?


Not that bagels are not
interesting in Lesson 1 and
2, but let’s get ready to look
at a flying machine.
                    Rights Reserved                            29/63
CPM–500 EVM/SE/TPM Integration




  The WBS for a UAV           TPMs Start With The WBS
1.1     Air Vehicle
1.1.1   Sensor Platform
1.1.2
1.1.2   Airframe
        Airframe
1.1.3   Propulsion
1.1.4   On Board Comm
1.1.5   Auxiliary Equipment
1.1.6   Survivability
        Modules
1.1.7   Electronic Warfare
        Module
1.1.8   On Board
        Application &
        System SW
1.3     Mission Control /
        Ground Station SW
1.3.1   Signal Processing
        SW
1.3.2   Station Display
1.3.3   Operating System
1.3.4   ROE Simulations
1.3.5   Mission Commands
                              Rights Reserved                            30/63
CPM–500 EVM/SE/TPM Integration


           What Do We Need To Know About
                This Program Through TPMs
 What WBS elements represent the TPMs?
 What Work Packages produce these WBS
  elements?
 Where do these Work Packages live in the IMS?
 What are the Earned Value baseline values for
  these Work Packages?
 How are going to measure all these variables?
 What does the curve look like for these
  measurements?
                    Rights Reserved                            31/63
CPM–500 EVM/SE/TPM Integration




                                               Let’s Connect The Dots
Technical and Programmatic
Risks Connected to the WBS
and IMS                                                          BCWS at the Work
                                                                 Package, rolled to the
                                                                 Control Account
IMS contains all
the Work
Packages, BCWS,                         Risk         Cost
Risk mitigation                                                         Named
plans, and rolls to           IMP/IMS          PMB         SOW          Deliverables
the Integrated                                                          defined in the WBS
Master Plan to                      WBS              TPM
measure
increasing maturity                                              TPMs attached to each
                                                                 critical deliverables in the
                                                                 WBS and identified in
The Products and                                                 each Work Package in the
Processes that produce                                           IMS, used to assess
them in a “well structured”                                      maturity in the IMP
decomposition in the WBS

                                         Rights Reserved                                       32/63
CPM–500 EVM/SE/TPM Integration



                                                            Verifying Each TPM
                                              Evidence that we’re in compliance
                                              With our submitted ROM what are the values we need to get
      Do we know what we promised to
CA                                            through Integrated Baseline Review «how do we measure
      deliver, now that we’ve won?
                                              weight for each program event»
                                              The contributors to the vehicle weight are confirmed and the
      Can we proceed into preliminary
SFR                                           upper limits defined in the product architecture and
      design?
                                              requirements flow down database (DOORS) into a model
      Can we proceed into the System          Do we know all drivers of vehicle weight? Can we bound their
SRR   Development and Demonstration           upper limits? Can the subsystem owners be successful within
      (SDD) phase                             these constraints uses a high fidelity model?
      Can we start detailed design, and       Does each subsystem designer have the target component
      meet the stated performance             weight target and have some confidence they can stay below
PDR   requirements                            the upper bound? Can this be verified in some tangible way?
      within cost, schedule,                  Either through prior examples or a lab model?
      risk, and other constraints?
      Can the system proceed to               Do we know all we need to know to start the fabrication of
      fabrication, demonstration, and test,   the first articles of the flight vehicle. Some type of example,
CDR
      with the within cost, schedule, risk,   maybe a prototype is used to verify we’re inside the lines
      and other system constraints.
      Can the system ready to                 Does the assembled vehicle fall within the weight range limits
TRR                                            Rights Reserved
      proceed into formal test?               for 1st flight – will this thing get off the ground?      33/63
CPM–500 EVM/SE/TPM Integration




                                                         TPM Trends & Responses
  Design Model
ROM in Proposal                                                    Detailed Design Model
                                                                             Bench Scale Model Measurement
    Technical Performance Measure




                                    28kg                                             Prototype Measurement
            Vehicle Weight




                                                                                                   Flight 1st Article
                                    26kg

                                    25kg


                                    23kg



                                           CA      SFR     SRR         PDR     CDR         TRR

                                     EV Taken, planned values met, tolerances kept, etc.
                                                         Rights Reserved                   Dr. Falk Chart – modified 34/63
CPM–500 EVM/SE/TPM Integration


               The Assessment Of Weight As A
                           Function Of Time
 At Contract Award there is a Proposal grade estimate of
  vehicle weight
 At System Functional Review, the Concept of
  Operations is validated for the weight
 At System Requirements Review the weight targets are
  flowed down to the subsystems components
 At PDR the CAD model starts the verification process
 At CDR actual measurements are needed to verify all
  models
 At Test Readiness Review we need to know how much
  fuel to put on board for the 1st flight test
                        Rights Reserved                              35/63
CPM–500 EVM/SE/TPM Integration




  The WBS for a UAV                                        Airframe Weight TPM
1.1      Air Vehicle                                         The planned weight is
1.1.1    Sensor Platform                                     25kg. The actual weight is
1.1.2    Airframe                                            25.5kg.
1.1.2   Airframe
                                                             Close to plan! So we are
                                                             doing okay, right?

                     CA           SFR            SRR             PDR             CDR              TRR
Planned Value      28.0kg        27.0kg         26.0kg          25.0kg           24.0kg          23.0kg

Actual Value       30.4kg        29.0kg         27.5kg          25.5kg

                  Moderate         Low           Low         Very Low (less
Assessed Risk
                  >2.0kg off    1–2 kg off    1–2 kg off     than 1.0 kg
   to TRR
                    target        target        target       off target)
                                                               Program–           Actual         Actual
                                               Program–
  Planned        “Similar to”                                unique design    measurement     measurement
                                  ROM        unique design
  Method          Estimate                                    model with      of bench–test   of prototype
                                                 model
                                                             validated data   components        airframe
  Actual         “Similar to”
  Method          Estimate
                                  ROM            ROM             ROM           Here’s the Problem

                                              Rights Reserved                                                36/63
CPM–500 EVM/SE/TPM Integration


                          Is This A Problem?
                    You Bet’ya It’s A Problem!
 The measurement is close to the planned value,
 But the planned method of measurement is a
  program unique design model with validated
  data,
 But the actual method of measurement is a
  rough order of magnitude estimate,
 No improvement in fidelity since the System
  Functionality Review (SFR), and
 The TPM provides no new information – so
  we’re probably late and don’t know it yet.
                     Rights Reserved                            37/63
CPM–500 EVM/SE/TPM Integration




     Raison d'etre for Technical
        Performance Measures

The real purpose of
                                          Risk         Cost
Technical Performance
Measures is to reduce               IMP/IMS      PMB         SOW

Programmatic and                         WBS           TPM
Technical RISK
                  Rights Reserved                                           38/63
CPM–500 EVM/SE/TPM Integration




                                                                                                                                                        Buying Down Risk with TPMs
                                                                    Risk: CEV-037 - Loss of Critical Functions During Descent
                                                                                                                                                                                                                “Buying down” risk is
24

22

20
                                                                                Correlate the analytical model
                                                                    Conduct focus splinter review
                                               Develop analytical model to de
                                                                                                                                                                                                                 planned in the IMS.
                                                                                                                                                                                                                MoE, MoP, and KPP
                            Conduct Force and Moment Wind
18
                                                                     Conduct Block 1 w ind tunnel te
16                                                                                                                    Conduct w ind tunnel testing of
                                                                                            Conduct w ind tunnel testing of



                                                                                                                                                                                                                 defined in the work
14
                                                                                                                                 Flight Application of Spacecra
12




                                                                                                                                                                                                                 package for the critical
10                                                                                                                                                    CEV block 5 w ind tunnel testin

8

6

4

2
                                                                                                                          In-Flight development tests of
                                                                                                                                                              Damaged TPS flight test
                                                                                                                                                                                                                 measure – weight.
                                                                                                                                                                                                                If we can’t verify
0
                                             3.Jul.06




                                                                                                                                                                                                    1.Jul.11
     31.Mar.05




                 5.Oct.05




                                                                                 1.Jun.07




                                                                                                                                                                   1.Jan.10




                                                                                                                                                                                        16.Dec.10
                                                        15.Sep.06
                                  3.Apr.06




                                                                                                           1.Apr.08


                                                                                                                      1.Aug.08




                                                                                                                                           1.Apr.09




                       Planned Risk Level

                             Weight risk
                            reduced from
                                                                                                                                  Planned (Solid=Linked, Hollow =Unlinked, Filled=Complete)

                                                                                                                                          Weight confirmed
                                                                                                                                           ready to fly – it’s
                                                                                                                                                                                                                 we’ve succeeded, then
                            RED to Yellow                                                                                                 GREEN at this point
                                                                                                                                                                                                                 the risk did not get
                                                                                                                                                                                                                 reduced.
                                                                                                                                                                                                                The risk may have
                                                                                                                                                                                                                 gotten worse.

                                                                                                                                                                                    Rights Reserved                                                  39/63
CPM–500 EVM/SE/TPM Integration


                Increasing the Probability of
             Success with Risk Management
 Going outside the TPM
  limits always means
  cost and schedule
  impacts
 “Coloring Inside the
  Lines” means knowing
  the how to keep the
  program GREEN, or at
  least stay close to           So much for our strategy of winning
  GREEN                            through technical dominance
                    Rights Reserved                                       40/63
CPM–500 EVM/SE/TPM Integration


                                Connecting the EV Variables
             Integrating Cost, Schedulele, and Technical Performance
    Assures Program Management has the needed performance information to deliver
                      on‒time, on‒budget, and on‒specification

                                             =
                             Technical Performance Measures
                Cost                         +                   Schedule
                                 Conventional Earned Value

       Cost Baseline               Technical Performance           Schedule Baseline
 Master Schedule used to        Earned Value is diluted      Requirements are
  derive Basis of Estimate        by missing technical          decomposed into
  (BOE) not the other way         performance.                  physical deliverables.
  around.                        Earned Value is diluted      Deliverables are
 Probabilistic cost              by postponed features.        produced through Work
  estimating uses past           Earned Value is diluted       Packages.
  performance and cost            by non compliant quality.    Work Packages are
  risk modeling.                 All these dilutions           assigned to accountable
 Labor, Materiel, and            require adjustments to        manager.
  other direct costs              the Estimate at Complete     Work Packages are
  accounted for in Work           (EAC) and the To              sequenced to form the
  Packages                        Complete Performance          highest value stream
 Risk adjustments for all        Index (TCPI).                 with the lowest technical
  elements of cost.                    Rights Reserved
                                                                and programmatic risk.
                                                                                                41/63
CPM–500 EVM/SE/TPM Integration


                                                         TPM Checklist
         MoE                        MoP                             TPM
Traceable to needs,       Traceable to applicable      Traceable to applicable MoPs,
goals, objectives, and    MOEs, KPPs, system level     system element performance,
risks                     performance requirements,    requirements, objectives,
                          and risks                    risks, and WBS elements
Defined with associated   Focused on technical risks   Further decomposed,
KPPs                      and supports trades          budgeted, and allocated to
                          between alternative          lower level system elements in
                          solutions                    the WBS and IMS
Each MoE independent      Provided insight into        Assigned an owner, the CAM
from others               system performance           and Work Package Manager
Each MoE independent Decomposed, budgeted              Sources of measure identified
of technical any solution and allocated to system      and processes for generating
                          elements                     the measures defined.
Address the required      Assigned an “owner,” the     Integrated into the program’s
KPPs                      CAM and Technical            IMS as part of the exit criteria
                          Manager Rights Reserved      for the Work Package         42/63
CPM–500 EVM/SE/TPM Integration


                  Increasing the Probability of
                    Program Success Means …
Building A Credible Performance Measurement Baseline


                  Risk                  Cost

          IMP/IMS         PMB              SOW

                 WBS                    TPM


      Using the Check List – “Connect the Dots”
                      Rights Reserved                                   43/63
CPM–500 EVM/SE/TPM Integration




Rights Reserved                            44/63
CPM–500 EVM/SE/TPM Integration




            Backup Materials

Knowledge is of two kinds. We know a
 subject ourselves, or we know where
    we can find information on it
          — Samuel Johnson

                   Rights Reserved                              45/63
CPM–500 EVM/SE/TPM Integration




         Many of Sources for Connecting the Dots
OMB Circular A–11, Section 300             Interim Defense Acquisition Guidebook (DAG)
                                           6/15/09
GAO Report 06–250                          Systems Engineering Plan (SEP) Preparation Guide
                                           4/08
DoDI 5000.02, Operation of the Defense     WBS Handbook, Mil–HDBK–881A (WBS) 7/30/05
Acquisition System (POL) 12/08
Integrated Master Plan (IMP) & Integrated Guide for Integrating SE into DOD Acquisition
Master Schedule Preparation & Use Guide Contracts 12/06
(IMS) 10/21/05
Defense Acquisition Program Support        Guide to the Project Management Institute Body of
Methodology (DAPS) V2.0 3/20/09            Knowledge (PMBOK Guide®), 4th Edition
Standard for Application and               Capability Maturity Model Integration (CMMI®)
Management of the SE Process (IEEE
1220)
IEEE 1220: 6.8.1.5                         Processes for Engineering a System (ANSI/EIA–632)
NASA EVM Guide NPG 9501.3
                                         Rights Reserved                                       46/63
CPM–500 EVM/SE/TPM Integration

                            Office of Management and
                                             Budget

Circular No. A–11, Section 300
 Planning, Budgeting, Acquisition and Management
  of Capital Assets
 Section 300–5
  – Performance–based acquisition management
  – Based on EVMS standard
  – Measure progress towards milestones
     •   Cost
     •   Capability to meet specified requirements
     •   Timeliness
     •   Quality

                              Rights Reserved                                 47/63
CPM–500 EVM/SE/TPM Integration


                   Need: Accurate Performance
                                 Measurement
                                        Findings and
    GAO Report 06–250                Recommendations
Information Technology:        2. If EVM is not
Improve the Accuracy and         implemented effectively,
Reliability of Investment        decisions based on
Information                      inaccurate and potentially
                                 misleading information
                               3. Agencies not measuring
                                 actual versus expected
                                 performance in meeting
                                 IT performance goals.
                       Rights Reserved                                 48/63
CPM–500 EVM/SE/TPM Integration


                                               DOD Guides:
                                     Technical Performance

Department of Defense Guidelines for Technical Performance Measures
DoDI 5000.02, Operation of the Defense Acquisition System (POL) 12/08
Interim Defense Acquisition Guidebook (DAG) 6/15/09
Systems Engineering Plan (SEP) Preparation Guide 4/08
WBS Handbook, Mil–HDBK–881A (WBS) 7/30/05
Integrated Master Plan (IMP) & Integrated Master Schedule Preparation &
Use Guide (IMS) 10/21/05
Guide for Integrating SE into DOD Acquisition Contracts (Integ SE) 12/06
Defense Acquisition Program Support Methodology (DAPS) V2.0 3/20/09



                                Rights Reserved                                  49/63
CPM–500 EVM/SE/TPM Integration




         DoD: TPMs in Technical Baselines and Reviews




                                                                  Engineering
                                                                  Integrated
                                                   IMP/IMS


                                                                  Systems


                                                                                    DAPS
                                             WBS
                                DAG
                          POL




                                       SEP
DoD Policy or Guide
Technical Baselines:
IMP/IMS
Functional (SFR)                                                                 
Allocated (PDR)
Product (CDR)
Event driven timing                                                          
Success criteria of
technical review                                                             
Entry and exit criteria
for technical reviews                                                           
Assess technical
maturity                         Reserved
                                Rights                                             50/63
CPM–500 EVM/SE/TPM Integration




                           DoD: TPMs in Integrated Plans




                                                                        Engineering
                                                                        Integrated
                                                         IMP/IMS


                                                                        Systems


                                                                                          DAPS
                                                   WBS
                                 DAG
                           POL




                                           SEP
DoD Policy or Guide
Integrated SEP with:
IMP/IMS
TPMs                                                                                 
EVM
Integrated WBS with
Requirement
Specification                                                                        
Statement of Work
IMP/IMS/EVMS
Link risk management,
technical reviews, TPMs,                                                               
EVM, WBS, IMS                    Rights Reserved                                            51/63
CPM–500 EVM/SE/TPM Integration


              Guidance in Standards, Models,
               and Defense Acquisition Guide
 Processes for Engineering a System (ANSI/EIA–632)
 Standard for Application and Management of the SE
  Process (IEEE 1220)
 Capability Maturity Model Integration (CMMI®)
  – CMMI for Development, Version 1.2
  – CMMI for Acquisition, Version 1.2
  – Using CMMI to Improve Earned Value Management,
    2002
 Guide to the Project Management Institute Body of
  Knowledge (PMBOK Guide®), 4th Edition

                      Rights Reserved                             52/63
CPM–500 EVM/SE/TPM Integration

                                    Technical Performance
                                          Measures (TPM)
                             More Sources
IEEE 1220: 6.8.1.5,    EIA–632: Glossary             CMMI for Development
Performance–based                                    Requirements
progress measurement                                 Development

TPMs are key to        Predict future value of key   Specific Practice (SP) 3.3,
progressively assess   technical parameters of       Analyze Requirements
technical progress     the end system based on       Typical work product:
                       current assessments           TPMs
Establish dates for    Planned value profile is      Subpractice:
 – Checking progress   time–phased achievement       Identify TPMs that will be
 – Meeting full        projected                     tracked during
 conformance to        • Achievement to date         development
 requirements          • Technical milestone
                       where TPM evaluation is
                       reported
                               Rights Reserved                                       53/63
CPM–500 EVM/SE/TPM Integration




                                         PMBOK® Guide

 10.5.1.1 Project Management Plan
 Performance Measurement Baseline:
  – Typically integrates scope, schedule, and cost
    parameters of a project
  – May also include technical and quality parameters




                       Rights Reserved                               54/63
CPM–500 EVM/SE/TPM Integration




                                         PMBOK® Guide

 8.3.5.4 Work Performance Measurements
 Used to produce project activity metrics
 Evaluate actual progress as compared to
  planned progress
 Include, but are not limited to:
  – Planned vs. actual technical performance
  – Planned vs. actual schedule performance, and
  – Planned vs. actual cost performance.

                       Rights Reserved                               55/63
CPM–500 EVM/SE/TPM Integration




                              TPMs in DAG and DAPS

Defense Acquisition Guide
 Performance measurement of WBS elements, using
  objective measures:
  – Essential for EVM and Technical Assessment activities
 Use TPMs and Critical Technical Parameters (CTP) to
  report progress in achieving milestones
DAPS
 Use TPMs to determine whether % completion
  metrics accurately reflect quantitative technical
  progress and quality toward meeting Key
  Performance Parameters (KPP) and Critical
  Technical Parameters
                         Rights Reserved                               56/63
CPM–500 EVM/SE/TPM Integration




                                           TPMs in DAG

 Compare the actual versus planned technical
  development and design
 Report progress in the degree to which system
  performance requirements are met.
 Plan is defined in terms of:
  – Expected performance at specific points
     • Defined in the WBS and IMS
  – Methods of measurement at those points
  – Variation limits for corrective action.
                         Rights Reserved                              57/63
CPM–500 EVM/SE/TPM Integration




                                        PMBOK® Guide

 11.6.2.4 Technical Performance Measurement
 Compares technical accomplishments… to … project
  management plan’s schedule of technical
  achievement
 Requires definition of objective quantifiable
  measures of technical performance which can be
  used to compare actual results against targets.
 Might include weight, transaction times, number of
  delivered defects, storage capacity etc.
 Deviation, such as demonstrating more or less
  functionality than planned at a milestone…forecast
  degree of success in achieving the project’s scope.
                      Rights Reserved                               58/63
CPM–500 EVM/SE/TPM Integration




                                       CMMI–ACQ

 Acquisition Technical Management
 SP 1.3 Conduct Technical Reviews
 Typical supplier deliverables
 Progress reports and process, product, and
  service level measurements
 TPMs



                     Rights Reserved                            59/63
CPM–500 EVM/SE/TPM Integration


                                  SMS Shall:
            Monitor Progress Against the Plan
 4.2.12.2 Monitoring
 Contractor SHALL monitor progress against plan to
  validate, approve, and maintain each baseline and
  functional architecture
 4.2.12.2.2 Required Product Attributes
 Each documented assessment includes:
 TPMs, metrics
 Metrics and technical parameters for tracking that
  are critical indicators of technical progress and
  achievement
                       Rights Reserved                            60/63
CPM–500 EVM/SE/TPM Integration


                                     NASA EVM Guide:
                                 Technical Performance
• NASA EVM Guide NPG 9501.3
  – 4.5 Technical Performance Requirements (TPR): When
    TPRs are used,
  – appropriate and relevant metrics…
  – must be defined in the solicitation
  – Appendix A.7, 14.1 TPR
  • Compares:
     • Expected performance and
     • Physical characteristics
     • With contractually specified values.
  • Basis for reporting established milestones
  • Progress toward meeting technical requirements

                            Rights Reserved                            61/63
CPM–500 EVM/SE/TPM Integration


                                                      Derivation and Flow
                                                           Down of TPMs

       Document, Baseline,
           IMS, EVM                                           Parameter
 IMP, Functional Baseline                      Measures Of Effectiveness (MOE)                     

 IMP, WBS, Functional Baseline                 Measures Of Performance (MOP)                       
 IMP, Allocated Baseline                       Technical Performance Measure                       
                                                TPM Milestones And Planned
 IMS                                                                                               
                                                Values
 Work Packages                                 TPM% Complete Criteria                              

        See next chart for linkage of technical baselines to technical reviews

                                         Rights Reserved                                          62/63
CPM–500 EVM/SE/TPM Integration




                  Interesting Attributes of TPMs

   Achieved to Date (sounds like EV)
   Current Estimate (sounds like EAC/ETC)
   Milestone
   Planned (target) value (sounds like PV)
   Planned performance profile (sounds like a PMB)
   Tolerance band (sounds like reporting thresholds)
   Threshold (yep, just what we thought)
   Variance (sounds like variance!)

                        Rights Reserved                             63/63

Mais conteúdo relacionado

Mais procurados

Red Bend Software: Optimizing the User Experience with Over-the-Air Updates
Red Bend Software: Optimizing the User Experience with Over-the-Air UpdatesRed Bend Software: Optimizing the User Experience with Over-the-Air Updates
Red Bend Software: Optimizing the User Experience with Over-the-Air UpdatesRed Bend Software
 
Automotive embedded systems part5 v2
Automotive embedded systems part5 v2Automotive embedded systems part5 v2
Automotive embedded systems part5 v2Keroles karam khalil
 
An approach towards sotif with ansys medini analyze
An approach towards sotif with ansys medini analyzeAn approach towards sotif with ansys medini analyze
An approach towards sotif with ansys medini analyzeBernhard Kaiser
 
UDS: Vehicle Diagnostics in AUTOSAR Software Architecture
UDS: Vehicle Diagnostics in AUTOSAR Software Architecture UDS: Vehicle Diagnostics in AUTOSAR Software Architecture
UDS: Vehicle Diagnostics in AUTOSAR Software Architecture Embitel Technologies (I) PVT LTD
 
Arm programmer's model
Arm programmer's modelArm programmer's model
Arm programmer's modelv Kalairajan
 
An optimal and progressive algorithm for skyline queries slide
An optimal and progressive algorithm for skyline queries slideAn optimal and progressive algorithm for skyline queries slide
An optimal and progressive algorithm for skyline queries slideWooSung Choi
 
RISC Vs CISC, Harvard v/s Van Neumann
RISC Vs CISC, Harvard v/s Van NeumannRISC Vs CISC, Harvard v/s Van Neumann
RISC Vs CISC, Harvard v/s Van NeumannRavikumar Tiwari
 
Embedded systems - UNIT-1 - Mtech
Embedded systems - UNIT-1 - MtechEmbedded systems - UNIT-1 - Mtech
Embedded systems - UNIT-1 - Mtechsangeetha rakhi
 
Automotive embedded systems part6 v1
Automotive embedded systems part6 v1Automotive embedded systems part6 v1
Automotive embedded systems part6 v1Keroles karam khalil
 
Simd programming introduction
Simd programming introductionSimd programming introduction
Simd programming introductionChamp Yen
 
Trusted Platform Module (TPM)
Trusted Platform Module (TPM)Trusted Platform Module (TPM)
Trusted Platform Module (TPM)k33a
 
Autosar software component
Autosar software componentAutosar software component
Autosar software componentFarzad Sadeghi
 
19 Jun 2018 - Hazard Analysis and Functional Safety Compliance
19 Jun 2018 - Hazard Analysis and Functional Safety Compliance 19 Jun 2018 - Hazard Analysis and Functional Safety Compliance
19 Jun 2018 - Hazard Analysis and Functional Safety Compliance Intland Software GmbH
 
DAIS19: On the Performance of ARM TrustZone
DAIS19: On the Performance of ARM TrustZoneDAIS19: On the Performance of ARM TrustZone
DAIS19: On the Performance of ARM TrustZoneLEGATO project
 

Mais procurados (20)

Ec8791 lpc2148 timer unit
Ec8791 lpc2148 timer unitEc8791 lpc2148 timer unit
Ec8791 lpc2148 timer unit
 
Autosar Basics hand book_v1
Autosar Basics  hand book_v1Autosar Basics  hand book_v1
Autosar Basics hand book_v1
 
Arm processor
Arm processorArm processor
Arm processor
 
Risc
RiscRisc
Risc
 
Red Bend Software: Optimizing the User Experience with Over-the-Air Updates
Red Bend Software: Optimizing the User Experience with Over-the-Air UpdatesRed Bend Software: Optimizing the User Experience with Over-the-Air Updates
Red Bend Software: Optimizing the User Experience with Over-the-Air Updates
 
Automotive embedded systems part5 v2
Automotive embedded systems part5 v2Automotive embedded systems part5 v2
Automotive embedded systems part5 v2
 
IEEE Posix Standards
IEEE Posix StandardsIEEE Posix Standards
IEEE Posix Standards
 
An approach towards sotif with ansys medini analyze
An approach towards sotif with ansys medini analyzeAn approach towards sotif with ansys medini analyze
An approach towards sotif with ansys medini analyze
 
UDS: Vehicle Diagnostics in AUTOSAR Software Architecture
UDS: Vehicle Diagnostics in AUTOSAR Software Architecture UDS: Vehicle Diagnostics in AUTOSAR Software Architecture
UDS: Vehicle Diagnostics in AUTOSAR Software Architecture
 
Arm programmer's model
Arm programmer's modelArm programmer's model
Arm programmer's model
 
An optimal and progressive algorithm for skyline queries slide
An optimal and progressive algorithm for skyline queries slideAn optimal and progressive algorithm for skyline queries slide
An optimal and progressive algorithm for skyline queries slide
 
RISC Vs CISC, Harvard v/s Van Neumann
RISC Vs CISC, Harvard v/s Van NeumannRISC Vs CISC, Harvard v/s Van Neumann
RISC Vs CISC, Harvard v/s Van Neumann
 
Embedded systems - UNIT-1 - Mtech
Embedded systems - UNIT-1 - MtechEmbedded systems - UNIT-1 - Mtech
Embedded systems - UNIT-1 - Mtech
 
Automotive embedded systems part6 v1
Automotive embedded systems part6 v1Automotive embedded systems part6 v1
Automotive embedded systems part6 v1
 
Simd programming introduction
Simd programming introductionSimd programming introduction
Simd programming introduction
 
Trusted Platform Module (TPM)
Trusted Platform Module (TPM)Trusted Platform Module (TPM)
Trusted Platform Module (TPM)
 
Autosar software component
Autosar software componentAutosar software component
Autosar software component
 
19 Jun 2018 - Hazard Analysis and Functional Safety Compliance
19 Jun 2018 - Hazard Analysis and Functional Safety Compliance 19 Jun 2018 - Hazard Analysis and Functional Safety Compliance
19 Jun 2018 - Hazard Analysis and Functional Safety Compliance
 
AUToSAR introduction
AUToSAR introductionAUToSAR introduction
AUToSAR introduction
 
DAIS19: On the Performance of ARM TrustZone
DAIS19: On the Performance of ARM TrustZoneDAIS19: On the Performance of ARM TrustZone
DAIS19: On the Performance of ARM TrustZone
 

Semelhante a Technical Performance Measures

Cpm500 d _alleman__tpm jun 2010 lesson 3 (v2)
Cpm500 d _alleman__tpm jun 2010 lesson 3 (v2)Cpm500 d _alleman__tpm jun 2010 lesson 3 (v2)
Cpm500 d _alleman__tpm jun 2010 lesson 3 (v2)Glen Alleman
 
Technical Performance Measures
Technical Performance MeasuresTechnical Performance Measures
Technical Performance MeasuresGlen Alleman
 
Cpm500 d _alleman__tpm lesson 3 (v1)
Cpm500 d _alleman__tpm lesson 3 (v1)Cpm500 d _alleman__tpm lesson 3 (v1)
Cpm500 d _alleman__tpm lesson 3 (v1)Glen Alleman
 
Implementing Technical Performance Measures
Implementing Technical Performance MeasuresImplementing Technical Performance Measures
Implementing Technical Performance MeasuresGlen Alleman
 
Automation Skills Gap Fix
Automation Skills Gap FixAutomation Skills Gap Fix
Automation Skills Gap Fixjason_cov
 
Building the Performance Measurement Baseline
Building the Performance Measurement BaselineBuilding the Performance Measurement Baseline
Building the Performance Measurement BaselineGlen Alleman
 
20141209_AM Pillar_Introduction
20141209_AM Pillar_Introduction20141209_AM Pillar_Introduction
20141209_AM Pillar_IntroductionHenrik Gantzel
 
2012 connectionpoint pcammisa_hpcm
2012 connectionpoint pcammisa_hpcm2012 connectionpoint pcammisa_hpcm
2012 connectionpoint pcammisa_hpcmInnovusPartners
 
OTS - Everything you wanted to know but didn't ask
OTS - Everything you wanted to know but didn't askOTS - Everything you wanted to know but didn't ask
OTS - Everything you wanted to know but didn't askJeff Hackney
 
Using Technical Performance Progress
Using Technical Performance ProgressUsing Technical Performance Progress
Using Technical Performance ProgressGlen Alleman
 
Earning Value from Earned Value Management
Earning Value from Earned Value ManagementEarning Value from Earned Value Management
Earning Value from Earned Value ManagementGlen Alleman
 
Arrelic Services | APM
Arrelic Services | APMArrelic Services | APM
Arrelic Services | APMArrelic
 
Measurement News Webinar
Measurement News WebinarMeasurement News Webinar
Measurement News WebinarGlen Alleman
 
Building a Credible Performance Measurement Baseling
Building a Credible Performance Measurement BaselingBuilding a Credible Performance Measurement Baseling
Building a Credible Performance Measurement BaselingGlen Alleman
 
Total Productive Maintenance
Total Productive MaintenanceTotal Productive Maintenance
Total Productive MaintenanceAnand Subramaniam
 
Case study of San Diego city schools
Case study of San Diego city schoolsCase study of San Diego city schools
Case study of San Diego city schoolsMayank Mittal
 
Boeing Case Study | e-enabled Advantage
Boeing Case Study | e-enabled AdvantageBoeing Case Study | e-enabled Advantage
Boeing Case Study | e-enabled AdvantageAdheesha Dharmakeerthi
 
Ad Hoc Automation is an Expensive Mistake
Ad Hoc Automation is an Expensive MistakeAd Hoc Automation is an Expensive Mistake
Ad Hoc Automation is an Expensive MistakeBMC Software
 
Cmms a basic guide
Cmms a basic guideCmms a basic guide
Cmms a basic guidenoahweeks17
 

Semelhante a Technical Performance Measures (20)

Cpm500 d _alleman__tpm jun 2010 lesson 3 (v2)
Cpm500 d _alleman__tpm jun 2010 lesson 3 (v2)Cpm500 d _alleman__tpm jun 2010 lesson 3 (v2)
Cpm500 d _alleman__tpm jun 2010 lesson 3 (v2)
 
Technical Performance Measures
Technical Performance MeasuresTechnical Performance Measures
Technical Performance Measures
 
Cpm500 d _alleman__tpm lesson 3 (v1)
Cpm500 d _alleman__tpm lesson 3 (v1)Cpm500 d _alleman__tpm lesson 3 (v1)
Cpm500 d _alleman__tpm lesson 3 (v1)
 
Implementing Technical Performance Measures
Implementing Technical Performance MeasuresImplementing Technical Performance Measures
Implementing Technical Performance Measures
 
Automation Skills Gap Fix
Automation Skills Gap FixAutomation Skills Gap Fix
Automation Skills Gap Fix
 
Building the Performance Measurement Baseline
Building the Performance Measurement BaselineBuilding the Performance Measurement Baseline
Building the Performance Measurement Baseline
 
20141209_AM Pillar_Introduction
20141209_AM Pillar_Introduction20141209_AM Pillar_Introduction
20141209_AM Pillar_Introduction
 
2012 connectionpoint pcammisa_hpcm
2012 connectionpoint pcammisa_hpcm2012 connectionpoint pcammisa_hpcm
2012 connectionpoint pcammisa_hpcm
 
OTS - Everything you wanted to know but didn't ask
OTS - Everything you wanted to know but didn't askOTS - Everything you wanted to know but didn't ask
OTS - Everything you wanted to know but didn't ask
 
Using Technical Performance Progress
Using Technical Performance ProgressUsing Technical Performance Progress
Using Technical Performance Progress
 
Rcm
RcmRcm
Rcm
 
Earning Value from Earned Value Management
Earning Value from Earned Value ManagementEarning Value from Earned Value Management
Earning Value from Earned Value Management
 
Arrelic Services | APM
Arrelic Services | APMArrelic Services | APM
Arrelic Services | APM
 
Measurement News Webinar
Measurement News WebinarMeasurement News Webinar
Measurement News Webinar
 
Building a Credible Performance Measurement Baseling
Building a Credible Performance Measurement BaselingBuilding a Credible Performance Measurement Baseling
Building a Credible Performance Measurement Baseling
 
Total Productive Maintenance
Total Productive MaintenanceTotal Productive Maintenance
Total Productive Maintenance
 
Case study of San Diego city schools
Case study of San Diego city schoolsCase study of San Diego city schools
Case study of San Diego city schools
 
Boeing Case Study | e-enabled Advantage
Boeing Case Study | e-enabled AdvantageBoeing Case Study | e-enabled Advantage
Boeing Case Study | e-enabled Advantage
 
Ad Hoc Automation is an Expensive Mistake
Ad Hoc Automation is an Expensive MistakeAd Hoc Automation is an Expensive Mistake
Ad Hoc Automation is an Expensive Mistake
 
Cmms a basic guide
Cmms a basic guideCmms a basic guide
Cmms a basic guide
 

Mais de Glen Alleman

Managing risk with deliverables planning
Managing risk with deliverables planningManaging risk with deliverables planning
Managing risk with deliverables planningGlen Alleman
 
A Gentle Introduction to the IMP/IMS
A Gentle Introduction to the IMP/IMSA Gentle Introduction to the IMP/IMS
A Gentle Introduction to the IMP/IMSGlen Alleman
 
Increasing the Probability of Project Success
Increasing the Probability of Project SuccessIncreasing the Probability of Project Success
Increasing the Probability of Project SuccessGlen Alleman
 
Process Flow and Narrative for Agile+PPM
Process Flow and Narrative for Agile+PPMProcess Flow and Narrative for Agile+PPM
Process Flow and Narrative for Agile+PPMGlen Alleman
 
Practices of risk management
Practices of risk managementPractices of risk management
Practices of risk managementGlen Alleman
 
Principles of Risk Management
Principles of Risk ManagementPrinciples of Risk Management
Principles of Risk ManagementGlen Alleman
 
Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...
Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...
Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...Glen Alleman
 
From Principles to Strategies for Systems Engineering
From Principles to Strategies for Systems EngineeringFrom Principles to Strategies for Systems Engineering
From Principles to Strategies for Systems EngineeringGlen Alleman
 
NAVAIR Integrated Master Schedule Guide guide
NAVAIR Integrated Master Schedule Guide guideNAVAIR Integrated Master Schedule Guide guide
NAVAIR Integrated Master Schedule Guide guideGlen Alleman
 
Building a Credible Performance Measurement Baseline
Building a Credible Performance Measurement BaselineBuilding a Credible Performance Measurement Baseline
Building a Credible Performance Measurement BaselineGlen Alleman
 
Integrated master plan methodology (v2)
Integrated master plan methodology (v2)Integrated master plan methodology (v2)
Integrated master plan methodology (v2)Glen Alleman
 
IMP / IMS Step by Step
IMP / IMS Step by StepIMP / IMS Step by Step
IMP / IMS Step by StepGlen Alleman
 
DHS - Using functions points to estimate agile development programs (v2)
DHS - Using functions points to estimate agile development programs (v2)DHS - Using functions points to estimate agile development programs (v2)
DHS - Using functions points to estimate agile development programs (v2)Glen Alleman
 
Making the impossible possible
Making the impossible possibleMaking the impossible possible
Making the impossible possibleGlen Alleman
 
Heliotropic Abundance
Heliotropic AbundanceHeliotropic Abundance
Heliotropic AbundanceGlen Alleman
 
Capabilities based planning
Capabilities based planningCapabilities based planning
Capabilities based planningGlen Alleman
 
Process Flow and Narrative for Agile
Process Flow and Narrative for AgileProcess Flow and Narrative for Agile
Process Flow and Narrative for AgileGlen Alleman
 
Program Management Office Lean Software Development and Six Sigma
Program Management Office Lean Software Development and Six SigmaProgram Management Office Lean Software Development and Six Sigma
Program Management Office Lean Software Development and Six SigmaGlen Alleman
 
Policy and Procedure Rollout
Policy and Procedure RolloutPolicy and Procedure Rollout
Policy and Procedure RolloutGlen Alleman
 
Integrated Master Plan Development
Integrated Master Plan DevelopmentIntegrated Master Plan Development
Integrated Master Plan DevelopmentGlen Alleman
 

Mais de Glen Alleman (20)

Managing risk with deliverables planning
Managing risk with deliverables planningManaging risk with deliverables planning
Managing risk with deliverables planning
 
A Gentle Introduction to the IMP/IMS
A Gentle Introduction to the IMP/IMSA Gentle Introduction to the IMP/IMS
A Gentle Introduction to the IMP/IMS
 
Increasing the Probability of Project Success
Increasing the Probability of Project SuccessIncreasing the Probability of Project Success
Increasing the Probability of Project Success
 
Process Flow and Narrative for Agile+PPM
Process Flow and Narrative for Agile+PPMProcess Flow and Narrative for Agile+PPM
Process Flow and Narrative for Agile+PPM
 
Practices of risk management
Practices of risk managementPractices of risk management
Practices of risk management
 
Principles of Risk Management
Principles of Risk ManagementPrinciples of Risk Management
Principles of Risk Management
 
Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...
Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...
Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...
 
From Principles to Strategies for Systems Engineering
From Principles to Strategies for Systems EngineeringFrom Principles to Strategies for Systems Engineering
From Principles to Strategies for Systems Engineering
 
NAVAIR Integrated Master Schedule Guide guide
NAVAIR Integrated Master Schedule Guide guideNAVAIR Integrated Master Schedule Guide guide
NAVAIR Integrated Master Schedule Guide guide
 
Building a Credible Performance Measurement Baseline
Building a Credible Performance Measurement BaselineBuilding a Credible Performance Measurement Baseline
Building a Credible Performance Measurement Baseline
 
Integrated master plan methodology (v2)
Integrated master plan methodology (v2)Integrated master plan methodology (v2)
Integrated master plan methodology (v2)
 
IMP / IMS Step by Step
IMP / IMS Step by StepIMP / IMS Step by Step
IMP / IMS Step by Step
 
DHS - Using functions points to estimate agile development programs (v2)
DHS - Using functions points to estimate agile development programs (v2)DHS - Using functions points to estimate agile development programs (v2)
DHS - Using functions points to estimate agile development programs (v2)
 
Making the impossible possible
Making the impossible possibleMaking the impossible possible
Making the impossible possible
 
Heliotropic Abundance
Heliotropic AbundanceHeliotropic Abundance
Heliotropic Abundance
 
Capabilities based planning
Capabilities based planningCapabilities based planning
Capabilities based planning
 
Process Flow and Narrative for Agile
Process Flow and Narrative for AgileProcess Flow and Narrative for Agile
Process Flow and Narrative for Agile
 
Program Management Office Lean Software Development and Six Sigma
Program Management Office Lean Software Development and Six SigmaProgram Management Office Lean Software Development and Six Sigma
Program Management Office Lean Software Development and Six Sigma
 
Policy and Procedure Rollout
Policy and Procedure RolloutPolicy and Procedure Rollout
Policy and Procedure Rollout
 
Integrated Master Plan Development
Integrated Master Plan DevelopmentIntegrated Master Plan Development
Integrated Master Plan Development
 

Último

DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfPrecisely
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 

Último (20)

DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 

Technical Performance Measures

  • 1. CPM–500 EVM/SE/TPM Integration CPM–500–B/C/F : Integrating Systems Engineering with Earned Value Management Lesson 3 CPM–500F: Technical Performance Measures Glen B. Alleman Lewis & Fowler galleman@lewisandfowler.com (303) 241 9633 June 2010 Naples, Florida Professional Education Program (Training Track) presented by PMI–College of Performance Management faculty Rights Reserved 1/63
  • 2. CPM–500 EVM/SE/TPM Integration The Purpose Of This Series  This is the last “connected” lesson in a 3–part series that explores the integration of System Engineering (SE) and Earned Value Management (EVM).  You don’t need to attend all three lessons – but it helps in Connecting The Dots  Why have this series? – Often, project control specialists see the planning of the cost and schedule baselines as an isolated activity disconnected from the technical or engineering aspects. – This type of thinking significantly impacts development of the baseline which results in poor maintenance and eventual irrelevance of the baseline – We claim “victory” if after any one of these lessons, you can recognize the importance of integrating these disciplines such that he/she will advocate greater cooperation between the engineering and project control elements of his/her respective organization Rights Reserved 2/63
  • 3. CPM–500 EVM/SE/TPM Integration Lesson 3 Objectives In this lesson we will:  Understand how to establish credibility for Technical Performance Measures by using Earned Value and Systems Engineering to measure progress  Establish the processes of defining and measuring technical performance  Establish the role of the technical baseline in program management  Learn how TPM’s are integrated into planning and execution at the control account level through examples and a hands on exercise Rights Reserved 3/63
  • 4. CPM–500 EVM/SE/TPM Integration Can Earned Value Alone Get Us To Our Destination?  How do we increase visibility into the program’s performance?  How do we reduce cycle time to deliver the product?  How do we foster accountability?  How do we reduce risk?  How do we start this journey to success? Increasing the Probability of Success means we have to Connect The Dots to Reach Our Destination Rights Reserved 4/63
  • 5. CPM–500 EVM/SE/TPM Integration To Achieve Success … We Need to … ©gapingvoid ltd www.gapingvoidgallery.com Rights Reserved 5/63
  • 6. CPM–500 EVM/SE/TPM Integration Learnings from Lesson 1 and 2 How cost and schedule (EV) are connected to the Performance Measurement Baseline Rights Reserved 6/63
  • 7. CPM–500 EVM/SE/TPM Integration Increasing the Probability of Program Success Means … Building A Credible Performance Measurement Baseline Risk Cost IMP/IMS PMB SOW WBS TPM This is actually harder than it looks! Rights Reserved 7/63
  • 8. CPM–500 EVM/SE/TPM Integration Doing This Starts With Some Guidance Systems engineering uses technical performance measurements to balance cost, schedule, and performance throughout the life cycle. Technical performance measurements compare actual versus planned technical development and design. They also report the degree to which system requirements are met in terms of performance, cost, schedule, and progress in implementing risk handling. Performance metrics are traceable to user–defined capabilities. ― Defense Acquisition Guide (https://dag.dau.mil/Pages/Default.aspx) In The End ― It’s All About Systems Engineering Rights Reserved 8/63
  • 9. CPM–500 EVM/SE/TPM Integration Guidance for Measures of Effectiveness, Performance, and Technical Performance Our starting point is not EVM, it’s Systems Engineering  MOE’s are an essential part of Systems Engineering, guided by IEEE 1220 and EIA 632.  System’s Engineers drive the content of all measurement items, customer or supplier. Rights Reserved 9/63
  • 10. CPM–500 EVM/SE/TPM Integration Just A Reminder Of The … Primary Elements of Earned Value Cost Funding margin Schedule margin for for under Over cost or Over cost or over target baseline performance under over (OTB) performance schedule Over Technical schedule or Performance under Schedule performing Schedule margin for underperformance or schedule extension Rights Reserved 10/63
  • 11. CPM–500 EVM/SE/TPM Integration Previous Approaches Using EV Are Mostly Unsuccessful In Connecting These  Traditional approaches to program management are retrospective – Cost and schedule of Earned Value – Risk Management – Systems Engineering  Reporting past performance – Sometimes 30 to 60 days old – Variances are reporting beyond the widow of opportunity for correction Rights Reserved 11/63
  • 12. CPM–500 EVM/SE/TPM Integration It’s All Been Said Before. We Just Weren’t Listening… … the basic tenets of the process are the need for seamless management tools, that support an integrated approach … and “proactive identification and management of risk” for critical cost, schedule, and technical performance parameters. ― Secretary of Defense, Perry memo, May 1995 Why Is This Hard To Understand?  We seem to be focused on EV reporting, not the use of EV to manage the program.  Getting the CPR out the door is the end of Program Planning and Control’s efforts, not the beginning. Rights Reserved 12/63
  • 13. CPM–500 EVM/SE/TPM Integration The Gap Seems To Start With A Common Problem Many Times, The Information from Cost, Schedule, Techncial Performance, and Risk Management Gets Mixed Up When We Try to Put Them Together Rights Reserved 13/63
  • 14. CPM–500 EVM/SE/TPM Integration When We Put The Cart Before The Horse, We Discover …  EVM really doesn’t do its job effectively  Most of the time EV has no measure of quality or compliance with technical requirements.  EV measures progress to plan in units of “money,” not tangible value to the customer  Most EV System Descriptions fail to connect the dots between cost, schedule, and technical performance – even though instructed to do so in the official guidance Rights Reserved 14/63
  • 15. CPM–500 EVM/SE/TPM Integration The NDIA EVM Intent Guide Says Notice the inclusion of Technical along with Cost and Schedule That’s the next step is generating Value from Earned Value EV MUST include the Technical Performance Measures Rights Reserved
  • 16. CPM–500 EVM/SE/TPM Integration Back To Our Technical Performance Measures Technical Performance Measures do what they say, Measure the Technical Performance of the product or service produced by the program Rights Reserved 16/63
  • 17. CPM–500 EVM/SE/TPM Integration What’s Our Motivation for “Connecting the Dots?” TPMs are a set of measures that provide the supplier and acquirer with insight into progress to plan of the technical solution, the associated risks, and emerging issues. Technical Performance Measures …  Provide program management with information to make better decisions  Increase the probability of delivering a solution that meets both the requirements and mission need We’ve been talking about this since as early as 1984, in Technical Performance Measurement Handbook, Defense Systems Management College, Fort Belvoir, VA 22060 Rights Reserved 17/63
  • 18. CPM–500 EVM/SE/TPM Integration Measure of Effectiveness (MoE) The operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in the operational environment, under a specific set of condition Measures of Effectiveness …  Are stated by the buyer in units meaningful to the buyer  Focus on capabilities independent of any technical implementation MoE Belong to the End User “Technical Measurement,” INCOSE–TP–2003–020–01 Rights Reserved 18/63
  • 19. CPM–500 EVM/SE/TPM Integration Measure of Performance (MoP) Measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions Measures of Performance are …  Attributes that assure the system has the capability to perform  Assessment of system to assure it meets design requirements necessary to satisfy the MOE MoP’s belong to the Program – Developed by the Systems Engineer, Measured By CAMs, and Analyzed by PP&C “Technical Measurement,” INCOSE–TP–2003–020–01 Rights Reserved 19/63
  • 20. CPM–500 EVM/SE/TPM Integration Key Performance Parameters (KPP) Represent the capabilities and characteristics so significant that failure to meet them can be cause for reevaluation, reassessing, or termination of the program Key Performance Parameters …  have a threshold or objective value  Characterize the major drivers of performance  Are considered Critical to Customer (CTC) The acquired defines the KPPs during the operational concept development – KPPs say what DONE looks like “Technical Measurement,” INCOSE–TP–2003–020–01 Rights Reserved 20/63
  • 21. CPM–500 EVM/SE/TPM Integration Technical Performance Measures (TPM) Attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal Technical Performance Measures …  Assess design progress  Define compliance to performance requirements  Indentify technical risk  Are limited to critical thresholds  Include projected performance “Technical Measurement,” INCOSE–TP–2003–020–01 Rights Reserved 21/63
  • 22. CPM–500 EVM/SE/TPM Integration Dependencies Between Measures Stakeholders Define Needs and Capabilities Suppliers Define Physical Solutions that in terms of Operational Scenarios meet the needs of the Stakeholders KPP Mission MoE MoP TPM Need Operational Measures that Measures used to measures of characterize assess design success related to physical or progress, the achievement functional compliance to of the mission or attributes relating performance operational to the system requirements, and objective being operation technical risks evaluated. “Coming to Grips with Measures of Effectiveness,” N. Sproles, Systems Engineering, Volume 3, Number 1, pp. 50–58 Rights Reserved 22/63
  • 23. CPM–500 EVM/SE/TPM Integration “Candidates” for Technical Measures Concept Description Useful Life Physical Size and Stability Weight Volumetric capacity Accuracy Functional Correctness Power performance Supportability Maintainability All the “ilities” Dependability Reliability Reliability = Mean Time Failure Utilization Efficiency Response time Throughput Suitability for Purpose Readiness Rights Reserved 23/63 INCOSE Systems Engineering Handbook
  • 24. CPM–500 EVM/SE/TPM Integration “Measures” of Technical Measures Attribute Description Measured technical progress or estimate of Achieved to Date progress Value of a technical parameter that is predicted to Current Estimate be achieved Point in time when an evaluation of a measure is Milestone accomplished Planned Value Predicted value of the technical parameter Planned Performance Profile representing the project time phased Profile demonstration of a technical parameter Tolerance Band Management alert limits Threshold Limiting acceptable value of a technical parameter  Demonstrated technical variance Variances INCOSE Systems Engineering Handbook  Predicted technical variance Rights Reserved 24/63
  • 25. CPM–500 EVM/SE/TPM Integration A Familiar Graphic of TPMs TPM Upper Limit Planned Profile Current Estimate Planned Value Mean To Between Failure Threshold Variance Lower Limit Achieved to Date Milestones Time = Program Maturity Rights Reserved 25/63
  • 26. CPM–500 EVM/SE/TPM Integration A Simple Method of Assembling the TPMs MOE / MOP KPP / TPM Risks Define the Assess the Select Technical planned impact on Risk Performance progress for from this Parameters each TPM progress Parameters Progress Risk  Weight XXXX  Speed XXXX  MTBF XXXX  Loiter Time XXXX Rights Reserved 26/63
  • 27. CPM–500 EVM/SE/TPM Integration TPMs from an Actual Program James Webb Space Telescope Rights Reserved 27/63
  • 28. CPM–500 EVM/SE/TPM Integration TPMs from an Actual Program Chandra X–Ray Telescope Rights Reserved 28/63
  • 29. CPM–500 EVM/SE/TPM Integration What Does A Real Technical Performance Measure Look Like? Not that bagels are not interesting in Lesson 1 and 2, but let’s get ready to look at a flying machine. Rights Reserved 29/63
  • 30. CPM–500 EVM/SE/TPM Integration The WBS for a UAV TPMs Start With The WBS 1.1 Air Vehicle 1.1.1 Sensor Platform 1.1.2 1.1.2 Airframe Airframe 1.1.3 Propulsion 1.1.4 On Board Comm 1.1.5 Auxiliary Equipment 1.1.6 Survivability Modules 1.1.7 Electronic Warfare Module 1.1.8 On Board Application & System SW 1.3 Mission Control / Ground Station SW 1.3.1 Signal Processing SW 1.3.2 Station Display 1.3.3 Operating System 1.3.4 ROE Simulations 1.3.5 Mission Commands Rights Reserved 30/63
  • 31. CPM–500 EVM/SE/TPM Integration What Do We Need To Know About This Program Through TPMs  What WBS elements represent the TPMs?  What Work Packages produce these WBS elements?  Where do these Work Packages live in the IMS?  What are the Earned Value baseline values for these Work Packages?  How are going to measure all these variables?  What does the curve look like for these measurements? Rights Reserved 31/63
  • 32. CPM–500 EVM/SE/TPM Integration Let’s Connect The Dots Technical and Programmatic Risks Connected to the WBS and IMS BCWS at the Work Package, rolled to the Control Account IMS contains all the Work Packages, BCWS, Risk Cost Risk mitigation Named plans, and rolls to IMP/IMS PMB SOW Deliverables the Integrated defined in the WBS Master Plan to WBS TPM measure increasing maturity TPMs attached to each critical deliverables in the WBS and identified in The Products and each Work Package in the Processes that produce IMS, used to assess them in a “well structured” maturity in the IMP decomposition in the WBS Rights Reserved 32/63
  • 33. CPM–500 EVM/SE/TPM Integration Verifying Each TPM Evidence that we’re in compliance With our submitted ROM what are the values we need to get Do we know what we promised to CA through Integrated Baseline Review «how do we measure deliver, now that we’ve won? weight for each program event» The contributors to the vehicle weight are confirmed and the Can we proceed into preliminary SFR upper limits defined in the product architecture and design? requirements flow down database (DOORS) into a model Can we proceed into the System Do we know all drivers of vehicle weight? Can we bound their SRR Development and Demonstration upper limits? Can the subsystem owners be successful within (SDD) phase these constraints uses a high fidelity model? Can we start detailed design, and Does each subsystem designer have the target component meet the stated performance weight target and have some confidence they can stay below PDR requirements the upper bound? Can this be verified in some tangible way? within cost, schedule, Either through prior examples or a lab model? risk, and other constraints? Can the system proceed to Do we know all we need to know to start the fabrication of fabrication, demonstration, and test, the first articles of the flight vehicle. Some type of example, CDR with the within cost, schedule, risk, maybe a prototype is used to verify we’re inside the lines and other system constraints. Can the system ready to Does the assembled vehicle fall within the weight range limits TRR Rights Reserved proceed into formal test? for 1st flight – will this thing get off the ground? 33/63
  • 34. CPM–500 EVM/SE/TPM Integration TPM Trends & Responses Design Model ROM in Proposal Detailed Design Model Bench Scale Model Measurement Technical Performance Measure 28kg Prototype Measurement Vehicle Weight Flight 1st Article 26kg 25kg 23kg CA SFR SRR PDR CDR TRR EV Taken, planned values met, tolerances kept, etc. Rights Reserved Dr. Falk Chart – modified 34/63
  • 35. CPM–500 EVM/SE/TPM Integration The Assessment Of Weight As A Function Of Time  At Contract Award there is a Proposal grade estimate of vehicle weight  At System Functional Review, the Concept of Operations is validated for the weight  At System Requirements Review the weight targets are flowed down to the subsystems components  At PDR the CAD model starts the verification process  At CDR actual measurements are needed to verify all models  At Test Readiness Review we need to know how much fuel to put on board for the 1st flight test Rights Reserved 35/63
  • 36. CPM–500 EVM/SE/TPM Integration The WBS for a UAV Airframe Weight TPM 1.1 Air Vehicle The planned weight is 1.1.1 Sensor Platform 25kg. The actual weight is 1.1.2 Airframe 25.5kg. 1.1.2 Airframe Close to plan! So we are doing okay, right? CA SFR SRR PDR CDR TRR Planned Value 28.0kg 27.0kg 26.0kg 25.0kg 24.0kg 23.0kg Actual Value 30.4kg 29.0kg 27.5kg 25.5kg Moderate Low Low Very Low (less Assessed Risk >2.0kg off 1–2 kg off 1–2 kg off than 1.0 kg to TRR target target target off target) Program– Actual Actual Program– Planned “Similar to” unique design measurement measurement ROM unique design Method Estimate model with of bench–test of prototype model validated data components airframe Actual “Similar to” Method Estimate ROM ROM ROM Here’s the Problem Rights Reserved 36/63
  • 37. CPM–500 EVM/SE/TPM Integration Is This A Problem? You Bet’ya It’s A Problem!  The measurement is close to the planned value,  But the planned method of measurement is a program unique design model with validated data,  But the actual method of measurement is a rough order of magnitude estimate,  No improvement in fidelity since the System Functionality Review (SFR), and  The TPM provides no new information – so we’re probably late and don’t know it yet. Rights Reserved 37/63
  • 38. CPM–500 EVM/SE/TPM Integration Raison d'etre for Technical Performance Measures The real purpose of Risk Cost Technical Performance Measures is to reduce IMP/IMS PMB SOW Programmatic and WBS TPM Technical RISK Rights Reserved 38/63
  • 39. CPM–500 EVM/SE/TPM Integration Buying Down Risk with TPMs Risk: CEV-037 - Loss of Critical Functions During Descent  “Buying down” risk is 24 22 20 Correlate the analytical model Conduct focus splinter review Develop analytical model to de planned in the IMS.  MoE, MoP, and KPP Conduct Force and Moment Wind 18 Conduct Block 1 w ind tunnel te 16 Conduct w ind tunnel testing of Conduct w ind tunnel testing of defined in the work 14 Flight Application of Spacecra 12 package for the critical 10 CEV block 5 w ind tunnel testin 8 6 4 2 In-Flight development tests of Damaged TPS flight test measure – weight.  If we can’t verify 0 3.Jul.06 1.Jul.11 31.Mar.05 5.Oct.05 1.Jun.07 1.Jan.10 16.Dec.10 15.Sep.06 3.Apr.06 1.Apr.08 1.Aug.08 1.Apr.09 Planned Risk Level Weight risk reduced from Planned (Solid=Linked, Hollow =Unlinked, Filled=Complete) Weight confirmed ready to fly – it’s we’ve succeeded, then RED to Yellow GREEN at this point the risk did not get reduced.  The risk may have gotten worse. Rights Reserved 39/63
  • 40. CPM–500 EVM/SE/TPM Integration Increasing the Probability of Success with Risk Management  Going outside the TPM limits always means cost and schedule impacts  “Coloring Inside the Lines” means knowing the how to keep the program GREEN, or at least stay close to So much for our strategy of winning GREEN through technical dominance Rights Reserved 40/63
  • 41. CPM–500 EVM/SE/TPM Integration Connecting the EV Variables Integrating Cost, Schedulele, and Technical Performance Assures Program Management has the needed performance information to deliver on‒time, on‒budget, and on‒specification = Technical Performance Measures Cost + Schedule Conventional Earned Value Cost Baseline Technical Performance Schedule Baseline  Master Schedule used to  Earned Value is diluted  Requirements are derive Basis of Estimate by missing technical decomposed into (BOE) not the other way performance. physical deliverables. around.  Earned Value is diluted  Deliverables are  Probabilistic cost by postponed features. produced through Work estimating uses past  Earned Value is diluted Packages. performance and cost by non compliant quality.  Work Packages are risk modeling.  All these dilutions assigned to accountable  Labor, Materiel, and require adjustments to manager. other direct costs the Estimate at Complete  Work Packages are accounted for in Work (EAC) and the To sequenced to form the Packages Complete Performance highest value stream  Risk adjustments for all Index (TCPI). with the lowest technical elements of cost. Rights Reserved and programmatic risk. 41/63
  • 42. CPM–500 EVM/SE/TPM Integration TPM Checklist MoE MoP TPM Traceable to needs, Traceable to applicable Traceable to applicable MoPs, goals, objectives, and MOEs, KPPs, system level system element performance, risks performance requirements, requirements, objectives, and risks risks, and WBS elements Defined with associated Focused on technical risks Further decomposed, KPPs and supports trades budgeted, and allocated to between alternative lower level system elements in solutions the WBS and IMS Each MoE independent Provided insight into Assigned an owner, the CAM from others system performance and Work Package Manager Each MoE independent Decomposed, budgeted Sources of measure identified of technical any solution and allocated to system and processes for generating elements the measures defined. Address the required Assigned an “owner,” the Integrated into the program’s KPPs CAM and Technical IMS as part of the exit criteria Manager Rights Reserved for the Work Package 42/63
  • 43. CPM–500 EVM/SE/TPM Integration Increasing the Probability of Program Success Means … Building A Credible Performance Measurement Baseline Risk Cost IMP/IMS PMB SOW WBS TPM Using the Check List – “Connect the Dots” Rights Reserved 43/63
  • 45. CPM–500 EVM/SE/TPM Integration Backup Materials Knowledge is of two kinds. We know a subject ourselves, or we know where we can find information on it — Samuel Johnson Rights Reserved 45/63
  • 46. CPM–500 EVM/SE/TPM Integration Many of Sources for Connecting the Dots OMB Circular A–11, Section 300 Interim Defense Acquisition Guidebook (DAG) 6/15/09 GAO Report 06–250 Systems Engineering Plan (SEP) Preparation Guide 4/08 DoDI 5000.02, Operation of the Defense WBS Handbook, Mil–HDBK–881A (WBS) 7/30/05 Acquisition System (POL) 12/08 Integrated Master Plan (IMP) & Integrated Guide for Integrating SE into DOD Acquisition Master Schedule Preparation & Use Guide Contracts 12/06 (IMS) 10/21/05 Defense Acquisition Program Support Guide to the Project Management Institute Body of Methodology (DAPS) V2.0 3/20/09 Knowledge (PMBOK Guide®), 4th Edition Standard for Application and Capability Maturity Model Integration (CMMI®) Management of the SE Process (IEEE 1220) IEEE 1220: 6.8.1.5 Processes for Engineering a System (ANSI/EIA–632) NASA EVM Guide NPG 9501.3 Rights Reserved 46/63
  • 47. CPM–500 EVM/SE/TPM Integration Office of Management and Budget Circular No. A–11, Section 300  Planning, Budgeting, Acquisition and Management of Capital Assets  Section 300–5 – Performance–based acquisition management – Based on EVMS standard – Measure progress towards milestones • Cost • Capability to meet specified requirements • Timeliness • Quality Rights Reserved 47/63
  • 48. CPM–500 EVM/SE/TPM Integration Need: Accurate Performance Measurement Findings and GAO Report 06–250 Recommendations Information Technology: 2. If EVM is not Improve the Accuracy and implemented effectively, Reliability of Investment decisions based on Information inaccurate and potentially misleading information 3. Agencies not measuring actual versus expected performance in meeting IT performance goals. Rights Reserved 48/63
  • 49. CPM–500 EVM/SE/TPM Integration DOD Guides: Technical Performance Department of Defense Guidelines for Technical Performance Measures DoDI 5000.02, Operation of the Defense Acquisition System (POL) 12/08 Interim Defense Acquisition Guidebook (DAG) 6/15/09 Systems Engineering Plan (SEP) Preparation Guide 4/08 WBS Handbook, Mil–HDBK–881A (WBS) 7/30/05 Integrated Master Plan (IMP) & Integrated Master Schedule Preparation & Use Guide (IMS) 10/21/05 Guide for Integrating SE into DOD Acquisition Contracts (Integ SE) 12/06 Defense Acquisition Program Support Methodology (DAPS) V2.0 3/20/09 Rights Reserved 49/63
  • 50. CPM–500 EVM/SE/TPM Integration DoD: TPMs in Technical Baselines and Reviews Engineering Integrated IMP/IMS Systems DAPS WBS DAG POL SEP DoD Policy or Guide Technical Baselines: IMP/IMS Functional (SFR)    Allocated (PDR) Product (CDR) Event driven timing        Success criteria of technical review        Entry and exit criteria for technical reviews     Assess technical maturity  Reserved Rights    50/63
  • 51. CPM–500 EVM/SE/TPM Integration DoD: TPMs in Integrated Plans Engineering Integrated IMP/IMS Systems DAPS WBS DAG POL SEP DoD Policy or Guide Integrated SEP with: IMP/IMS TPMs      EVM Integrated WBS with Requirement Specification      Statement of Work IMP/IMS/EVMS Link risk management, technical reviews, TPMs,    EVM, WBS, IMS Rights Reserved 51/63
  • 52. CPM–500 EVM/SE/TPM Integration Guidance in Standards, Models, and Defense Acquisition Guide  Processes for Engineering a System (ANSI/EIA–632)  Standard for Application and Management of the SE Process (IEEE 1220)  Capability Maturity Model Integration (CMMI®) – CMMI for Development, Version 1.2 – CMMI for Acquisition, Version 1.2 – Using CMMI to Improve Earned Value Management, 2002  Guide to the Project Management Institute Body of Knowledge (PMBOK Guide®), 4th Edition Rights Reserved 52/63
  • 53. CPM–500 EVM/SE/TPM Integration Technical Performance Measures (TPM) More Sources IEEE 1220: 6.8.1.5, EIA–632: Glossary CMMI for Development Performance–based Requirements progress measurement Development TPMs are key to Predict future value of key Specific Practice (SP) 3.3, progressively assess technical parameters of Analyze Requirements technical progress the end system based on Typical work product: current assessments TPMs Establish dates for Planned value profile is Subpractice: – Checking progress time–phased achievement Identify TPMs that will be – Meeting full projected tracked during conformance to • Achievement to date development requirements • Technical milestone where TPM evaluation is reported Rights Reserved 53/63
  • 54. CPM–500 EVM/SE/TPM Integration PMBOK® Guide  10.5.1.1 Project Management Plan  Performance Measurement Baseline: – Typically integrates scope, schedule, and cost parameters of a project – May also include technical and quality parameters Rights Reserved 54/63
  • 55. CPM–500 EVM/SE/TPM Integration PMBOK® Guide  8.3.5.4 Work Performance Measurements  Used to produce project activity metrics  Evaluate actual progress as compared to planned progress  Include, but are not limited to: – Planned vs. actual technical performance – Planned vs. actual schedule performance, and – Planned vs. actual cost performance. Rights Reserved 55/63
  • 56. CPM–500 EVM/SE/TPM Integration TPMs in DAG and DAPS Defense Acquisition Guide  Performance measurement of WBS elements, using objective measures: – Essential for EVM and Technical Assessment activities  Use TPMs and Critical Technical Parameters (CTP) to report progress in achieving milestones DAPS  Use TPMs to determine whether % completion metrics accurately reflect quantitative technical progress and quality toward meeting Key Performance Parameters (KPP) and Critical Technical Parameters Rights Reserved 56/63
  • 57. CPM–500 EVM/SE/TPM Integration TPMs in DAG  Compare the actual versus planned technical development and design  Report progress in the degree to which system performance requirements are met.  Plan is defined in terms of: – Expected performance at specific points • Defined in the WBS and IMS – Methods of measurement at those points – Variation limits for corrective action. Rights Reserved 57/63
  • 58. CPM–500 EVM/SE/TPM Integration PMBOK® Guide  11.6.2.4 Technical Performance Measurement  Compares technical accomplishments… to … project management plan’s schedule of technical achievement  Requires definition of objective quantifiable measures of technical performance which can be used to compare actual results against targets.  Might include weight, transaction times, number of delivered defects, storage capacity etc.  Deviation, such as demonstrating more or less functionality than planned at a milestone…forecast degree of success in achieving the project’s scope. Rights Reserved 58/63
  • 59. CPM–500 EVM/SE/TPM Integration CMMI–ACQ  Acquisition Technical Management  SP 1.3 Conduct Technical Reviews  Typical supplier deliverables  Progress reports and process, product, and service level measurements  TPMs Rights Reserved 59/63
  • 60. CPM–500 EVM/SE/TPM Integration SMS Shall: Monitor Progress Against the Plan  4.2.12.2 Monitoring  Contractor SHALL monitor progress against plan to validate, approve, and maintain each baseline and functional architecture  4.2.12.2.2 Required Product Attributes  Each documented assessment includes:  TPMs, metrics  Metrics and technical parameters for tracking that are critical indicators of technical progress and achievement Rights Reserved 60/63
  • 61. CPM–500 EVM/SE/TPM Integration NASA EVM Guide: Technical Performance • NASA EVM Guide NPG 9501.3 – 4.5 Technical Performance Requirements (TPR): When TPRs are used, – appropriate and relevant metrics… – must be defined in the solicitation – Appendix A.7, 14.1 TPR • Compares: • Expected performance and • Physical characteristics • With contractually specified values. • Basis for reporting established milestones • Progress toward meeting technical requirements Rights Reserved 61/63
  • 62. CPM–500 EVM/SE/TPM Integration Derivation and Flow Down of TPMs Document, Baseline, IMS, EVM Parameter  IMP, Functional Baseline Measures Of Effectiveness (MOE)   IMP, WBS, Functional Baseline Measures Of Performance (MOP)   IMP, Allocated Baseline Technical Performance Measure  TPM Milestones And Planned  IMS  Values  Work Packages TPM% Complete Criteria  See next chart for linkage of technical baselines to technical reviews Rights Reserved 62/63
  • 63. CPM–500 EVM/SE/TPM Integration Interesting Attributes of TPMs  Achieved to Date (sounds like EV)  Current Estimate (sounds like EAC/ETC)  Milestone  Planned (target) value (sounds like PV)  Planned performance profile (sounds like a PMB)  Tolerance band (sounds like reporting thresholds)  Threshold (yep, just what we thought)  Variance (sounds like variance!) Rights Reserved 63/63