The document outlines an architecture for a cost analytics process that includes capturing cost data in a database from various project phases and cost types, iterating the data through multiple estimation events, and using the stored data and associated schedule information to perform cost analytics and reporting across the project lifecycle. It details the types of fundamental cost and schedule data to be collected, how the data will be iterated through estimation events, and tools that can be used to analyze and report on the stored cost data.

1. Architecture for Cost
Analytics Process
Environment
Tony Harvey
Daren Elliott
NASA 2012 PM Challenge
February 22, 2012
 Los Angeles  Washington, D.C.  Boston  Chantilly  Huntsville  Dayton  Santa Barbara
 Albuquerque  Colorado Springs  Goddard Space Flight Center  Johnson Space Center  Ogden  Patuxent River  Washington Navy Yard
 Ft. Meade  Ft. Monmouth  Dahlgren  Quantico  Cleveland  Montgomery  Silver Spring  San Diego  Tampa  Tacoma
 Aberdeen  Oklahoma City  Eglin AFB  San Antonio  New Orleans  Denver  Vandenberg AFB
PRT 114 03 January 2012 1 of 42

2. Outline/Agenda
 Project Lifecycle Process
Context
 Cost Analytics Framework
Overview
 Fundamental Cost Data
 Data Iteration
 Cost Analytics
 Tools for Cost Analytics
 Cost Analytics in the
Project Lifecycle
PRT 114 03 January 2012 Approved For Public Release 2 of 42

3. Project Lifecycle Process Overview
 High-level Project Lifecycle
 Definition Phase (Pre-Phase A)
 Simple Definition of Functionality with Value Proposition (High Level Cost
and Schedule )
 Acquisition Phase (Phase A)
 Detailed Definition with Cost and Schedule through End-of-Life
 Development Phase (Phases B to D)
 Cost and Schedule updates
 Operation Phase (Phase E)
 Cost and Schedule Updates
 Use Cost Analytics in Each Phase
PRT 114 03 January 2012 Approved For Public Release 3 of 42

4. Project Life Cycle Conceptual View
Definition Phase Acquisition Phase Development Phase Operations Phase
Concept
Definition
Feasibility Measurement
Analysis Criteria
Project
Definition
Cost / Schedule Management
Calibration Decisions
Project
Implementation
Analysis
Tools Performance Management
Analysis Decisions
Project Analytics Data Store Project
Modification/
Sustaining
Project
Project
Management
Database Reporting Performance
Database
Tools Analysis
PRT 114 03 January 2012 Approved For Public Release 4 of 42

5. Infrastructure Goals
 Functionality
 Provide Cost and Schedule Data Management
 Provide Cost and Schedule Analysis Tools
 Provide Project Engineering support for Performance analyses
 Provide Program Management support for Schedule and Budget
excursion analyses
 Maintainability
 Support Easy Database Changes
 Support Easy Tool Additions
 Support Data Load/Unload
PRT 114 03 January 2012 Approved For Public Release 5 of 42

6. Infrastructure Options
 Infrastructure Options for Technology Solution
 Use a Web UI and MS/SQLServer
 Hosted on Network Servers
 Accessible to Authorized Users in NDC domain
 Use Desktop Application and Apache Derby Database
 Installed and Executed on a PC/Laptop
 Restricted to PC Users
 Uses Network Drives for inter-User communications
PRT 114 03 January 2012 Approved For Public Release 6 of 42

7. Web Infrastructure
Network
Servers MOP
Cost Analysis Data
Authenticated Database
IP Network / JSC VPN
Users
(2) Log into (3) File
(4) Create
Infrastructure to Uploaded into
(1) Prepare Cost Reports
upload DB
Data XLS File(s)
PRT 114 03 January 2012 Approved For Public Release 7 of 42

8. Desktop Infrastructure
Network Drive
Transfer/Reporting Medium
Embedded
Database User
Cost Analysis Data
(2) File (3) Export Cost
Uploaded into Data and
(1) Prepare Cost
DB Reports
Data XLS File(s)
PRT 114 03 January 2012 Approved For Public Release 8 of 42

9. Fundamental Cost Data
 QUESTION: What do you get when you… ?
 Identify EVERY Cost Driver and capture them in an Excel Spreadsheet
 Identify the Schedule Tasks associated with each Cost Item
 Separate different Cost Data Types into separate Tabs
 E.g. Material, Travel, Level of Effort, Software Development scope
 ANSWER: Something like…
CxP MOP Cost Analysis Data Requirements (CADRe)
 CADRe data is a NASA HQ requirement to demonstrate “credible
cost estimates”
 MOP delivered CADRe data to HQ up to MOP PDR
 A Normalized data format was used for the MOP CADRe data
suitable for storage in a relational database
 Tecolote developed a CADRe database on an MOD network server
PRT 114 03 January 2012 Approved For Public Release 9 of 42

10. Changing to Schedule Oriented Cost data
 MOP CADRe data was Cost Data allocated to Fiscal
Years
 When the Schedule changed the Cost data needed to be
changed to re-spread it across Fiscal Years to reflect the
Schedule changes
 Recently the CADRe data has been redesigned to
become Schedule Oriented
 Cost Items are associated with Schedule Tasks
 Changes to the Project Schedule automatically re-spread the cost
PRT 114 03 January 2012 Approved For Public Release 10 of 42

11. CADRe data
 What data is supported?
 Schedule Data
 MS/Project files can be used to supply schedule data
 All costs are associated with a Schedule Task
 Work Breakdown Structure (WBS) Data
 CADRe data can use ANY Project defined WBS
 All costs are associated with a WBS Item
 Travel Cost Data
 Material Cost Data
 Level of Effort Cost Data
 Software Development Cost Data
PRT 114 03 January 2012 Approved For Public Release 11 of 42

12. Schedule Data
Low Likeliest High
Schedule Task Name Start End Duration Predecessors % Complete Status Date Multiplier Multiplier Multiplier
MSProj11 10/3/2011 9/30/2020 2347 10% 15Jun2012 0.9 1 1.2
Kickoff 10/3/2011 10/3/2011 0 100% 15Jun2012 1 1 1
APA Development 10/3/2011 4/26/2013 410 62% 15Jun2012 0.9 1 1.2
APA Design 10/3/2011 10/21/2011 15 1 100% 15Jun2012 0.9 1 1.2
APA PBR 10/21/2011 10/21/2011 0 3 100% 15Jun2012 1 1 1
APA Implementation 10/24/2011 3/29/2013 375 63% 15Jun2012 0.9 1 1.2
APA User Interface 10/24/2011 3/29/2013 375 4 50% 15Jun2012 0.9 1 1.2
APA Constraints Checker 10/24/2011 11/2/2012 270 4 70% 15Jun2012 0.9 1 1.2
APA Database Manager 10/24/2011 11/9/2012 275 4 60% 15Jun2012 0.9 1 1.2
APA Schedule Manager 10/24/2011 8/24/2012 220 4 80% 15Jun2012 0.9 1 1.2
Subject Matter
Expert
PRT 114 03 January 2012 Approved For Public Release 12 of 42

13. WBS Data
WBS # WBS Name
1. Another Planning Application
1.1 APA Management
1.2 APA Development
1.2.1 APA Systems Engineering
1.2.2 APA Implementation
1.2.2.1 APA User Interface
1.2.2.2 APA Constraints Checker
1.2.2.3 APA Database Manager
1.2.2.4 APA Schedule Manager
1.2.3 APA Deployment
1.3 APA Sustaining
1.4 APA Modifications
PRT 114 03 January 2012 Approved For Public Release 13 of 42

14. Material Cost Data
Activity Material Purchase Unit Price FY of
WBS # WBS Name Type Type Material Description BOE Type $ price
1. Another Planning Application Dev COTS HW Development Server BOM #3 Buy $5,000 10
Single Unit
Config Num Spares Cost Low Likeliest High
Qty Configs Qty Assoc. Task Distribution Multiplier Multiplier Multiplier FY11 FY12
1 1 - APA Development Start 0.90 1 1.2 $0 $5,253
Schedule Task Name Start End Duration
MSProj11 10/3/2011 9/30/2020 2347
Kickoff 10/3/2011 10/3/2011 0
APA Development 10/3/2011 4/26/2013 410
PRT 114 03 January 2012 Approved For Public Release 14 of 42

15. Travel Cost Data
Num Daily Trip
Activity Num Num Days Per Lodging Travel BY of Total Cost
WBS # WBS Name Type People Trips /Trip Diem Cost Cost price (BY)
1. Another Planning Application Dev 2 4 3 $75 $150 $1,000 10 $13,400
BY of Cost Low Likeliest High
price Assoc. Task Distribution Multiplier Multiplier Multiplier FY11 FY12 FY13 FY14
10 APA Development Level 1 1.1 1.2 $0 $9,855 $5,783 $0
Schedule Task Name Start End Duration
MSProj11 10/3/2011 9/30/2020 2347
Kickoff 10/3/2011 10/3/2011 0
APA Development 10/3/2011 4/26/2013 410
PRT 114 03 January 2012 Approved For Public Release 15 of 42

16. Level of Effort (LOE) Data
Activity Level of
WBS # WBS Name Cost Type Type Assoc. Task Effort FY11 FY12 FY13 FY14 FY15
1.1 APA Management FTE Dev APA Development 0.5 - 0.50 0.29 - -
1.2.1 APA Systems Engineering EP Dev APA Design 4.0 - 0.20 - - -
1.2.3 APA Deployment EP Dev APA Deployment 4.0 - - 0.28 - -
1.3 APA Sustaining EP Sus APA Sustaining 2.0 - - 1.02 2.00 2.00
1.4 APA Modifications EP Mod APA Modification - - - - - -
Schedule Task Name Start End Duration
APA Development 10/3/2011 4/26/2013 410
APA Design 10/3/2011 10/21/2011 15
APA Deployment 4/1/2013 4/26/2013 20
APA Sustaining 3/29/2013 9/30/2020 1957
PRT 114 03 January 2012 Approved For Public Release 16 of 42

17. (SEER) Software Data
Cost Activity Size Newly Size Pre- % change to
WBS # WBS Name Type Type Units Developed exists Pre-exist
1.2.2.1 APA User Interface EP Dev KSLOC 70 - 0%
1.2.2.2 APA Constraints Checker EP Dev KSLOC - 81 10%
1.2.2.3 APA Database Manager EP Dev KSLOC - 50 10%
1.2.2.4 APA Schedule Manager EP Dev KSLOC - 90 10%
Software
Size % change COTS Type - # Separate Phase At
Reuse to Reuse Size Programs Estimate Language Application Type
- 0% None 2 Proposal Java Graphical User Interface
- 0% None 2 Proposal Java Expert System
- 0% None 2 Proposal Java Data Warehousing
- 0% None 2 Proposal Java Data Warehousing
Development Development ROM Size / Staff Assoc. Task for
Method Standard Staff Year Years Phase1 FY11 FY12 FY13 FY14
Incremental DO-178B Level C 4.0 8.75 APA User Interface - 5.74 3.01 -
RUP Full DO-178B Level C 4.0 3.43 APA Constraints Checker - 3.12 0.30 -
Spiral DO-178B Level C 4.0 1.83 APA Database Manager - 1.64 0.19 -
Spiral DO-178B Level C 4.0 3.70 APA Schedule Manager - 3.70 - -
Schedule Task Name Start End Duration
APA User Interface 10/24/2011 3/29/2013 375
APA Constraints Checker 10/24/2011 11/2/2012 270
APA Database Manager 10/24/2011 11/9/2012 275
APA Schedule Manager 10/24/2011 8/24/2012 220
PRT 114 03 January 2012 Approved For Public Release 17 of 42

18. Data Iteration
 Cost Estimation Events are used to record Multiple Data
Snapshots in the Database
 MCR, SDR, PDR, CDR, etc.
 Quarterly Baseline Reviews
 Monthly Status Reports
 Weekly Project Reviews
 Multiple data snapshots provide more opportunities for
Cost Analysis
 Performance analyses from comparing two snapshots
 Schedule Performance Analyses
 Cost Performance Analyses
 Trending analyses
 Changes in scope over time
 Changes in Cost Estimate over time
 Changes in Schedule over time
PRT 114 03 January 2012 Approved For Public Release 18 of 42

19. Basic Cost Analytics
Requirements
+ Design
Effort Cost Loaded
Work to be done Schedule
(i.e. estimated labor + A plan to do the work
material cost)
Technology
Productivity
Metrics
Resources
Cost Profiles
Cost analytics
Budget
06 07 08 09 10 11 12 13 14
PRT 114 03 January 2012 Approved For Public Release 19 of 42

20. Extended Cost Analytics
Requirements
+ Design Total Project Cost
(w/ LOE)
Effort Cost Loaded p
Work to be done Schedule
(i.e. estimated labor + A plan to do the work
material cost)
Technology
$
Productivity Project Risks
Metrics
Resources
Emperical
Heuristics Cost Profiles Project
External Cost analytics Completion Date
Events
Budget Utilization
Capacity to do
work
06 07 08 09 10 11 12 13 14
= Source of Uncertainty
PRT 114 03 January 2012 Approved For Public Release 20 of 42

21. Tools for Cost Analytics
 SEER-SEM for Software Estimation
 ACEIT for Cost Estimation
 ACEIT for Cost and Schedule Confidence Level
Analyses
 Schedule Comparison for Schedule Performance
Analyses
 Cost Profile Comparison for Cost Performance Analysis
PRT 114 03 January 2012 Approved For Public Release 21 of 42

22. Cost Analytics in the Project Lifecycle
 Definition Phase (Pre-Phase A)
 Project Business Case Justification
 Acquisition Phase (Phase A)
 Detailed Budget Analyses
 Development Phase (Phases B to D)
 Development Cost and Schedule Performance Analyses
 Operation Phase (Phase E)
 Operations Cost and Schedule Performance Analyses
PRT 114 03 January 2012 Approved For Public Release 22 of 42

23. Process/ Data Flow Architecture
Data Mined from
Project Business
Development Project
Case
Process Dashboard
Project Scorecard
Updated Cost / Project
+ Detailed Cost / Heuristic Data /
Schedule Data Analytics
Schedule Data Metrics
Database
Planning Data Multi-
Budget Data
Updates Project
Dashboard
Pre-Phase A Phases A to E
PRT 114 03 January 2012 Approved For Public Release 23 of 42

24. Data Collection - Project Definition
 Primary Focus is the “Business Case”
 Use Business Case Surveys
 Use Business Case Scorecard
 Use CADRe Detailed Cost Data
 Collect Risk (Threat) Data
 Collect “Replaceable” Costs
 Costs that will occur if Project is NOT performed
 Costs that will be replaced if Project IS performed
 Create Business Case Summary from input data
 Summarize detailed cost/schedule data
 Derive ROI data
PRT 114 03 January 2012 Approved For Public Release 24 of 42

25. Business Case Data
Quantitative Cost Savings ROI/NRI after 10 years $381,000
Balanced Project Scorecard
Category Rating Score
 Status items created include Reuse
Extensibility
>80%
Med
27
9
Multiple
Future Software Cost Avoidance
 Project Scorecard Reusability
Flexibility
Groups
Significant
9
15
Standards Compliant 27
Ops Need Mandatory 45
 Cost Profile Qualitative
Ops Scope
Multiple
Groups 9
Customer Satisfaction
$ Avoidance Med 15
 Savings (ROI) profile Accessibility Some 12
Security Low 18
Integration Complexity
 Project Schedule/ Milestone Chart TouchPoints Med 6
192
Qual Score
Context 56%
PRT 114 03 January 2012 Approved For Public Release 25 of 42

26. Data Collection - Project Development
 Cost and Schedule Data is collected and stored throughout the
Project Lifecycle
– Capture the pre-kickoff baseline data
– Establish reporting cycles for capturing
data iterations
– Process Data for status and performance
analyses
PRT 114 03 January 2012 Approved For Public Release 26 of 42

27. Project Schedule Status
 Schedule data captured in database
 Summary Status charts created for Project Dashboard
PRT 114 03 January 2012 Approved For Public Release 27 of 42

28. Project Cost Performance
 Shows Planned vs. Actual Cost Performance
 Burn-down Chart shows Baseline Cost-to-complete over time and
Actual Costs
PRT 114 03 January 2012 Approved For Public Release 28 of 42

29. Project Schedule Performance Metrics
Percent Start Finish Old Projected
Name Start Finish Duration Complete Slip Slip PDWS PDWP ADWP SV SPI SCPI Duration Duration
Task 1 1-Jan-00 12-Jan-00 12 100% 0 2 10 10 12 0 1 0.834 12 12
Task 2 5-Jan-00 14-Jan-00 10 80% 0 0 10 8 10 -2 0.8 0.8 10 10.5
Task 3 8-Jan-00 17-Jan-00 10 70% 0 0 8 7 8 -1 0.875 0.875 10 10.43
Task 4 14-Jan-00 18-Jan-00 5 0% 0 0 2 0 2 -2 0 0.715 5 7
 Provides Performance Indices for Schedule Tasks much like EVM
indices
 Compares two iterations of the Project’s schedule
 Measures actual progress against planned progress during interval
between schedules
 Identifies Schedule items that need attention
 Provides Projected Task Duration based on measured performance
EXAMPLE
PRT 114 03 January 2012 Approved For Public Release 29 of 42

30. Project Excursions (What-Ifs)
 Program Management must handle many pressures to control
projects due to
 Budget Cuts
 Project Overruns
 Multiple Projects Competing for limited Funds
 Cost Analytics must support
 Schedule Excursions
 Budget Constraint Excursions
PRT 114 03 January 2012 Approved For Public Release 30 of 42

31. Web Framework Status
 There is a CxP MOP CADRe Database on an MOD Network Server
 The Web UI to the MOP CADRe DB can be used to Import and
Export data
 CxP MOP users can request access to the Web UI
 CxP MOP users can be authorized to access to the Web UI
 The SEER-SEM tool runs on user request to create Software
Development Estimates
 The new Database Schema for Schedule Based Cost Data has been
defined
PRT 114 03 January 2012 Approved For Public Release 31 of 42

32. Desktop Framework Status
 Developed Primarily as an Analytics Tool Development Platform
 The UI supports multiple database instances
 Tools exist to
 Import Schedule Based Cost Data
 Export Cost Data Spreadsheets
 Update CADRe data from Project Schedules
 Create Schedule Performance Status
 Create a Milestone Excursion ACEIT model data
 Perform Budget Constraint Analyses
PRT 114 03 January 2012 Approved For Public Release 32 of 42

33. Thank You
Tony Harvey (aharvey@tecolote.com)
PRT 114 PRT 114 03 2012
03 January January 2012 Approved For Public Release 33 of 42

34. Backup
 CADRe Submission to HQ
 Detailed Process Flow Charts
 Multi-Layered Management Concept
 Detailed Business Case Results
PRT 114 03 January 2012 Approved For Public Release 34 of 42

35. When is CADRe Data Required?
NASA HQ requires CADRe submissions 5 times in a project’s life
Program
Phases
Formulation Implementation
Flight Projects Pre-Phase A: Phase A: Phase B: Phase C: Phase D: Phase E:
Phase F:
Life Cycle Concept Concept Fabrication, Operations &
Disposal
Phases Studies Development Preliminary Design Detailed Design Assembly & Test Sustainment
SDR PDR CDR Launc
Traditional h
Waterfall
Development
or Directed 1 1a 2 2a 3 4 5
Missions
Down
AO-Driven Select CR
Select Step 2
Projects Step 1
1 2 2a 3 4 5
Legend
GPMC Mission Decision Review/ICR All parts of CADRe due 30 days
2
4 CADRe, All Parts
after site review
1
All parts of CADRe due 30 days 90 days after launch,
after site review as built or as deployed
2a CADRe update, if necessary configuration
1a CADRe update, if necessary
5 CADRe, Part C only
3 Update as necessary during
CADRe delivered; based on 30 days after CDR
11
last year of planned
Concept Study Report (CSR)
project life
and winning proposal
PRT 114 03 January 2012 Approved For Public Release 35 of 42

36. Project Process Details
PRT 114 03 January 2012 Approved For Public Release 36 of 42

37. Program Management Detail
PRT 114 03 January 2012 Approved For Public Release 37 of 42

38. Multi-Layered Management Concept
PRT 114 03 January 2012 Approved For Public Release 38 of 42

39. Project Cost Profile
 Provides statement of Resource Requirements over time
 Total Cost per BCC
 By Tasks using WBS
 Shows Cost Breakdown over time
 Collect CADRe Data at Detailed Level
 Identify Summary WBS Items in Detailed Data
 By Activity Type
 Shows cost separated by Development, Modification, and Sustaining
$2,000
Source Activity
Data Type Type Cost Type FY10 FY11 FY12 FY13 FY14 FY15 $1,800
SW Dev FTE - - - - - -
$1,600
SW Mod FTE - - - - - -
SW Sus FTE - - - - - - $1,400
SW Dev EP 17.49 8.72 - - - -
$1,200
SW Mod EP - 3.87 7.74 7.71 3.85 -
SW Sus EP - 0.51 1.03 1.03 0.51 - $1,000
RY$, M
LOE Dev FTE 1.00 0.50 - - - -
LOE Mod FTE - - - - - - $800
LOE Sus FTE - 1.00 2.00 2.00 1.00 -
$600
LOE Dev EP - - - - - -
LOE Mod EP - - - - - - $400
LOE Sus EP - 5.00 10.00 10.00 5.00 -
$200
Trav Dev Travel $319.91 $165.10 $0.00 $0.00 $0.00 $0.00
Trav Mod Travel $0.00 $1,807.01 $0.00 $0.00 $0.00 $0.00 $0
Trav Sus Travel $0.00 $0.00 $0.00 $0.00 $1,736.13 $0.00 FY07 FY08 FY09 FY10 FY11 FY12 FY13 FY14 FY15 FY16 FY17 FY18 FY19 FY20
Mat Dev Material $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 Task I Task II Task III Task IV Task V
Mat Mod Material $0.00 $0.00 $0.00 $0.00 $0.00 $0.00
Mat Sus Material $0.00 $0.00 $0.00 $0.00 $0.00 $0.00
PRT 114 03 January 2012 Approved For Public Release 39 of 42

40. ROI Chart
 Shows Costs vs. Savings
 Per BCC
 Calculate Savings From
 CADRe Detailed Cost Data
 “Replaceable” Costs
 Costs that will occur if Project is NOT performed
 Costs that will be replaced if Project IS performed
 As Costs change ROI can be Re-Calculated
PRT 114 03 January 2012 Approved For Public Release 40 of 42

41. Project Scorecard
Quantitative Cost Savings ROI/NRI after 10 years $381,000
 Provides Qualitative Balanced Project Scorecard
Category Rating Score
Rating Reuse >80% 27
Extensibility Med 9
 Final Score can be Future Software Cost Avoidance
Multiple
Reusability Groups 9
used for Option Flexibility Significant 15
Standards Compliant 27
comparison Qualitative
Ops Need Mandatory 45
Multiple
 Identical to current Customer Satisfaction
Ops Scope
$ Avoidance
Groups
Med
9
15
BCC scorecard Accessibility Some 12
Security Low 18
 With textual rating Integration Complexity
TouchPoints Med 6
192
drop-down lists Qual Score
Quantitative Cost Savings 10 yr ROI $381,000 $381,000
Context $381,000 56%
 Data provided by Qualitative Criteria Option 1 Option 2 Option 3
Project team and Qualitative
Scores
Future Reuse
Software Cost Extensibility
>80%
Med
< 10%
High
30-40%
Low
stored in DB Avoidance Reusability
Flexibility
Multiple Groups
Significant
Limited Use
Complete
Throughout MOD
Little
Standards Compliant Non-standard Little
Customer Ops Need Mandatory Nice to have Big benefits
Satisfaction Ops Scope Multiple Groups Limited Use Throughout MOD
$ Avoidance Med None Low
Accessibility Some Restricted Not restricted
Integration Security Low High Low
Complexity TouchPoints Med Med Med
Numerical Score 192 57 141
Qual Score Context 56% 17% 41%
PRT 114 03 January 2012 Approved For Public Release 41 of 42

42. Project Schedule/ Milestone Chart
 Provides Summary Schedule Data
 Schedule Data is collected in CADRe data
 CADRe data can be updated from Project Schedule
 CADRe data identifies Summary Schedule items
 Milestone Chart created from Project Schedule data held in the
CADRe data
PRT 114 03 January 2012 Approved For Public Release 42 of 42