Dev Dives: Streamline document processing with UiPath Studio Web
Gonzalez.matthew
1. Planning and Conducting an
Integrated Baseline Review (IBR)
Matthew T. Gonzales, Project Business Manager
Jet Propulsion Laboratory
818-354-3629
Used with Permission Pg 1
3. Organization
‘If you can't describe what you are doing as a process, you don't know what you're doing.’
W. Edward Deming
Pg 3
4. IBR as a Process
Pre-Phase A emphasizes the Phase A emphasizes the EV Phase B emphasizes the Phase C emphasizes use of EVM Phase D emphasizes use of EVM
contractors and projects readiness Process and Initial Baseline validity of baseline to complete and the validity of EAC to and the validity of EAC to
for EVM Plan work complete work complete work
Formulation Implementation
Pre-Phase A Phase A Phase B Phase C Phase D
Project Emphasis: Project Emphasis: Project Emphasis: Project Emphasis: Project Emphasis:
- System Definition - Complete System Definition - Project Requirements - Complete Flight and Ground - Integration of Flight HW
- Identify potential contractors - Establish Requirements Baselined Designs and SW
and teammates - Select Contractors - Preliminary Design - Conduct component and SS - Integrate Payload and
- Identify contractors above - Award Contracts with EVM Established CDRs Launch Vehicle
$20M DRDs - LL Procurements Let - Build BB and EMs
- Initiate Acceptance Test
Program for Flight HW
EVM Emphasis: EVM Emphasis: EVM Emphasis: EVM Emphasis: EVM Emphasis:
- EVM implementation team - Contractor teams develop - EV System is mature - Implement EV - Implement EV
defined initial project cost and schedule - Reports Submitted - Project Office conducts IBR with - Project Office conducts IBR with
- Contractors develop EVM baseline - Logs Established emphasis on Risk to Reserves emphasis on Risk to Reserves
plan - Contractors submit EV reports - Change Control Exercised - Does Project Plan support - Does Project Plan support I&T
- Progress Assessment Visit per DRDs - Project Office conducts IBR with detail design Program
(PAV); Contractor presents - Project Office conducts IBR emphasis on Risk to Reserves - Is EV data reliable as a - Is EV data reliable as a predictor
EVM plan with emphasis on Process - Project Plan content predictor - Quality of VARs
- EVM team evaluates - Compliance to EV Process - Completeness of Planning - Quality of VARs - Is the project using EVM data to
- If Validation Review is - Data Quality - Change control process - Is the project using EVM data manage
required, EVM teams works - CAM Knowledge - Basis of Estimate for Cost and to manage - Test EAC process
with local DCMA - Change Control Schedule - Test EAC process - Quality BOEs
- Conduct follow-up PAV - Validity of Baseline relative - Can design be built with - Quality BOEs - Can I&T be completed for
- Support KDP A to Scope of Work planned resources - Can HW be built for estimated cost
- Support KDP B - Overall robustness of EVM estimated cost - Is there sufficient schedule
system - Is there sufficient schedule - Are Risks known and mitigated
- Reconciliation to Phase A - Are Risks known and - Reliable reporting
Baseline mitigated - Support KDP E
- Support KDP C - Reliable reporting
- Support KDP D
Pg 4
7. Impetus
Impetus for an Integrated Baseline Review
Comply with customer requirements (7120.5d)
Significant change to system design
New Launch Date
‘Grass roots’ EAC Developed
Over Target Baseline incorporated
Pg 7
8. Planning
In preparing for battle I
have always found that
plans are useless, but
planning is indispensable.
Dwight D. Eisenhower
Pg 8
9. Planning Process Flow
Timeline Events Process
Project IMS restructured for Project IMS resources Project Plan approved
Re-plan new launch date and estimated and loaded by SMD PMC.
design change into Turbo for pricing Authorized to Proceed.
EV Turbo estimates MS Project milestones Project integrated
Database imported into MPM. imported in MPM. baseline complete.
Complete PMB established. Weight applied. EV Set ‘Over Target
Jul ‘06 Plan Set Baseline’
1st Schedule updated on Update to MPM Generate and submit
Month weekly cadence. database. CPR Format 1 in
EV update Weekly metrics ACWP and BCWP for support of customer
cycle implemented. July month end MMR
Review EV metrics Corrective actions Changes to S/C and
& analysis reveals assigned/reviewed. Mtl plans and EV
Aug ‘06
Analysis planning Implementation approved methods complete
anomalies. by PO and implemented prior to Aug close.
Pg 9
10. Planning Process Flow
Timeline Events Process
2nd Submit CPR Format 1. Corrective actions IBR date negotiated
Month Planning changes closed. EV and VARs with all affected
EV update incorporated. EV reviewed at project parties. Set for Nov.
cycle metrics improve. MMR
Sep ‘06
Project office IBR scope,
Contractor, HQ/DPO
Project generates and briefs schedule and team
and project team
CDR IBR information membership
briefed on IBR plan
package. finalized.
Oct ‘06
IBR Coordination Day 1 -IBR In-Brief
Update to MPM Day 2 – Data Review
meeting with Review JPL database.
Project Office Review team audits
Team week before BATC EV Surveillance
ACWP and BCWP for CAM notebooks.
Project review BATC Project end
July month Office Team caucus.
IBR
EV metrics
Day 3 – CAM Interviews Day 4 – Data Review/
analysis reveals
Project Technical Lead Informal Out-brief Formal
Nov ‘06
planning
conducts interview. All team Review Survey Results Out-brief
anomalies.
members complete survey 3-4 slide contractor feedback
Pg 10
11. Planning
Plan for your review
Anticipated KDP
Project Phase
Working with contractors
Define IBR as early as possible
Scope, Goals, Purpose
Team membership with roles and responsibilities
Required documents for review
Work with contractor on logistics
Communicate
3 briefings in preparation for review
Need to train interviewers as well as interviewees
Provide contractor with questions ahead of time
Pg 11
13. John Wooden: A Coaching Legend
UCLA's basketball program has the international reputation of being No. 1. There is a major reason
for that his name is John Robert Wooden, who announced his retirement after the 1974-75 season
(his 27th campaign) as the Bruins' head coach with the winningest record in all of the sport's
history. Wooden celebrated his 92nd birthday on Oct. 14, 2002.
Wooden concluded his 40 years as a head coach that season and his 885-203 overall career win-
loss record (a percentage of .813) is unequaled. A large part of that success was at UCLA. In 27
years as Bruin coach, his teams registered 620 wins, and only 147 losses while earning far more
national honors than any other university.
Under Wooden, UCLA won an unprecedented 10 NCAA championships, including seven
consecutive (1966-73). Included in the string is one of the most amazing win streaks in all of
sports, 38 straight NCAA tournament victories.
In addition, there is the all-time NCAA consecutive winning-streak record of 88 games over four
seasons, which included consecutive 30-0 seasons in 1971-72 and 1972-73. UCLA also won 149
of 151 games in Pauley Pavilion during his Bruin tenure.
John Wooden is the only coach to compile four undefeated seasons of 30-0 and his Bruin teams
captured 19 conference championships (the record of which Wooden is most proud).
Coach Wooden is the first person to be inducted into the National Basketball Hall of Fame as
both a player and coach
Pg 13
14. John Wooden Quotes
“It’s the little details that are vital. Little things make big things happen”
“If you don’t have time to do it right, when will you have the time to do it over”
“Winning takes talent, to repeat it takes character”
“It is what we learn after we know it all that really counts”
“Be prepared and be honest”
“Things turn out best for the people who make the best of the way things turn
out”
Pg 14
17. Data Review - Wed, Nov. 15th 8am – 5pm
• Control Account Managers Notebooks
– Turbo reports • Reviewing for completeness
– Schedules , by WBS element and by IPT and consistency in the plans.
– Control Account Plans to Work Package level • Is the work fully accounted for
of detail and Earned Value Milestone Report and plan appropriately.
– Current WBS and WBS Dictionary • Are all resources planned and
available when required.
– Estimating methodology and documentation
to enable assessment of the basis of the • Is the time phasing of cost and
estimates schedule consistent.
– Work Authorization Documents for authorized • Is EV planned in a way to yield
control accounts. accurate results.
– Performance Measurement Reports (CPR • Are analysis tools available
Format 1) and utilized.
– Budget Change Requests (if applicable) • Are changes tracked and
managed
– Approved liens and KVRs and recovery plans
• Are they adhering to the BATC
• Other Ball documentation required for the IBR: way of planning and managing
– Current Contract SOW projects.
– BATC System Description, Kepler EVM Plan
Supplement
Pg 17
19. Conducting Interviews
Select WBS elements for review
Risk driven (technical and/or programmatic)
Phase specific
Select appropriate questions
Phase specific
Basis of Estimate
Discern risk
Communicate early
Develop interview teams
Led by technical representative
Small teams
Provide training on questions
Discussions/Interviews
Conversational
~ 2 hours
Document results
Pg 19
20. CAM Interviews – Nov. 16th 8am – 5pm
IBR CAM Interview Schedule
DRAFT WBS WBS Description Day Interview Time
6.1.1 FS Management Thurs 08:00 – 10:00
5.2.6 Attitude Control Thurs 08:00 – 10:00
5.2.8 Communications Thurs 08:00 – 10:00
6.1.3 FS Mission Assurance Thurs 10:00 – 12:00
6.3.2 FPAA Thurs 10:00 – 12:00
6.3.3 Electrical Thurs 10:00 – 12:00
6.3.4 Telescope Thurs 1:00 – 3:00
5.2.3 Mechanical Subsystem Thurs 1:00 – 3:00
6.3.5 Photo Integration Test Thurs 3:00 – 5:00
5.2.11 Spacecraft I&T Thurs 3:00 – 5:00
5.3 System ATLO
IBR CAM Interview Schedule Technical Leads NASA Reps Discovery Program Office Support
Peg Leslie Jim Sarah Michael Renee Pam Jack Steve Nate John Robert Dan Maura Bill Tim Cal Joe
WBS WBS Description Frerking Livesay Fanson Gavit Moore Leck Marcum Graham McClard Porter Keidel Bitten Graham Fujieh Epping Clark Cambers Fischetti
6.1.1 FS Management 1 1 1 1
5.2.6 Attitude Control 1 1 1 1
5.2.8 Communications 1 1 1 1
6.1.3 FS Mission Assurance 1 1 1 1
6.3.2 FPAA 1 1 1 1
6.3.3 Electrical 1 1 1 1
6.3.4 Telescope 1 1 1 1
5.2.3 Mechanical Subsystem 1 1 1 1
6.3.5 Photo Integration Test 1 1 1 1
5.2.11 Spacecraft I&T 1 1 1 1
5.3 System ATLO
Pg 20
21. CAM Questionnaire
Questions Rating Low Med High
1.What is the Manager’s scope of work? Does the Manager know what scope of work is
his/her responsibility? Cost 1 2 3 4 5
–The Manager should be able to show the team a Statement of Work (SOW) paragraph, a WBS,
PWBS, or CWBS narrative, and a Work Authorization Document that correlates to the task. Schedule 1 2 3 4 5
Comments:
Technical 1 2 3 4 5
2.How did the Manager plan the scope of work into control accounts?
Cost 1 2 3 4 5
–The SOW defines the effort. The WBS, PWBS, or CWBS provides specifics, such as work
definition. The work authorization and change documentation should show information such as the Schedule 1 2 3 4 5
dollars/hours, period of performance, and description of the scope of work and any changes.
Comments:
Technical 1 2 3 4 5
3.How did the Manager ensure that all elements of the scope of work were planned?
Cost 1 2 3 4 5
–The Manager should be able to show the team the work packages breaking down the scope of
work, and the budget associated with each work package. The sum of the work packages should be Schedule 1 2 3 4 5
the control account budget. Each control account should contain a specific scope of work.
Comments:
Technical 1 2 3 4 5
Pg 21
23. Caucus and Review
Review survey responses
By interview, by question
Look for outliers and discuss.
Opportunity to share thoughts and observations
Normalize metrics
Do metrics support subjective risk observations
Pg 23
24. Caucus and Review
Low Med High
12 36 60 Risk Rating
WBS CAM Intvr No Questions Cost Sched Tech Total
5.2.6 CS SG 1 1.What is the Manager’s scope of work? Does the Manager
1 2 3 6
know what scope of work is his/her responsibility?
5.2.6 CS DG 1 1.What is the Manager’s scope of work? Does the Manager
1 1 2 4
know what scope of work is his/her responsibility?
5.2.6 CS RB 1 1.What is the Manager’s scope of work? Does the Manager
1 1 1 3
know what scope of work is his/her responsibility?
5.2.6 CS BE 1 1.What is the Manager’s scope of work? Does the Manager
1 1 1 3
know what scope of work is his/her responsibility?
Totals 4 5 7 16
5.2.6 CS DG 2 2.How did the Manager plan the scope of work into control
1 1 1 3
accounts?
5.2.6 CS SG 2 2.How did the Manager plan the scope of work into control
1 1 1 3
accounts?
5.2.6 CS RB 2 2.How did the Manager plan the scope of work into control
1 1 1 3
accounts?
5.2.6 CS BE 2 2.How did the Manager plan the scope of work into control
1 1 1 3
accounts?
Totals 4 4 4 12
5.2.6 CS DG 3 3.How did the Manager ensure that all elements of the scope of
2 2 2 6
work were planned?
5.2.6 CS SG 3 3.How did the Manager ensure that all elements of the scope of
1 1 1 3
work were planned?
5.2.6 CS RB 3 3.How did the Manager ensure that all elements of the scope of
1 1 1 3
work were planned?
5.2.6 CS BE 3 3.How did the Manager ensure that all elements of the scope of
1 1 1 3
work were planned?
Totals 5 5 5 15
5.2.6 CS BE 4 4.How did the Manager obtain the resources for assigned work?
3 3 3 9
5.2.6 CS RB 4 4.How did the Manager obtain the resources for assigned work?
2 2 1 5
5.2.6 CS DG 4 4.How did the Manager obtain the resources for assigned work?
1 1 1 3
5.2.6 CS SG 4 4.How did the Manager obtain the resources for assigned work?
1 1 1 3
Pg 24
29. Risk Rating by WBS
Risk Rating
WBS No Questions Cost Sched Tech Total
6.1.1 1 1.What is the Manager’s scope of work? Does the Manager know what
1.33 1.33 1.40 3.83
scope of work is his/her responsibility?
6.1.1 2 2.How did the Manager plan the scope of work into control accounts?
1.17 1.33 1.20 3.50
6.1.1 3 3.How did the Manager ensure that all elements of the scope of work
1.20 1.40 1.20 3.17
were planned?
6.1.1 4 4.How did the Manager obtain the resources for assigned work? 1.33 1.33 1.00 3.33
6.1.1 5 5.Ask the Manager to describe the resource development process. 1.40 1.17 1.00 3.00
6.1.1 6 6.How did the Manager come to an agreement of the scope, hours,
1.50 1.33 1.00 3.50
dollars and schedule?
6.1.1 7 7.Ask the Manager to show the team the schedule milestones used in the
planning of the control accounts. The Manager should discuss: 1.00 1.00 1.00 2.83
6.1.1 8 8.Ask the Manager how the work was time phased to achieve the
1.40 1.33 1.00 3.17
schedule.
6.1.1 9 9.Ask the Manager to identify any known risks that may affect the plan.
1.67 1.17 1.20 3.83
6.1.1 10 10.Are the Manager's resources adequately planned and correctly time
phased to meet the plan? Progress of each plan should relate to earned 1.33 1.17 1.00 3.33
value
6.1.1 11 11.Ask the Manager why he/she chose a particular type of earned value
1.33 1.33 1.40 3.83
method?
6.1.1 12 12.Does the earned value method used make sense for the type of work
being performed? Can you measure performance? Is the Manager using
milestones to measure accomplishments? Does the Manager use the 1.20 1.00 1.00 2.50
LOE method; and is the LOE in support of measurable
6.1.1 13 13.How does the Manager manage his/her plan? 1.50 1.50 1.20 4.00
6.1.1 14 14.How is the baseline maintained? 2.00 1.80 1.80 4.67
6.1.1 15 15.What is the project’s risk management process? How is risk
1.50 1.50 1.50 3.00
incorporated into the EAC?
6.1.1 16 16.What is the process for reviewing and updating EACs? 2.00 1.75 1.50 3.50
6.1.1 17 17.How does the CAM update the schedule? How often are schedules
1.67 1.67 1.00 4.33
updated?
FLIGHT SEGMENT MANAGEMENT Total Scores by WBS 1.44 1.36 1.20 3.49
Pg 29
30. Risk Rating
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.Ask the Manager
to identify any
9
known risks that
may affect the plan.
15.What is the
project’s risk
15
management
process? How is
16.What is the
process for
16
reviewing and
updating EACs?
7.Ask the Manager
to show the team
7
the schedule
milestones used in
4.How did the
Manager obtain the
4
resources for
assigned work?
10.Are the
Manager's
10
resources
adequately
6.How did the
Manager come to
6
an agreement of
the scope, hours,
13.How does the
Manager manage
13
his/her plan?
5.Ask the Manager
to describe the
5
resource
development
12.Does the
12 earned value
method used make
sense for the type
14.How is the
baseline
14
maintained?
8.Ask the Manager
how the work was
8
time phased to
achieve the
Risk Rating by Question
2.How did the
Manager plan the
2
scope of work into
control accounts?
11.Ask the
Manager why
11
he/she chose a
particular type of
3.How did the
Manager ensure
3
that all elements of
the scope of work
1.What is the
Manager’s scope
1
of work? Does the
Manager know
Pg 30
17.How does the
CAM update the
17
schedule? How
often are
31. Risk Rating by Question
Risk Rating
No Questions Cost Sched Tech Total
9 9.Ask the Manager to identify any known risks that may affect the plan. 2.44 2.40 2.24 6.86
15 15.What is the project’s risk management process? How is risk incorporated into the EAC? 2.43 2.45 2.24 6.69
16 16.What is the process for reviewing and updating EACs? 2.20 2.07 1.75 5.78
7 7.Ask the Manager to show the team the schedule milestones used in the planning of the control
1.82 1.99 1.43 5.15
accounts. The Manager should discuss:
4 4.How did the Manager obtain the resources for assigned work? 1.83 1.90 1.43 5.10
10 10.Are the Manager's resources adequately planned and correctly time phased to meet the plan?
1.87 1.72 1.49 4.96
Progress of each plan should relate to earned value
6 6.How did the Manager come to an agreement of the scope, hours, dollars and schedule? 1.85 1.72 1.55 4.88
13 13.How does the Manager manage his/her plan? 1.68 1.61 1.50 4.78
5 5.Ask the Manager to describe the resource development process. 1.72 1.66 1.42 4.59
12 12.Does the earned value method used make sense for the type of work being performed? Can you
measure performance? Is the Manager using milestones to measure accomplishments? Does the
1.76 1.61 1.30 4.58
Manager use the LOE method; and is the LOE in support of measurable
14 14.How is the baseline maintained? 1.84 1.70 1.50 4.58
8 8.Ask the Manager how the work was time phased to achieve the schedule. 1.64 1.70 1.44 4.52
2 2.How did the Manager plan the scope of work into control accounts? 1.60 1.52 1.37 4.48
11 11.Ask the Manager why he/she chose a particular type of earned value method? 1.64 1.54 1.30 4.44
3 3.How did the Manager ensure that all elements of the scope of work were planned? 1.57 1.59 1.38 4.42
1 1.What is the Manager’s scope of work? Does the Manager know what scope of work is his/her
1.45 1.38 1.30 4.08
responsibility?
17 17.How does the CAM update the schedule? How often are schedules updated? 1.58 1.58 1.31 4.06
Pg 31
