1. Project Management
Challenge 2007
Dave Lengyel
NASA Exploration Systems Mission Directorate
NASA Exploration Systems Mission Directorate
Integrated Risk and Knowledge Management
Integrated Risk and Knowledge Management
February 6, 2007
February 6, 2007

2. Outline
• Why Integrate Risk and Knowledge Management?
• Lessons Learned on Lessons Learned
• ESMD Integrated Risk and Knowledge Management Practices
– Pause and Learn
– Knowledge-Based Risks
– Communities of Practice
– Knowledge-Sharing Forums
– Experience-Based Training
• Summary
IRKMO001_hq20061219awh 2

3. Why Integrate Risk and Knowledge Management?
Designing a complex architecture of hardware, software, ground and
space-based assets to return to the Moon and then go on to Mars will
require:
1) an effective strategy to learn from past lessons, and
2) a set of processes to generate and share knowledge for reuse as we
progress forward. ESMD risk and knowledge management communities
have embarked on an effort to integrate risk and knowledge management
(KM) over the lifecycle of the Constellation and Advanced Capabilities
Programs using a set of inter-related strategies, which include:
Practice 1: Establish Pause and Learn Processes
Practice 2: Generate and Infuse Knowledge-Based Risks (KBRs)
Practice 3: Establish Communities of Practice (CoP)
Practice 4: Provide Knowledge Sharing Forums
Practice 5: Promote Experienced-Based Training
IRKMO001_hq20061219awh 3

4. Practice 1: Pause and Learn (PaL)
“The Need to Pause, Reflect, and Learn” PAL is modeled after the Army After
Action Review (AAR) system by
Dr. Ed Rogers KM Architect at the GSFC.
While the AAR was developed to learn
from training exercises, the 25 years of
experience, theoretical foundations and
practical tools make it a valuable source
of lessons for NASA.
The idea is to create a learning event
at the end of selected critical events in
the life of a project. End of project
reflections are good but are too infrequent
for the organization to learn in a timely
manner.
PAL meetings are intended to be
integrated into the project life cycle at key
points as a natural part of the process.
PAL meetings
are structured and facilitated by
specialists who are not project members
IRKMO001_hq20061219awh 4

5. Pause and Learn (PaL) Process
Facilitator & Facilitator, Note-taker & Facilitator &
Team Lead Entire Team Team Lead
Investigation Phase Facilitation Phase Documentation Phase
Research Literature Begin
Non-attribution Review Non-attribution
(Project Websites,
Discussion Notes
MSRs, Etc.)
Discussion Points Create Hotwash Report
Interview Stakeholders
Facilitator and Note-taker
(Projects Leader, Introduction
Project Members, etc.) Ground Rules and
Expectations Send Hotwash Report
Brief High-Level Overview to Project Manager
of Project
Develop Areas of
Facilitated Discussion
Discussion Follow-up meeting with
Facilitated Review of Major
Project Manager
Discussion Points
Discuss Future of Hotwash
Report
Evaluate PaL Process
IRKMO001_hq20061219awh 5

6. Lessons Learned on Lessons Learned
• Start Early
• Need to Capture, Learn From and Repeat Successes--Need to Learn from and
Prevent Failures, Mishaps, Near Misses
• There was a limited number of useful lessons learned in the NASA Lessons
Learned Information System database. The good ones are masked by the
hundreds of poor ones, so that extensive effort is required to sort them out.
• Lesson Learned – Well-understood mechanisms for “transfer of knowledge”
during Program development are crucial to a successful long-term Program.
• Flow all applicable Lessons Learned into Requirements, Processes, and Plans
and Periodically Verify Compliance. Institutionalize the Use of Lessons Learned.
• Provide Sufficient Resources, Planning, and Management Support to Analyze
and Incorporate Lessons Learned. NASA and Contractor Must Work Together
• The best lessons learned for running a major program should be captured in a
living handbook of best practices. New lessons learned should be screened for
applicability, and included in the handbook.
Why ESMD Is Taking a New Approach to Lessons Learned?
Why ESMD Is Taking a New Approach to Lessons Learned?
IRKMO001_hq20061219awh 6

7. Practice 2: Knowledge-Based Risks
Definition
Knowledge-Based Risk n. 1. A risk based
on lessons learned from previous experience.
2. A closed risk with documented lessons
learned appended. 3. A means of
transferring knowledge in a risk context.
IRKMO001_hq20061219awh 7

8. Knowledge-Based Risks Strategy
The ESMD KBR strategy is intended to convey risk-
related lessons learned and best practices to ESMD
personnel. This strategy integrates the existing
Continuous Risk Management (CRM) paradigm used
at NASA with knowledge management--with the
primary focus on integrating transfer of knowledge
through existing work processes and not adding an
additional burden to the workforce to incorporate
new KM tools and concepts.
IRKMO001_hq20061219awh 8

9. Knowledge-Based Risks
Active Risk
Manager®
(ARM)
Database
Knowledge-Based Risks
Knowledge-Based Risks ESMD Risk List
ESMD Risk List
- Orbital Space Plane
- Orbital Space Plane
- XXX
- XXX
- Space Shuttle
- Space Shuttle
- XXX
- XXX
- Space Station
- Space Station
- Next Gen Launch Technology
- Next Gen Launch Technology ESMD Watch List
ESMD Watch List
- Other
- Other
Over Time: Orion, Ares Risks… - XXX
- XXX
Over Time: Orion, Ares Risks…
- XXX
- XXX
IRKMO001_hq20061219awh 9

10. Risk Management Assurance Mapping
ARM = Active Risk Manager
Risk Management Assurance Mapping
CEV = Crew Exploration Vehicle
CLV = Crew Launch Vehicle
C3PO = Commercial Crew and Cargo Program Office 1
CxCB = Constellation Control Board NASA
ECANS = Exploration Communications and Navigation Systems Administrator
(Griffin)
EVA = Extravehicular Activities
IRMA = Integrated Risk Management Application
LAS = Launch Abort System
LCROSS = Lunar Crater Observation Sensing Satellite TCB
LRO = Lunar Reconnaissance Orbiter
LPRP = Lunar Precursor Robotic Program
OSPP = Office of Security and Program Protection
RMWG = Risk Management Working Group
SOMD = Space Operations Mission Directorate
SR&QA = Safety, Reliability, and Quality Assurance
97
2 Manage Risk
SOMD AA
ESMD AA Manage Risk Data in
(Gerstenmaier)
(Horowitz) Data in ARM CxIRMA
RMP
ESMD RMWG
ESMD Divisions / Orgs
3 7 6 8 9
4 5 10
ESMD Deputy ESMD Advanced ESMD Constellation ESMD Resource ESMD Strategic
Manage Risk ESMD Chief ESMD Directorate ESMD Chief
Associate Capabilities Systems Division Management Integration and
Data in ARM Engineer Integration Office Chief Management Officer Information Officer
Administrator Division Chief Officer
(Lyles) (Woodward) (Stanton) (Allen) (McManus)
(Cooke) (Walz) (Hunter)
C3PO Advanced Capabilities Constellation Program SOMD Programs Mission Support Offices
Programs and Projects Manage Risk Manage Risk 13
11
Data in ARM Data in CxCB Space Shuttle
International 80 23
Infrastructure and 31
CxIRMA Space Station ECANS Project
110 111 Program (SSP) Administration Chief Health & OSPP
86 (ISS) Program Manager
ARES-I Risk Mgt Constellation Constellation Manage Risk Manager (I&A) Manager Medical Officer (?)
C3PO Project Manager (Ondrus)
Constellation Program Office Working Group Program Manager Risk Manager Data in (Hale) (Dominguez) (Williams)
Manager (Hanley) (Turner) (Suffredini)
CxIRMA
(Lindenmoyer) 102
98 104
38
40 42 44
46
48
Constellation
100
Constellation Test
Constellation Constellation 106
108
Constellation
RMP SSP RRB ISS RRB ECANS RRB
Lunar Robotic Technology Centennial Lunar Precursor Operations Systems Engineering Constellation 24 32
LCROSS Program Human Research Program Planning & Verification Advanced Projects 14
Orbiter (LRO) Development Challenges Robotic Program Integration & Integration (SE&I) SR&QA Manager 12 Infrastructure and
Chief Health & OSPP Risk
Manager Program Manager & Control Manager Manager Manager International Space 81
C3PO RRB (Andrews)
Program Manager Program Manager Program Manager
(Laurini)
(LPRP) Manager
(Waddell) (Arceneaux)
Manager Manager (Hansen)
(Noriega) Space Shuttle Station (ISS) Manage Risk ECANS Risk
Administration (I&A) Manage Risk
Medical Officer Manager
(Tooley) (Peri) (?) (Lavoie) (Castle) (Hardcastle) Program (SSP) Risk Manage Risk Program Risk
Risk Manager Data in ARM Risk Manager (?)
Data in Data in Manager (Hoyt)
87 Manager Manager
(Turner) SSP_IRMA ISS_IRMA (Naves) (Liskowsky)
C3PO Risk (Lutomski)
Manager
39
41 43 45
47
49
99
Constellation
101
103
Constellation
105 107 109 RMP RMP RMP
(Erminger) Lunar Robotic Technology Centennial Lunar Precursor Constellation Test Constellation Constellation Constellation
LCROSS Risk Human Research Robotic Program Program Planning Operations Systems Engineering
Orbiter (LRO) Risk Development Risk Challenges Risk & Verification Risk SR&QA Risk Advanced Projects
Manager Risk Manager (LPRP) Risk & Control Risk Integration Risk & Integration (SE&I)
Manager Manager Manager Manager Manager Risk Manager Manager Risk Manager
(Squires) (Morrison) Manager Manager
(Rad) (Cerro) (?) (Powell) (Schwarz) (Curiel) (Turner) (Dorris)
(Holsomback) (Nise)
Indicates risks are housed in
IRMA (either CxIRMA, SSP
Manage Risk IRMA, or ISS IRMA). Note Constellation Projects
Data in IRMA that CxIRMA will be
integrated with ARM 33
Shuttle Transition
Orion (CEV) CEV RRB ARES-I (CLV) CLV RRB Other Constellation Projects Office Manager
Indicates risks are housed in (Lightfoot)
Manage Risk
Data in ARM
ARM 50 51 66 67
Manage Risk Orion Risk Mgt Orion (CEV) Orion (CEV) Risk Manage Risk ARES-I Risk Mgt ARES-I (CLV) ARES-I (CLV) Manage Risk
Data in Working Group Project Manager Manager Data in Working Group Project Manager Risk Manager Data in
Indicates a Risk Review CxIRMA (Hatfield) (Perera) RMP CxIRMA (Cooke) (Kulpa) EEV RMP CxIRMA
34
RRB Board
Shuttle Transition
Office Risk
Manager
Indicates that a risk (Araiza)
82 84 78
RMP management plan has been
88
ARES-V (CaLV)
Ground Mission EVA Project
74 Operations Project Operations Project Manager
created at this level 52 54 56 64 72 76 Project Manager
58 60 68 70 ARES-I Flight Test Manager Manager (Lutz)
Orion Program Orion Vehicle Orion Safety and Orion Test & ARES-I Upper ARES-I Vehicle (Cooke / Kynard)
Orion Crew Orion Service ARES-I First Stage ARES-I Upper & Integration (Talone) (Webb)
Planning and Integration Mission Assurance Verification Stage Engine Integration
Indicates Chain of Command Control Manager Manager Manager
Module Manager Module Manager
Manager
Manager Stage Manager
(J2X) Manager
Office (FITO)
Manager SSP Project ISS Project
(Piatek) (Free) (Burt) (Davis) Manager
interface that provides RM (Floyd) (Gahring) (Thiessen) (Hurlbert) (Snoddy) (Reuter)
EVA RRB
(Taylor) CaLV RRB GO RRB MO RRB
status and escalated risks
53 73 75 83 85
55 57 59 61 65 71 77 89 79
Orion Program 69 ARES-I Upper ARES-I Flight Test Ground Mission
Indicates information Orion Vehicle Orion Safety and Orion Crew Orion Service Orion Test & ARES-I Upper ARES-I Vehicle ARES-V (CaLV)
Operations Risk Operations Risk EVA Risk Manager
Planning and ARES-I First Stage Stage Engine & Integration Risk Manager
interface that provides RM Integration Risk Mission Assurance Module Risk Module Risk Verification Risk Stage Risk Integration Risk Manager Manager (Mulligan)
Control Risk
Manager Risk Manager Manager Manager Manager
Risk Manager
Manager
(J2X) Risk Office (FITO) Risk
Manager (Kulpa / Coleman) EEV RMP
Manager (Mabry) Manager Manager (Anderson) (Smith)
status and escalated risks (Manuel) (Jenkins) (Peterson) (Gibson) (See) (Jasper / Ogle) (Kulpa / Wood)
(Floyd) (Byrd / Ogle) (Cole / Wood)
62 63
LAS Manager LAS Risk Manager
(Stover) (Wood / Cole)
Multiple KBR Capture Points – Multiple Delivery Points
IRKMO001_hq20061219awh 10

11. Knowledge-Based Risks (Continued)
Standalone Program Risks More access to risk information is
Number of Documented Risks
in ARM required to close “knowledge gaps”
L&MS
CLV
CEV
Integrated and Knowledge-
L&MS
Based Risks in ARM
Number of Documented Risks
CLV
2012
2013
2006
2007
2008
2009
2010
2011
CEV
KBRs
Integrated Program Risks
Number of Documented Risks
in ARM
L&MS
2006
2007
2008
2009
2010
2011
2012
2013
CLV
CEV
2006
2007
2008
2009
2010
2011
2012
2013
IRKMO001_hq20061219awh 11

12. Knowledge-Based Risks (Continued)
NASA Standard WBS
ARM allows automated delivery of new KBRs
IRKMO001_hq20061219awh 12

13. Knowledge-Based Risks (Continued)
• Embedded 3-8 min
Video Nugget with
Transcript
• Related Knowledge
Bundles
• Related Content –
reports, documents, etc.
• Threaded discussion
(blog) feature to be
added to comment on
each KBR
• To be hosted on ESMD
R&KM portal
IRKMO001_hq20061219awh 13

14. First Closed Risk KBR – Lunar Recon Orbiter
LRO Spacecraft Delta II Booster Atlas V Booster LCROSS
Spacecraft
• The design of the LRO propulsion tanks was influenced by a number of
factors including launch vehicle characteristics. The Delta II Expendable
Launch Vehicle’s (ELV) spin stabilized upper stage made the Nutation Time
Constant (NTC) a key parameter in assessing the stability of the spacecraft.
The uncertainty in predicting the effects of liquid propellant motions and the
relatively large propellant load and mass fraction for the LRO tank resulted in
the identification of a potential risk. Close coordination and communication
with all levels of management early in the design trade study process allowed
for the effective mitigation of the risk and provided additional lunar
exploration opportunity.
IRKMO001_hq20061219awh 14

15. Practice 3: Communities of Practice (CoPs)
Knowledge resides with people and is often lost via actions like:
• Downsizing
• Retirements
• Shuttle Transition
• People Movement (Job Changes, Project Realignment, etc.)
5 2 1
L CoPs allow the capture of
I 4
K 4 8 1 knowledge which enables risk
E 3 mitigation strategies to be
L 2 3 3 developed or enhanced in
I
H 1
multiple discipline areas
O
O
D
1 2 3 4 5
CONSEQUENCE
IRKMO001_hq20061219awh 15

16. Communities of Practice
• “Communities of Practice (CoP) are groups of people who share a
concern, a set of problems, or a passion about a topic, and who
deepen their knowledge and expertise in this area by interacting on an
ongoing basis”
• “CoPs share information, insight and advice. They help each other
solve problems.”
• “They may create tools, standards, generic designs, manuals, and
other documents —”
• “Cultivating CoP in strategic areas is a practical way to manage
knowledge as an asset, just as systematically as companies manage
other critical assets.”
Communities of Practice. Wenger, et al
IRKMO001_hq20061219awh 16

17. Communities of Practice
ESMD CoPs…
• Risk & Knowledge Management
• Risk & Reliability Analysis
• Earned Value Management
• Technology Protection
• Systems Engineering & Integration
• Human Research
• Research and Technology
IRKMO001_hq20061219awh 17

18. NASA CoP Tool – PBMA Example
Process-Based Mission Assurance (PBMA) Portal
The PBMA toolkit provides
NASA CoPs with a secure
environment to share
documents, conduct threaded
discussions, polls, manage
calendars, locate expertise,
collaborate and learn.
PBMA boasts over 7,500 users
in 275+ communities of
practice, communities of
interest, and workgroups
Information Security – ESMD KM Artifacts Must Be Secure
IRKMO001_hq20061219awh 18

19. NASA CoP Tool – Wiki Example
Risk & Reliability Analysis Wiki
• Wiki software selected -
Confluence
• Several CoPs or Offices
using Wiki (CxP
avionics/software,
ESMD risk and
reliability analysts,
DIO…)
• Establishing Wiki
training capability
Information Security – ESMD KM Artifacts Must Be Secure
IRKMO001_hq20061219awh 19

20. KM Practices Integration
Portal
CoP
s
ka ges
age
d Liin k
Ln
an d
n an
iio n
o
eg
tte rat
gra t
h IIn
h n
Riic
Rc
Wiki
Knowledge-
Sharing
Forums
IRKMO001_hq20061219awh 20

21. Practice 4: Support Knowledge-Sharing Forums
ESMD Alumni Sharing Events
• Engage NASA Alumni
• Utilize APPEL Forums
• Will utilize Webcast to
broaden audience
• Post Videos on ESMD
IRKMS portal
APPEL Master’s Forums
Knowledge Sharing Workshops and Seminars
IRKMO001_hq20061219awh 21

22. Practice 5: Experience-Based Management
and Technical Training
APPEL NESC
Leverage the existing infrastructure of
training courses throughout NASA, but
with the emphasis on transferring
knowledge across its projects and
across ESMD.
ESMD will help shape existing courses
by providing ESMD-related lessons,
gleaned from case studies, PaLs, KBRs,
and other sources of lessons.
NASA
SATERN Safety
Center
IRKMO001_hq20061219awh 22

23. Summary
“ESMD faces exciting opportunities and formidable challenges. To
reduce risk and apply knowledge more effectively, ESMD should
integrate its KM, RM and OL initiatives into a comprehensive plan
that will accomplish more with less bureaucracy. The goal is not
compliance with detailed processes and procedures but
compliance with intent: the intent to learn, to share and probe
every possible angle so ESMD’s missions have the highest
possible chance of success. ESMD must take risks with ‘eyes
wide open’ and ‘minds fully engaged’ at every decision, every
trade and with every residual risk.”
From: Strategy for Exploration Systems Mission Directorate
Integrated Risk Management, Knowledge Management
and Organizational Learning Whitepaper
Dave Lengyel & Dr. Ed Rogers
IRKMO001_hq20061219awh 23