Using Risk Management for Validation

How Much Validation is Enough?

The Breadth and the Depth

Robert Sturm
rmsturm@hotmail.com

Estimating the Dilbert Way (1 of 2)

Estimating the Dilbert Way (2 of 2)

Agenda
• Brief Overview of Validation and Verification
• Do we have to Validate the System?
• Identify Overall System Risk

• Identify Type of System Change
• Identify Needed Validation Documents
• Identify Extent of Test Scripts

• Workshop

DISCLAIMER

• Everything in this presentation
represents my opinion and not
that of my employer.

• It’s all about RISK!

Validation and Verification
• Verify each part: Test and document results

• Validation = ∑ verified parts
It’s the entire process!

VERIFY

Potential Validation Documents
(GAMP 5)
Validation Plan Validation Summary

Data Migration (DM)
User Risk Analysis
Requirements PQ Protocol/ Scripts

Functional Specs OQ Protocol/ Scripts

Design specs IQ Protocol/ Scripts

Trace Matrix

Key Validation Deliverables
• Number of validation documents
– How many?

vs.

• Extent of the test scripts
– What and how much to test?

vs.

How Much? QA vs. IT

VALIDATION

IT &
QA
CUSTOMER

Validation Estimating Workflow
1 2 3 4
Do we Determine Determine Determine
have to Quality & Functionality Overall
validate? Regulatory & Distribution System
Risk Risk Risk

8 7 6 5
Identify Identify Determine Determine
extent of Validation Document Type of
test scripts Documents Level System
Change

START

1 Do We Have to Validate?
• Is this a GxP system - GLP, GCP or GMP?
– Laboratory, Clinical or Manufacturing

• Does the system affect Patient Safety,
Product Quality or Data Integrity?

2 What is the Quality & Regulatory Risk?
• Evaluate the criticality of the data processed by the
system as Low, Medium or HIGH risk

• For each risk category identify examples of GMP, GCP
and GLP records in the system
– HIGH: GMP (Master records, batch records, etc.)

– Medium: GCP (Clinical trial patient data)

– Low: GLP (Training records, facilities records, etc.)
• Use the highest identified risk level

3 What is the Functionality Risk?
• Assign values based on functions the system
performs
– Electronic data capture 3
– Data entry/modification/calculation 3
– Submission creation 2
– Data analysis and reporting 2
– Data/document storage 1
– Data browsing 1

3 What is the Distribution Risk?
• Custom system 5
• Multi-industry, limited use 4

• Regulated industry, limited use 3
• Regulated industry, broad use 2

• Multi-industry, broad use 1

3 What is the Complexity?
• Add Functionality and Distribution Risk

• Complexity Level
HIGH = 7 to 8
Medium = 4 to 6

Low = 2 to 3

4 Determine Overall System Risk
Quality/Regulatory Complexity System Value
HIGH + HIGH = HIGH 10
HIGH + Medium = HIGH 9

HIGH + Low = HIGH 8
Medium + HIGH = Medium 7

HIGH + Low = HIGH 8
Medium + Medium = Medium 6
Medium + Low = Medium 5
Low + HIGH = Medium 4

HIGH + Low = HIGH 8
Medium + Medium = Medium 6
Medium + Low = Medium 5
Low + HIGH = Medium 4
Low + Medium = Low 3
Low + Low = Low 2

HIGH
Drug Clin
Safety Data Elec Data
MedDRA
WHO Mgt Cap
Drug SAS ERP
Drug
Clin Stability Doc Submission
Supplies Publishing
Software
Upgrade Mgt
MEDIUM
Hardware/
R Drug Network
Clin Trial
Coding Upgrade
I Mgt
S Excel

K
Scanner
Low

Low MEDIUM HIGH

COMPLEXITY

HIGH
Drug Clin
MedDRA
WHO Mgt Cap
Drug SAS ERP
Drug
Clin Stability Doc Submission
Supplies Publishing
Software
Upgrade Mgt
MEDIUM
Hardware/
R Drug Network
Clin Trial
Coding Upgrade
I Mgt
S Excel

K
Scanner
Low
Low MEDIUM HIGH

COMPLEXITY

H-l H-m H-H
HIGH
Drug Clin
MedDRA
m-H WHO Mgt Cap
Drug SAS ERP
Drug
m-m Clin Stability Doc Submission
Supplies Publishing
Software
Upgrade Mgt
MEDIUM
m-l Hardware/
R Drug Network
Clin Trial Mgt
Coding Upgrade
I l-H
S l-m Excel

K
l-l Scanner
Low
Low MEDIUM HIGH

COMPLEXITY

What do we know at this point?
• We know the overall system risk level

–H - H, H - M, H - L, M - H, M - M, ….
L – M, L – L
• We have a value for the overall system risk
level

Are we there yet?

5 What Kind of Change is Happening?
• Release (Major) – x.0: Adding functionality;
– Change value of 1.0
• Upgrade (Minor) – x.y: Changing functionality;

• Patch – x.y.z: Not changing functionality;
• Hot Fix – x.y.z.w: Smell for smoke;

6 Determine Overall Document Level
System Risk Value Release Upgrade Patch Hot Fix
Change value: 1.0 0.8 0.5 0.2
H - H
H - M
H - L
M - H

M - M
M - L
L - H
L - M
L - L

Change value: 1.0 0.8 0.5 0.2
H - H 10 10.0 8.0 5.0 2.0
H - M 9 9.0 7.2 4.5 1.8
H - L
M - H

M - M
M - L
L - H
L - M
L - L

Change value: 1.0 0.8 0.5 0.2
H - H 10 10.0 8.0 5.0 2.0
H - M 9 9.0 7.2 4.5 1.8
H - L 8 8.0 6.4 4.0 1.6
M - H 7 7.0 5.6 3.5 1.4

M - M
M - L
L - H
L - M
L - L

Change value: 1.0 0.8 0.5 0.2
H - H 10 10.0 8.0 5.0 2.0
H - M 9 9.0 7.2 4.5 1.8
H - L 8 8.0 6.4 4.0 1.6
M - H 7 7.0 5.6 3.5 1.4

M - M 6 6.0 4.8 3.0 1.2
M - L 5 5.0 4.0 2.5 1.0
L - H 4 4.0 3.2 2.0 0.8
L - M 3 3.0 2.4 1.5 0.6
L - L 2 2.0 1.6 1.0 0.4

7 Identify Validation Documents
Document Value 10 - 9 8-7 6-4 3-0
Validation Plan I I I C O
User Requirements I C C C C O
Risk Analysis I C C C O
Functional Specs I C C O O O
Design specs I C O O O
Trace matrix I C C C O
I = Individual, C = Combined, O = Optional
IT’S A GREY AREA!

7 Identify Validation Documents
Document Value 10 - 9 8-7 6-4 3-0
IQ Protocol/Scripts I C I C C C O
OQ Protocol/Scripts I C I C C C O
PQ Protocol/Scripts I C I C C C O
DM Protocol/Scripts I I C C C O
Validation Summary I I I C O
Production Release I O I O I O O
Memo

I = Individual, C = Combined, O = Optional

Now where are we?
• We identified what validation documents are
needed for the project

• We have QA buy-in

Validation documents

8 Identify Extent of Test Scripts
• Determine level of testing
• Apply a Functional Risk Assessment to all user
requirements
• Prioritize requirements as HIGH (H),
Medium (M) or Low (L)
• Assess requirements as regulatory/ business
risk Critical (C) or Not critical (N)
• High and Critical are the highest risk

8 Prioritize Requirements (Reqmnt)
• HIGH - H (Mandatory): Reqmnt must be present for
the system to operate
• Medium - M (Desirable to have): Reqmnt need not
be present for the system to operate but it is a ‘nice
to have” feature
• Low - L (Low impact/may be useful): Reqmnt need
not be present for the system to operate, may be
useful and has low impact to the prime user
department
• Null - N (Not used): Reqmnt will not be used and will
not be validated

8 Prioritize Risk of the Requirement
• Critical (C): If this Reqmnt is not done there is
a risk of non-compliance. Reqmnt is critical
for business reasons (data are correct, system
is Web available, data entry and output are
correct, etc.)

• Not critical (N): Reqmnt has no regulatory or
business risk associated with it

8 To Test or Not to Test?
Priority Risk Test (Y/N)
(H/M/L) (C/N)
H C Y
M C Y
L C Y
-- -- --
H N Y
M N N
L N N
N N N

Assessing Changes to a Validated
System for Company XYZ
Description Priority Risk Test
(H/M/L) (C/N) (Y/N)
Changes to the safety database schema H C Y
and tables
Ability to capture partial dates M N N
Able to print reports for FDA submissions H C Y
on a representation of the revised
MedWatch form
Customer fixes for expedited reports M N N
Lab test name encoding from Case Form L N N
Electronic Submission: Updated feature -- N N
not used

Interactive Exercise
• Our vendor sends us a minor upgrade to our
Drug Safety system
• It affects our existing requirements

• What validation documents are needed?
• What should be tested?

The Next Step Is Yours

OR

Bonus Information: References
• “General Principles of Software Validation, Final Guidance for
Industry and FDA Staff”, FDA, Jan 11, 2002. www.fda.gov
• Annex 11 Computerised Systems
• Computer Validation: The 100 Worst Mistakes You Can Make,
Tamara Follett, 2003
• The Computer System Risk Management and Validation Life Cycle, R.
Timothy Stein, 2006 (many references)
• GAMP 5 A Risk-Based Approach to Compliant GxP Computerized
Systems, ISPE, 2008
• “GIP Guidance Module 1: Validation and Verification”, HIMSS, 2011,
himss.learn.com/learncenter.asp?id=178409&page=184
• Google “Software Compliance Science John Murray FDA” for a PDF
file: www.softwarecpr/.../download.asp

Bonus Information: References
• “RAMP: An Approach to Risk-Based Computer System Validation
and Part 11 Compliance”, Drug Information Journal, Vol 41, pp 69-
79, 2007 (Use to estimate Quality, Regulatory, Functionality and
Distribution Risk)
• “Validation of Software for Regulated Processes”, Assoc. for
Advancement of Medical Instrumentation, 13-Dec 2007
• “Effective and Practical Risk Management Options for Computerised
System Validation,” R. D. McDowall, Quality Assurance Journal, Vol
9, Issue 3, 2005
• IT Pharma Validation Europe: http://www.ccs-inspired.com
• Ask About Validation: http://www.askaboutvalidation.com
• Risk Doctor Network: http://www.risk-doctor.com
• Compliance Webinars: http://www.complianceonline.com/

Four Assessment Questions
• Does the system automate a process or
manage date regulated by these regulations?

– GMP (21CFR 210 and 211)
– GCP (21 CFR 312)
– GLP (21 CFR 58)
– PDMA (21 CFR 203)

• If yes to any of the above, it is GxP

Bonus Information: Validation Life Cycle
Folder Organization
1 Admin
IT Computer System Validation and Document Flow: Key Validation Documents 2 Pre xQ
3 IQ
** EVALUATE ** ** PLAN ** 4 During Xq
2 7 - Configuration & Training
- Major release 1 1 1
- SOPs and WIs
A B K Validation/
- Added functionality Qualification Project Supplier Audit - Migration
Plan 5 OQ
6 PQ
7 Post xQ
8 Post Impl
** REQUIREMENTS ** ** EXECUTION ** ** EXECUTION **
** DEPLOYMENT **
IDENTIFY REQUIREMENTS WRITE PROTOCOL & EXECUTE TESTS SUMMARIZE TEST RESULTS
TEST SCRIPTS WRITE FINAL REPORTS

3 3 3
7
Installation
2 IQ Installation Qualification Installation Validation/
2 Qualification
User 2 Protocol & Test Scripts Qualification Test Summary Report Qualification
D (Validation Env.) Execution
Requirement C (Val Env.) Summary Report
Specifications (Val Env.)

2 7
4 4
Functional 2 4 4 4
E F G F G 5
Specifications Training Records
Design Operational
5
Specifications 5 Qualification
Summary Report
Operational Operational 7
Qualification Protocol Qualification Test
& Test Scripts Execution Support Model
6

Performance
2 6
4 Qualification
4
Performance Summary Report 7
Requirements Trace H I
Matrix Qualification Test Production Release
6 Execution memo with
7 Workarounds, if any
Performance
Qualification Protocol 7 J
& Test Scripts Pre
J
3 ** MAINTENANCE **
3
IQ 3 IQ Summary Post Implementation
IQ Protocol & IQ Execution Report
(Prod. Env.) (Prod Env.) 8
Test Scripts
Optional: C-Risk Analysis, D-Training Plan,
D-Training Plan, Final WIs/ SOPs
(Production Env.)
E-Detailed Design Specs., F-Configuration Spec.,
F-Configuration Spec., 4 7 7
4
F F J Post J
G-Training & Manuals, H-Migration Qualification Protocol,
G-Training & Manuals, H-Data Migration Protocol, 8
CHANGE
I-Draft WIs/ SOPs, J-Data Migration Report Report
J-Migration Qualification CONTROL

Administrative: A-GxP Assessment, B-IT Statement of Work,
K-MS Project Plan
** RETIREMENT **

Using Risk Management for Validation

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (20)

Semelhante a Using Risk Management for Validation

Semelhante a Using Risk Management for Validation (20)

Último

Último (20)

Using Risk Management for Validation