SUMMIT 2013

Avoca Quality
Consortium Summit
May 9, 2013
The Westin Princeton at
Forrestal Village
Princeton, NJ

3
Avoca Quality Consortium Summit
Stephen Cutler, PhD
Group President, ICON Clinical Research

5
Summit Co-Chairs
Christopher J. Hilton
Vice President, Development and Clinical
Alliance Management, Pfizer, Inc.
Michael D. Jones
Senior Director, Clinical Trial: Materials,
Implementation and Transformation,
Eli Lilly and Company

6
Welcome
Jeffrey Kasher, PhD.
Vice President, Clinical Trial:
Materials, Implementation and
Transformation, Eli Lilly and
Company

7
Welcome
Ann Meeker-O’Connell
Acting Division Director, Good Clinical Practice
Compliance, US Food & Drug Administration

8
Summit Theme
Achieving the right balance between meeting
timelines, ensuring cost containment and
achieving the highest level of quality

9
Our Anchor – Why We Are Here
Improving human health and bringing life saving
medicines to patients

10
Summit Focus
●More collaborative and effective
partnerships
●Greater efficiency
●Mitigation of risk
●Highest levels of quality

11
Summit Topics
Gaining clarity in the areas of:
• Proactive Quality Management
• Effective oversight
• Risk Assessment and Risk
Management
• Change Management
• Culture and Quality

12
Avoca Quality Consortium
Our vision:
● To serve as a catalyst for the acceleration of best practices for
proactive quality management
Consortium Sponsors:
The work of the Avoca Quality Consortium is in
helping members optimize their approaches to proactive
quality management with an emphasis on bringing
sponsors and CROs
into greater alignment.

13
Pharma/Biotech Industry Participation to Date

14
CRO Participation

15
Ongoing Initiatives
● Quality Agreement Template
● Short list of Quality Metrics
● Specific tools for Proactive Quality Management
● Guidelines
● Definitions
● Setting Expectations tool
● Research
● Benchmarking of each company’s Quality Management practices
● Assessment of approaches to prequalification and routine audits
● Comprehensive assessment of the way our Members are assessing and
managing risk

16
Our Request for Today
Please join the
discussion!

17
A Message from Jeff
Jeffrey Kasher, PhD.
Vice President, Clinical Trial:
Materials, Implementation and
Transformation, Eli Lilly and
Company

18
Our Moderators
Marc Monseau
Founder and Managing Partner,
Mint Collective, LLC
Steve Whittaker
Senior Consultant and Director of the Avoca
Quality Consortium, The Avoca Group

19
Today’s Agenda
Breakfast sponsored by
Morning Sessions:
Introductory Remarks
╸Michael Jones, Sr. Director, Clinical Trial: Materials, Implementation
and Transformation, Eli Lilly & Co.
╸Stephen Cutler, PhD, Group President, ICON Clinical Research
╸Christopher J. Hilton, Vice President, Development and Clinical
Alliance Management, Pfizer, Inc.

20
Today’s Agenda
Morning Sessions (continued):
Strategies for Ensuring Effective Oversight
╸Insights from Ann Meeker-O’Connell, Acting Division Director, Good
Clinical Practice Compliance, US FDA
╸Data from Avoca’s Consortium Research
╸Panel Discussion facilitated by Steve Whittaker

21
Today’s Agenda
Morning Sessions (continued):
Risk Assessment and Risk Management in Outsourced Clinical Trials
╸Insights from Ann Meeker-O’Connell,
╸Data from Avoca’s Consortium Research
╸Panel Discussion facilitated by Steve Whittaker

22
Today’s Agenda
Lunch (12:10 pm)sponsored by:
Afternoon Sessions:
Master Class in Leading Change
Peter Bregman, CEO, Bregman Partners
╸Peter Bregman will lead an interactive working session on change
management
Creating a Culture of Quality
╸Components that Comprise a Culture of Quality: Margaret Davis,
Margaret Davis Consulting
╸Panel discussion moderated by Steve Whittaker

23
Today’s Agenda
Afternoon Sessions (continued):
Crowdsourcing Solutions
╸Group brainstorming – generating ideas to enhance quality
╸Facilitator: Marc Monseau
Call to Action
╸Panel discussion focused on implementing new strategies for
proactive quality management
╸Facilitators: Marc Monseau and Steve Whittaker
Conversation continues during a cocktail reception (5:15 pm)
sponsored by

24
Avoca Quality Consortium Summit
A message from John Hubbard, Worldwide Head of
Development, Pfizer

25
Introductory Remarks
Stephen Cutler, PhD
Group President, ICON Clinical
Research
Vice President, Development and
Clinical Alliance Management,
Pfizer, Inc.
Michael Jones
Sr. Director, Clinical Trial: Materials,
Implementation and Transformation,
Eli Lilly and Company

27
Effective Oversight

Session 2: Strategies for
Ensuring Effective
Oversight
May 9, 2013

29
Outline:
 Interim results from 2013
Quality Consortium
Assessment
 Approach for creating
guidelines for proactive quality
management and effective
oversight
 Executive panel discussion

30
Quality Consortium Assessment
Sponsor: The operational teams
involved in overseeing our CROs
consistently micromanage to ensure
that the quality of deliverables meets
expectations.
CRO: The sponsors with which I work
consistently micromanage to ensure
that the quality of deliverables meets
expectations.
N=208 N=134
21%
31%
33%
11%
4% 7%
47%31%
12%
3%
7%
5%
1%
4%
5%
4%
47%
31%
31%
25%
14%
21%
31%
34%
33%
34%
41%
29%
12%
24%
24%
29%
34%
30%
3%
5%
10%
8%
6%
16%
0% 20% 40% 60% 80% 100%
The sponsors with which I work consistently
micromanage to ensure that the quality of
deliverables meets expectations.
The sponsors with which I work clearly document
CRO oversight practices, roles, and responsibilities.
The technology systems used by my company and its
sponsor partners promote efficient oversight
practices.
The sponsors with which I work clearly define the
roles of internal and CRO staff so as to minimize
duplication of effort.
The sponsors with which I work are efficient in the
use of resources applied to the oversight of
outsourced trials.
There is a direct positive relationship between the
intensity of sponsor oversight and the quality of CRO
deliverables.
5 - Strongly agree 4 3 - Neither agree nor disagree 2 1 - Strongly disagree

31
Executive Roundtable
CRO: There is a direct positive
relationship between the intensity of
sponsor oversight and the quality of
CRO deliverables.
4%
21%
29%
30%
16%
7%
5%
1%
4%
5%
4%
47%
31%
31%
25%
14%
21%
31%
34%
33%
34%
41%
29%
12%
24%
24%
29%
34%
30%
3%
5%
10%
8%
6%
16%
0% 20% 40% 60% 80% 100%
practices.
outsourced trials.
deliverables.
N=135
Sponsor: I personally have received
high-quality deliverables from CROs,
even when overseeing their work at a
"strategic" rather than detailed level.
6%
35%
34%
17%
8%
N=193

32
Sponsor: My company
clearly documents CRO oversight
practices, roles, and responsibilities.
clearly document CRO oversight
practices, roles, and responsibilities.
N=208 N=131
12%
40%
23%
20%
5% 5%
31%
34%
25%
5%
7%
5%
1%
4%
5%
4%
47%
31%
31%
25%
14%
21%
31%
34%
33%
34%
41%
29%
12%
24%
24%
29%
34%
30%
3%
5%
10%
8%
6%
16%
0% 20% 40% 60% 80% 100%
practices.
outsourced trials.
deliverables.

33
Sponsor: My company
clearly defines the roles of internal
and CRO staff so as
to minimize duplication of effort.
clearly define the roles of internal
and CRO staff so as
to minimize duplication of effort.
N=209 N=127
6%
34%
34%
18%
8% 4%
25%
34%
29%
8%
7%
5%
1%
4%
5%
4%
47%
31%
31%
25%
14%
21%
31%
34%
33%
34%
41%
29%
12%
24%
24%
29%
34%
30%
3%
5%
10%
8%
6%
16%
0% 20% 40% 60% 80% 100%
practices.
outsourced trials.
deliverables.

34
Sponsor: My company
is efficient in the use of resources
applied to the oversight of
outsourced trials.
are efficient in the use of resources
applied to the oversight of
outsourced trials.
N=207 N=128
5%
28%
29%
32%
6% 5%
14%
41%
34%
6%
7%
5%
1%
4%
5%
4%
47%
31%
31%
25%
14%
21%
31%
34%
33%
34%
41%
29%
12%
24%
24%
29%
34%
30%
3%
5%
10%
8%
6%
16%
0% 20% 40% 60% 80% 100%
practices.
outsourced trials.
deliverables.

35
Sponsor: The technology systems
used by my company and its CRO
partners promote efficient oversight
practices.
CRO: The technology systems used by
my company and its sponsor partners
promote efficient oversight practices.
N=204 N=136
2%
19%
31%
32%
16%
2%
31%
33%
24%
10%
7%
5%
1%
4%
5%
4%
47%
31%
31%
25%
14%
21%
31%
34%
33%
34%
41%
29%
12%
24%
24%
29%
34%
30%
3%
5%
10%
8%
6%
16%
0% 20% 40% 60% 80% 100%
practices.
outsourced trials.
deliverables.

Approach for Creating
Guidelines/Tools for
Effective Oversight

37
Approach for Creating Guidelines/Tools
Governance
Organizational
Construct
Processes
Communication
Oversight
Leadership
Metrics /
Analytics /
Technology
Roles /
Responsibilities
Oversight
Define
Plan
Lead
Decide
Measure
Deliver
Proactive
Quality
Management

39
Effective Oversight: Panel Discussion
Mike Collins
Vice President Global Clinical Operations, Alexion
Pharmaceuticals
Stephen Cutler, Ph.D.
Group President, ICON Clinical Research Services
James D. Eisenhart, Ph.D.
CEO, Chiltern International
Vice President, Development & Clinical Alliance
Management, Pfizer Development Operations
Michael D. Jones
Senior Director, Clinical Trial: Materials,
Implementation and Transformation, Eli Lilly and
Company
Jeffrey McMullen
Vice Chairman, InVentiv Health, Inc.

Risk Management and
Risk Assessment

42
Risk Assessment and Risk Management

Risk Assessment and
Risk Management in
Outsourced Clinical
Trials

44
Denise Calaprice-Whitty, Ph.D.
Executive Director, Survey Research and
Relationship Management Programs,
The Avoca Group

46
Overview
Topics Explored
● Risk-sharing models
● Risk assessment: Is it formally done? How is it done? Are assessments
just qualitative, or also quantitative? Has it been successful? In what
ways yes, and in what ways no?
● Risk-based management approaches: To what tasks have such
approaches been applied (e.g. monitoring, CRO management)? Have the
approaches used formal quantitative modeling, or have they been
qualitative? Have they been successful? Is the success realized mostly in
time, cost, or quality? What have been the downsides?
● Oversight

47
Interim Analysis
● Sponsors:
╸14 companies
o 268 respondents
● CROs:
╸7 companies
o 174 respondents
Interim Analysis

48
Use of Systematic Risk Assessment
N=165
Sponsor Data
50%
23%
13%
13%
1%
N=119
Approximately how often is a systematic risk assessment process employed
for clinical trials conducted by the teams with which you work?
CRO Data
35%
19%
19%
14%
13%
For >75% of outsourced
clinical trials
For 51% - 75% of outsourced
clinical trials
clinical trials
clinical trials
Never

49
Please briefly describe the conditions under which a systematic
risk assessment process is used:
● Not many: very systematic and metrics based
● Not consistent, don’t know;; often done “in silos”
● Done, but not necessarily “systematic and rigorous”
╸Too high level and “standard” – copy and paste from previous
studies – no time to think about study-specific nuances

50
Please briefly describe the conditions under which a systematic
risk assessment process is used:
● Only…
╸During RFP process
╸At governance level
╸Ad hoc
╸At program level, not study level
╸For full-service CRO relationships
╸For high-risk, complex, or large studies
╸For audit strategy
╸Upon sponsor request
╸For key clients
╸When resource limitations are a big concern

52
24%
55%
13%
8%
Generally
performed
primarily by my
company
Generally a joint
CRO-sponsor
process, with
approximately
equal
contributions
Generally
conducted
primarily by the
CRO
It depends.
N=134
Sponsors: For outsourced clinical trials,
to what extent is the CRO generally
involved in the systematic risk
assessment?
Involvement of CRO in Risk Assessment
CROs: To what extent are your
company's project team members
generally involved in the systematic
risk assessment?
61%
33%
1%5%
Generally
conducted
primarily by my
company
Generally a joint
provider-sponsor
process, with
approximately
equal
contributions
Generally
performed
primarily by the
sponsor
It depends
N=128

55
Sponsors: In general, how satisfied are you with the performance of your in-
house teams with respect to each of the following?
8%
8%
6%
7%
12%
4%
7%
7%
48%
43%
38%
39%
25%
34%
25%
26%
31%
35%
35%
33%
41%
45%
44%
42%
12%
10%
19%
13%
17%
11%
19%
20%
2%
4%
3%
8%
5%
5%
4%
5%
0% 20% 40% 60% 80% 100%
Proactive identification of potential risks
Appropriateness of measures suggested or taken in
reaction to risk-related information
Proactive risk analysis and evaluation
Communications regarding risk-related trial
information
Frequency of review of risk-related trial information
Overall performance on risk assessment and
management related activities
Compilation of risk-related trial information during
a trial (observations, trends, etc.)
Rigor of review of risk-related trial information
5 - Very satisfied 4 3 - Neither satisfied nor dissatisfied 2 1 - Very dissatisfied
N Mean
170 3.5
159 3.4
162 3.3
156 3.3
151 3.2
159 3.2
161 3.1
149 3.1
Satisfaction with Risk Assessment: Sponsor Teams

56
Sponsors: In general, how satisfied are you with the performance of your CRO
partners with respect to each of the following?
3%
2%
3%
4%
1%
1%
2%
1%
23%
20%
17%
17%
16%
16%
14%
15%
45%
40%
41%
31%
38%
39%
34%
29%
20%
25%
28%
31%
35%
31%
32%
38%
9%
13%
11%
15%
10%
13%
19%
17%
0% 20% 40% 60% 80% 100%
information
a trial
N Mean
148 2.9
146 2.7
157 2.7
143 2.6
147 2.6
148 2.6
139 2.5
150 2.5
Satisfaction with Risk Assessment: CRO Partners

57
Sponsors: In general, how satisfied are you with the performance of your other
service providers with respect to each of the following?
1%10%
10%
6%
3%
2%
6%
4%
2%
46%
40%
33%
37%
34%
31%
32%
31%
22%
37%
40%
36%
44%
36%
37%
41%
21%
13%
22%
23%
20%
27%
27%
26%
0% 20% 40% 60% 80% 100%
information
N Mean
87 2.5
98 2.5
86 2.2
86 2.2
91 2.2
83 2.2
81 2.1
85 2.1
Satisfaction with Risk Assessment: Other Service
Providers

58
CROs: In general, how satisfied are you with the performance of your company's
teams with respect to each of the following?
10%
9%
3%
5%
8%
3%
7%
4%
45%
39%
42%
36%
28%
37%
26%
27%
26%
31%
40%
37%
40%
35%
41%
42%
17%
19%
10%
19%
21%
19%
21%
22%
2%
2%
5%
3%
3%
6%
4%
4%
0% 20% 40% 60% 80% 100%
information
N Mean
128 3.5
126 3.3
119 3.3
123 3.2
121 3.2
123 3.1
123 3.1
121 3.0
Satisfaction with Risk Assessment: In-house (CRO)
Teams

61
Sponsors: In general, how difficult is it to
gain agreement between your company and
your clinical service provider (including
CRO) partners about the probability and
importance of the various risks associated
with clinical trial conduct?
21%
46%
29%
4%
Easy
Slightly
difficult
Somewhat
difficult
Very difficult
Difficulty of Gaining Agreement re. Risks
CROs: In general, how difficult is it to gain
agreement between your company and your
sponsor partners about the probability and
importance of the various risks associated
with clinical trial conduct?
16%
43%
35%
6%
N=140 N=95

64
Benefits of Risk Assessment/Management:
Efficiency
In general, have your risk assessment and management approaches
resulted in more efficient use of resources for your company and/or your
CRO partner?
16%
39%17%
28%
Sponsor Data CRO Data
9%
40%
17%
34%
Yes
Sometimes
No
Don't know/
too soon to tell
N=192 N=135

66
Benefits of Risk Assessment/Management: Quality
In general, have your risk assessment and management approaches
resulted in increased quality?
26%
44%
7%
23%
19%
39%9%
33%
Yes
Sometimes
No
Don't know/
too soon to tell
N=191 N=134

69
Understanding of Best Practices in Risk
Assessment/Management
How would you rate your understanding of best practices in risk assessment and
management in clinical trials?
8%
53%
32%
7%
8%
40%
41%
11% Very strong
understanding
Good
understanding
Fair
understanding
Poor
understanding
N=195 N=134

70
Keys to Successful Risk Assessment/Management
● Vested interest in success
╸Trust/transparency between partners
╸Budgetary allowance/support from management
╸Enforcement of/accountability for process and stipulated
actions
● Education on risk assessment and resource allocation according to
risk
● Appropriate “mature” experience (therapeutic area, operational,
development) and multifunction team to “integrate” risks
● Well-defined roles and responsibilities in assessment and in the
action plan

71
● Right tools/technology/approach to define and monitor risks
╸Initial assessment in parallel with protocol writing
╸Examples of previous plans to trigger discussion – use of
Lessons Learned
╸Good metrics
╸Beyond recruitment
╸Qualitative and quantitative, with scoring
● Update/review throughout a trial - recognized that risks are
"dynamic" in that risks may change in probability or level of impact,
and the responses, mitigations, or contingencies may need to
change as well.

72
Challenges of Successful Risk
Challenges in Successful Risk Assessment/Management
● People see it as adding complexity – they know the risks anyway
● Management or partner buy-in for proactive rather than reactive action
(“extra cost”)
● Getting all stakeholders (departments, partners) to understand
(appropriate background), prioritize, and participate
● Timing – needs focus at study start, when other activities are also
“burning”
╸Not part of the “day job” but a “bolt on”
● Lack of basic understanding of concepts/process
╸Poor facilitation that is list driven rather than analysis driven
╸Difficult to identify the actual risks correctly, focus on the “wrong risks”
╸No visible in-house experts to rely on/mentor

73
Challenges of Successful Risk
Challenges in Successful Risk Assessment/Management
● Poor tools
╸“Box checking” risk registers
╸Poor (lagging) metrics
● Lack of appropriate data to populate tools
● Unclear accountability, for analysis and triggered actions (including
CRO vs. sponsor)
● Inconsistency within sponsors in how this is handled– vendors
don’t know what to expect
● Process “Completely foreign” to vendors beyond CROs
● Fear of “being exposed”

74
How often do the project/program teams with which you work (including the CRO
partners) use a risk-based approach to the monitoring of Investigative Sites for
outsourced clinical trials?
Use of Risk-Based Monitoring
19%
15%
12%
21%
22%
11%
17%
19%
12%
9%
16%
27%
For >75% of outsourced
clinical trials
clinical trials
clinical trials
clinical trials
clinical trials
Never
N=133 N=97

75
In general, have your risk-based monitoring approaches resulted in more efficient
use of resources for your company and/or your partner?
Benefits of Risk-Based Monitoring: Efficiency
19%
19%
13%
49%
13%
22%
6%
59%
Yes
Sometimes
No
Don't know/
too soon to tell
N=165 N=113

76
In general, have your risk-based monitoring approaches resulted in increased
quality?
Benefits of Risk-Based Monitoring: Quality
17%
25%
7%
51%
14%
16%
10%
60%
Yes
Sometimes
No
Don't know/
too soon to tell
N=163 N=112

77
Merry Chu
Director, Medical Quality, Regional and Third
Party Management, Eli Lilly Canada Inc.
Heather Jorajuria
Global Head, Product Development, Quality
Assurance, F. Hoffmann-La Roche, Ltd. And
Genentech, Inc.
Jeffrey A. Kueffer
Senior Vice President, Global Operations
Management, INC Research®
Margaret Keegan
Senior Vice President & Global Head, Integrated
Processes and Technologies, Quintiles
Angelika Tillmann FRQA
Vice President Global Clinical Quality Assurance &
Compliance, General Manager Europe, Theorem
Clinical Research

May Quality Summit
The Westin Princeton at
Forrestal Village
Princeton, NJ

81
Masterclass: Leading Change
Peter Bregman
CEO, Bregman Partners

83
Creating a Culture of Culture of Quality
Margaret Davis
President, Margaret Davis Consulting

Culture of Quality
Avoca Consortium on Quality
May 9, 2013
Meg Davis
Margaret Davis Consulting

85
Culture
The collectively held beliefs and behaviors
of a group

86
Consistency:
Coordination,
Agreement,
Values
Involvement:
Empowerment,
Team,
Skills
Mission:
Vision, Strategy,
Goals
Adaptability:
Customer,
Learning,
Change
Beliefs
High Performing Culture

87
Hear others talk about quality
Feel quality all around
See others take quality-focused actions
Transfer quality to peers
What is a Culture of Quality?

88
How common is Culture of Quality?
Culture of Quality index scores - Quintiles

89
Does Culture of Quality affect performance?
Culture’s impact on mistakes addressed by employees
Weekly, difference between top and bottom 20% in Culture of Quality index

90
What activities differentiate strong
Cultures of Quality from weak?

91
Key Differentiating Activities
Employee Ownership
● Understand quality fit with job
● Comfortable raising quality concerns
& challenging directives
● Empowered to make decisions about
quality
Peer Involvement
● Have strong network of peers for
discussion & guidance
● Peers are involved in and held
accountable for quality performance
Message Credibility
Messages on quality and priorities are:
● delivered by respected sources
● easy to understand
● appeal to me personally
Leadership Involvement
● Leadership states that quality is a
priority
● Manager emphasizes quality’s
importance, and walks the talk on
quality

92
Employee
ownership
Peer
involvement
Leadership
emphasis
Message
credibility
Consistency:
Coordination,
Agreement,
Values
Involvement:
Empowerment,
Team,
Skills
Mission:
Vision, Strategy,
Goals
Adaptability:
Customer,
Learning,
Change
Beliefs
Culture of Quality

93
Summary
● Culture of Quality can be defined and measured
● Culture of Quality affects performance
● Culture of Quality can be driven by controllable activities

94
A Culture of Quality: Panel Discussion
Peter Bregman
CEO, Bregman Partners
Hazel Collie
Vice President, Head of Global Compound
Development Quality & Systems, Grünenthal
GmbH
Margaret Davis
President, Margaret Davis Consulting
James R. Dixon
Senior Vice President, Global Quality and
Compliance, PPD
Mitchell Katz, Ph.D.
Executive Director, Medical Research
Operations, Purdue Pharma L.P.

Let’s Crowdsource:
Generating Ideas to
Enhance Quality

96
Approach
3:00-3:05 – The Approach
3:05-3:45 – Small Group Ideation (Led by group leaders)
3:45-4:00 – Reporting and Capturing Insights

97
Challenge Question
Quality
Agreement
Guidance
Document
Tools
Templates/M
odels
Once these resources are
created, how do you
operationalize them?

98
Challenge Question
How can we develop approaches to significantly
enhance the effective implementation of
deliverables, initiatives and programs within
diverse organizations -- both sponsor companies
and CROs?
Identify ONE approach, method or resource that
can be developed to enable the implementation of
these practical initiatives.

100
Call to Action: Panel Discussion
Stephen Cutler, PhD
Group President, ICON Clinical Research
Vice President, Development & Clinical Alliance
Management, Pfizer Development Operations
Michael D. Jones
Senior Director, Clinical Trial: Materials, Implementation
and Transformation, Eli Lilly and Company
Harris Koffer
Pharm. D, President, Research Pharmaceutical Services,
Inc.
Jeffrey McMullen
Vice Chairman, InVentiv Health, Inc.
John Potthoff, PhD
President and CEO, Theorem Clinical Research

102
Upcoming Events
Fall Meetings:
• Consortium Members
Meeting
• Executive Meeting

Thank you
Contact Avoca at:
(609) 252-9020
www.theavocagroup.com
info@theavocagroup.com
179 Nassau Street
Suite 3A
Princeton, NJ 08542

SUMMIT 2013

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a SUMMIT 2013

Semelhante a SUMMIT 2013 (20)

Mais de The Avoca Group

Mais de The Avoca Group (15)

Último

Último (20)

SUMMIT 2013