Final phase ii scientific reviewer training

PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE

Pa#ent-‐Centered
Outcomes
Research
Ins#tute
(PCORI)

Phase
II
Scien4ﬁc
Reviewer
Training

October
2012

Housekeeping
–
Presenta4on
Mode

Ø  A9endee
phone
lines
are
muted

Ø  Ques#ons
may
be
submi9ed
via
Chat
in

the
lower
right
hand
side
of
your
screen

à

Ø  Please
send
ques#ons
as
they
occur
to

you.
They
will
be
answered
at
the
end
of
1. Type
your
the
session,
as
#me
permits
question
here.

Ø  Press
“0”
on
the
phone
for
a
private
help

session
with
the
operator

2. Click
Send

Agenda

1.  Introduc4on
and
Announcements

2.

Background

3.

Program
Funding
Announcements

4. The
Applica#on
and
Review
Process

5. Merit
Review:
Phase
I
–
Scien#ﬁc
Review

6.

Merit
Review:
Phase
II
–
Impact
Review

7.

Merit
Review:
Phase
II
–
In-‐Person
Panel

8.

Phase
II:
PCORI
Online
–
Process
and
Procedures

Our
SROs

Assessment
of
Preven4on,
Diagnosis,
and
Communica4on
and
Dissemina4on

Treatment
Op4ons

Research

Howard
Underwood,
MD,
Marianne
H.
Kimberly
A.
Marschhauser,

MBA,
MS
Alcia#,
Ph.D.
Ph.D.

Jessica
Nadler,
Ph.D.

Improving
Healthcare
Systems
Addressing
Dispari4es

Lev
Nevo,
MD
Sabina
I.
Robinson,
Ph.D.

Parag
Aggarwal,
Ph.D.

Announcements

Open
session
to
any
per4nent
announcements

Key
Dates

Phase
II
Assignments
Released
–
Oct
12th

Preliminary
Scores
Due
–
November
2nd
at
5:00
pm

Op#onal
Dinner
–
Nov
14th
in
Washington,
DC
from
6:00
pm
–
9:30
pm

Phase
II
Panels
–
Nov
15th
in
Washington,
DC
from
7:00
am
–
3:00
pm

Agenda

1.  Introduc#on
and
Announcements

2.

Background

3.

Program
Funding
Announcements

4. The
Applica#on
and
Review
Process

5. Merit
Review:
Phase
I
–
Scien#ﬁc
Review

6.

Merit
Review:
Phase
II
–
Impact
Review

7.

Merit
Review:
Phase
II
–
In-‐Person
Panel

8.

Phase
II:
PCORI
Online
–
Process
and
Procedures

PCORI
Mission,
Vision
and
PCOR

Ø  PCORI
is
a
non-‐governmental,
non-‐profit

organiza#on
founded
by
the
Pa#ent

Protec#on
and
Affordable
Healthcare
Act
of
PCORI
Mission
Statement

2010

(
h9p://www.pcori.org/assets/ PCORI
helps
people
make
informed
healthcare

PCORI_EstablishingLeg.pdf)
decisions
and
improves
healthcare
delivery
and

outcomes
by
producing
and
promo#ng
high
integrity,

Ø  PCORI
aims
to
fund
pa#ent-‐centered
evidence-‐based
informa#on
that
comes
from

research
that
will
improve
healthcare
research
guided
by
pa#ents,
caregivers,
and
the

outcomes
for
pa#ents,
their
caregivers,
and
broader
health
care
community.

other
stakeholders

Ø  Pa4ent-‐centered
outcomes
research

(PCOR)
helps
people
and
their
caregivers
Vision

communicate
and
make
informed

Pa#ents
and
the
public
have
the
informa#on
they

healthcare
decisions,
allowing
their
voices

need
to
make
decisions
that
reflect
their
desired

to
be
heard
in
assessing
the
value
of

health
outcomes.

healthcare
op#ons

For
more
informa#on
on
PCOR,
please
reference
the
PCORI
Methodology
report
at:

h9p://pcori.org/assets/MethodologyReport-‐Comment.pdf

PCORI’s
Na4onal
Priori4es

Purpose
Methodologies
Research
Agenda

Provide
informa#on
to
PCORI
Support
iden#ﬁca#on
of
Support
the
collec#on
of
preliminary

that
informs
future
itera#ons
of
research
methodologies
data
to
advance
the
ﬁeld
of
pa#ent-‐
na#onal
research
priori#es
for
that
advance
pa#ent-‐ centered
outcomes
research,
providing

pa#ent-‐centered
outcomes
centered
outcomes
the
plagorm
for
an
evolving
PCORI

research.

research
research
agenda.
PCORI’s
ini#al

Research
Agenda:

The
Na#onal
Priori#es
are:

1.  Comparisons
of
Preven#on,

1.  Compara#ve
Assessments
of
Diagnosis,
and
Treatment
Op#ons

Preven#on,
Diagnosis,
and
2.  Improving
Healthcare
Systems

Treatment
Op#ons
3.  Communica#on
&
Dissemina#on

2.  Improving
Healthcare
4.  Addressing
Dispari#es

Systems
5.  Accelera#ng
Pa#ent-‐Centered
and

3.  Communica#on
and
Methodological
Research

Dissemina#on

4.  Addressing
Dispari#es

5.  Accelera#ng
Pa#ent-‐
Centered
and

Methodological
Research

Why
PCORI
is
Unique

PCORI
is
unique
because:
(a)
it
requires
stakeholders
included
as
part
of
the
research

team,
and
(b)
research
must
be
focused
on
pa4ent-‐centered
outcomes

•  Projects
must
include
stakeholders
as
partners
with
signiﬁcant
involvement
at
all
appropriate

stages
of
the
research
project

•  Tangible,
meaningful
outcomes
are
the
ul#mate
goal
of
all
funded
research

Who
are
Stakeholders?

•  Pa#ents
and
caregivers
•  Payers

•  Pa#ent
and
caregiver
organiza#ons
•  Industry

•  Clinician
and
clinician
organiza#ons
•  Researchers

•  Organiza#onal
Providers
•  Policymakers

•  Purchasers
•  Training
ins#tu#ons

•  Others
who
can
bring
insight

Stakeholder
Engagement

PCORI
is
seizing
the
opportunity
to
engage
stakeholders
in
unprecedented
ways:

•  Partners
in
the
research
project
enterprise

Why Engage
•  Inclusion
as
equal
partners
in
research
review
Stakeholders?

•  Leverage
their
value,
including
wisdom
and
unique
To
create
more
relevant

decision-‐making
tools
to
assure

exper#se
be9er
pa#ent
outcomes

•  Increase
the
relevance
and
impact
of
research
by

integra#ng
mul#ple
stakeholders
into
the
process

•  Foster
environments
that
facilitate
cross-‐fer#liza#on
and
novel
collabora#ons

PCORI
Funding
Announcements

PCORI
Funding
Announcements
(PFAs)
are
the
mechanisms

by
which
PCORI
gives
out
research
funding

The
current
funding
cycle
has
four
issued
PFAs:
And
coming
this
fall
2012:

Assessment
of
Improving

Accelera'ng
Pa'ent-‐
Preven4on,
Healthcare

Centered
Outcomes

Diagnosis,
and
Systems

Research
and

Treatment
Op4ons

Methodological

Research

Communica4on

and
Addressing

Dissemina4on
Dispari4es

Research

PFAs

Assessment
of
Preven4on,

Diagnosis,
and
Treatment
Op4ons

Projects
that
address
cri4cal
decisions
that
pa4ents,
their
caregivers,
and
clinicians
face

with
too
lible
informa4on

In
this
PFA
we
seek
to
fund
projects
that:

Available
funds:

$48
Million

•  Address
cri'cal
decisions
that
face
pa'ents,
their

caregivers,
and
clinicians
every
day
and
with
too
Expected
awards:

54
awards

li?le
informa'on

Maximum
project
period:
3

years

•  Address
consequen'al
decisions
now
occurring

without
key
evidence
about
the
compara've
Ini#al
funding
period:

eﬀec'veness
of
two
or
more
op'ons
December
2012
–
January

2013

•  Beneﬁt
pa'ents/caregivers
with
new
knowledge
in

ways
that
are
clear
and
important

PFAs

Improving
Healthcare
Systems

Projects
that
address
cri4cal
decisions
that
face
healthcare
systems,
the
pa4ents
and

caregivers
who
rely
on
them,
and
the
clinicians
who
work
within
them

In
this
PFA
we
seek
to
fund
projects
that:

Available
funds:

$24
Million

•  Address
cri'cal
decisions
that
face
healthcare

system
leaders
and
policymakers,
clinicians,
and
Expected
awards:

27
awards

the
pa'ents
and
caregivers
who
rely
on
them

Maximum
project
period:
3

years

•  Oﬀer
substan'al
poten'al
that
pa'ents/caregivers

will
beneﬁt
from
the
new
knowledge
in
ways
that
Ini#al
funding
period:

are
important
December
2012
–
January

2013

PFAs

Communica4on
and

Dissemina4on
Research

Projects
that
address
cri4cal
elements
in
the
communica4on
and
dissemina4on
process

among
pa4ents,
their
caregivers
and
clinicians

In
this
PFA
we
seek
to
fund
projects
that:

Available
funds:

$12
Million

•  Address
cri'cal
knowledge
gaps
in
the

Expected
awards:

14
awards

communica'on
and
dissemina'on
process

Maximum
project
period:
3

•  Gaps
to
consider:

years

Ø  The
communica'on
and
dissemina'on
of

research
results
to
pa'ents,
their
caregivers,
Ini#al
funding
period:

and
clinicians

December
2012
–
January

Ø  The
communica'on
between
pa'ents,
2013

caregivers,
and
clinicians
in
the
service
of

enabling
pa'ents
and
caregivers
to
make
the

best
possible
decisions
in
choosing
among

available
op'ons
for
care
and
treatment

PFAs

Addressing
Dispari4es

Projects
that
will
inform
the
choice
of
strategies
to
eliminate
dispari4es

In
this
PFA
we
seek
to
fund
projects
that:

•  Will
inform
the
choice
of
strategies
to
eliminate
Available
funds:

$12
Million

dispari'es

Ø  We
are
not
interested
in
studies
that
Expected
awards:

14
awards

describe
dispari'es;
instead,
we
want

studies
that
will
iden'fy
best
op'ons
for
Maximum
project
period:
3

elimina'ng
dispari'es
years

Ini#al
funding
period:

•  Focus
on
areas
of
importance
to
pa'ents
and
December
2012
–
January

their
caregivers,
where
there
are
cri'cal
2013

dispari'es
that
disadvantage
members
of
a

par'cular
group
and
limit
their
ability
to
achieve

op'mal,
pa'ent-‐centered
outcomes

Agenda

1.  Introduc#on
and
Announcements

2.

Background

3.

Program
Funding
Announcements

4. The
Applica4on
and
Review
Process

5. Merit
Review:
Phase
I
–
Scien#ﬁc
Review

6.

Merit
Review:
Phase
II
–
Impact
Review

7.

Merit
Review:
Phase
II
–
In-‐Person
Panel

8.

Phase
II:
PCORI
Online
–
Process
and
Procedures

Applica4on
and
Review
Process
Summary

Merit

Applica4on
Approval

Review

•  LOI
submission
via
•  Phase
I:
Scien#ﬁc/
•  PCORI
Business

PCORI
Online
Technical
Review
Review
and

Balance
Analysis

•  Applica#on
•  Phase
II:
In-‐person

submission
via
panel
–
Impact
•  Board
of

PCORI
Online
Review
Governors

Approval

•  Internal
quality

control

The
Merit
Review
Process

The
process
by
which
applica4ons
for
research
funding
are

evaluated
–

Phase
I

•  Each
applica#on
is
assigned
to
a
pre-‐
determined,
set
number
of
reviewers
Phase
II

•  Phase
I
Reviewers
have
scien#fic
•  Scien#st
and
Pa#ents/Stakeholders

exper#se,
and
assess
the
applica#on
for
assess
Phase
I
cri#que
and
assign
one
(1)

scien#fic
rigor
and
research
approach
preliminary
score
and
cri#que

•  Reviewers
assign
an
ini#al
priority
score
•  Panel
of
chairs
plus
two
scien#fic,
one

of
1
to
9
based
on
all
eight
PCORI
merit
stakeholder,
and
one
pa#ent
reviewer

review
criteria
convene
in-‐person
for
discussion
and
re-‐
•  Scores
are
compiled

score

•  Top
scoring
applica#ons
proceed
to

Phase
II

Merit
Review
Phase
II:
Overview

As
a
Phase
II
Reviewer,
you
are
responsible
for
assessing
and

appropriately
scoring
your
assigned
applica4ons

Assign
In-‐Person

Access
Assigned
Final
Impact

COI
Preliminary
Score
Review

Cri4ques
&
Scores
Scoring

&
Cri4que
Panels

Key Tasks

1.  Access
Phase
II
scores
and
cri#ques
in
PCORI
Online

2.  Conflict
of
Interest
(COI):
Ensure
no
conflict
exits

3.  Assign
preliminary
numerical
preliminary
impact
score
(1-‐9)
and
provide
cri#que

4.  Panels
convene
and
discuss

5.  Assign
final
impact
scores

Raise
issues,
risks,
and
request
support
as
needed

Conflicts
of
Interest

What
is
a
conflict
of
interest?

As
defined
by
PCORI’s
establishing
legisla#on,
a
conflict
of
interest
is
any
“associa#on,
including
a

financial
or
personal
associa#on;
that
has
the
poten#al
to
bias
or
have
the
appearance
of
biasing

an
individual’s
decisions
in
ma9ers
related
to
the
Ins#tute
or
the
conduct
of
ac#vi#es”.

Conflicts
of
interest
will
be
considered
and
prohibited
throughout
every
step
of
the
review
and

selec#on
process,
including
but
not
limited
to:
the
technical
and
programma#c
reviews,
the

selec#on
and
assignment
of
scien#fic
and
stakeholder
reviewers,
Board
of
Governors

delibera#ons,
and
post-‐award
nego#a#ons
and
monitoring.

More
informa4on
is
included
in
the
PCORI
Online
confiden4ality
and
conflict
of
interest
document

that
you
must
agree
to
in
order
to
access
your
applica4ons.

Your
Role

Some
addi4onal
guidance
about
your
role
and
ac4vi4es
as
a
Phase

II
Scien4fic
Reviewer:

Before
the
in-‐person
review
panels
on
November
15th:

•  Access
your
assigned
applica#ons
in
PCORI
Online

•  Score
and
provide
wri9en
comments

•  Be
prepared
to
substan#vely
qualify
and
discuss
your
score
and
comments
during

the
in-‐person
review
panel

During
the
in-‐person
review
panel:

•  Reviewers
assigned
to
each
applica#on
will
briefly
discuss
their
preliminary
score

and
provide
feedback

•  Open
to
panel-‐wide
discussion

•  All
reviewers
assign
a
final
score
in
real-‐#me,
on
personal
laptops
via
PCORI

Online

•  Scores
are
compiled
and
averaged
by
PCORI

•  Top
scoring
applica#ons
will
be
forwarded
and
receive
recommenda#on
for

funding

Agenda

1.  Introduc#on
and
Announcements

2.

Background

3.

Program
Funding
Announcements

4. The
Applica#on
and
Review
Process

5. Merit
Review:
Phase
I
–
Scien4ﬁc
Review

6.

Merit
Review:
Phase
II
–
Impact
Review

7.

Merit
Review:
Phase
II
–
In-‐Person
Panel

8.

Phase
II:
PCORI
Online
–
Process
and
Procedures

Merit
Review
Criteria

•  Impact
of
the

Condi4on

•  Innova#on/

Criterion
1:
Impact
of
the
Condi?on

on
the
Health
of
Individuals
and
Popula?ons

Poten#al
for

Improvement
§  Does
the
applica#on
specify
the
burden
of
the
disease
or
area
under

•  Impact
on
considera#on,
with
a
preference
for
the
U.S.
popula#on,
including:

Healthcare
Ø  The
frequency
of
the
disease/condi#on,

Performance
Ø  Expected
mortality
and
burden
of
suffering
from
symptoms,

•  Pa#ent-‐ Ø  Complica#ons
or
other
consequences
of
the
disease/condi#on,

Centeredness
Ø  The
frequency
with
which
the
interven#on
or
treatment
would

•  Rigorous
Research
apply,

Methods
Ø  Costs
to
the
US
popula#on
(healthcare
services
u#liza#on),
and

to
individual
pa#ents
(out-‐of-‐pocket
and
intangible
costs).

•  Inclusiveness
of

Different
§  Primary
emphasis
is
on
chronic
condi#ons,
as
well
as
preven#on
and

Popula#ons
treatment
of
common
acute
events
that
may
have
long-‐term

•  Team
and
consequences.

Environment

§  Studies
that
are
relevant
to
pa#ents
with
two
or
more
condi#ons
are

•  Efficient
Use
of
also
of
interest.
Also
of
interest
are
rare
diseases.

Resources

Merit
Review
Criteria

•  Impact
of
the

Condi#on

•  Innova4on/

Poten4al
for
Criterion
2:
Innova?on
and
Poten?al
for
Improvement
Through

Improvement
Research

•  Impact
on
§  How
will
the
research
influence
current
prac#ce
and
lead
to
meaningful

Healthcare
improvement
in
pa#ent
health,
well-‐being,
or
quality
of
care?

Performance

§  Does
the
research
involve
a
novel
interven#on
or
employ
an
innova#ve

•  Pa#ent-‐ approach
in
terms
of
analy#cs,
study
popula#on,
or
research
team
that

Centeredness
makes
it
more
likely
to
change
prac#ce?

•  Rigorous
Research
§  Does
preliminary
data
suggest
that
the
comparison
will
show
large

Methods
differences
in
effec#veness?

of
§  Does
the
research
ques#on
address
a
cri#cal
gap
in
current
knowledge?

Different
Has
it
been
iden#fied
as
important
by
pa#ent,
caregiver,
or
clinician

Popula#ons
groups?
Have
other
agencies
iden#fied
this
topic
as
a
priority?

•  Team
and

§  How
quickly
could
posi#ve
findings
be
disseminated
to
affect
changes
in

Environment

current
prac#ce?
How
will
the
research
findings
support
improved

•  Efficient
Use
of
decision-‐making
for
pa#ents?

Resources

Merit
Review
Criteria

•  Impact
of
the

Condi#on

•  Innova#on/

Poten#al
for

Improvement

Criterion
3:
Impact
on
Healthcare
Performance

•  Impact
on

Healthcare

§  What
is
the
impact
of
the
proposed
research
on
the
efficiency
of

Performance
pa#ent
care,
for
individual
pa#ents
or
for
pa#ent
popula#ons?

•  Pa#ent-‐ §  For
example,
do
the
findings
lead
to
be9er
outcomes
for
a
given

Centeredness
investment
of
#me,
personnel,
or
other
resources?
Or
does
the

research
promise
poten#al
improvements
in
convenience
or

•  Rigorous
Research

elimina#on
of
wasted
resources,
while
maintaining
or
improving

Methods

pa#ent
outcomes?

of

Different

Popula#ons

•  Team
and

Environment

•  Efficient
Use
of

Resources

Merit
Review
Criteria

•  Impact
of
the

Condi#on

•  Innova#on/

Poten#al
for
Criterion
4:
Pa?ent-‐Centeredness

Improvement

§  Is
the
proposed
research
focused
on
ques#ons
and
outcomes
of

•  Impact
on
specific
interest
to
pa#ents
and
their
caregivers?
Pa4ent-‐
Healthcare

centeredness
is
a
perspec4ve
on
health
that
is
derived
from
and

Performance

directly
relevant
to
the
pa4ent’s
experience
of
illness
and
of
care.

•  Pa4ent-‐
Centeredness
§  Does
the
research
address
one
or
more
of
the
key
ques#ons

men#oned
in
PCORI’s
defini#on
of
pa#ent-‐centered
outcomes

•  Rigorous
Research

research?

Methods

of
§  Are
the
outcomes
proposed
of
importance
to
pa#ents?
Is
the

Different
absence
of
any
par#cularly
important
outcomes
discussed?

Popula#ons

§  Pa#ent
engagement
in
the
research
team
is
dis#nct
and
discussed
in

•  Team
and
Criterion
7,
Team
and
Environment.

Environment

•  Efficient
Use
of

Resources

Merit
Review
Criteria

•  Impact
of
the

Condi#on

•  Innova#on/

Poten#al
for
Criterion
5:
Rigorous
Research
Methods

Improvement

§  Does
the
research
use
appropriate
and
rigorous
research
methods

•  Impact
on
to
generate
pa#ent-‐centered
evidence?

Healthcare

Performance
Ø  Applicants
are
encouraged
to
refer
to
the
contents
of
the

first
dras
of
the
PCORI
Methodology
Report,
at

•  Pa#ent-‐
h9p://www.pcori.org/what-‐we-‐do/methodology,
in

Centeredness

developing
their
research
plan.
Because
the
dras
report
will

•  Rigorous
Research
not
have
been
finalized
with
the
benefit
of
public
comment

Methods
before
the
July
31st,
2012
applica#on
deadline,
adherence
to

of
the
Report’s
standards
will
not
be
a
required
element
of

Different
applica#ons
for
this
funding
cycle.

Popula#ons
Ø  How
likely
is
it
that
the
proposed
study
popula#on,
study

•  Team
and
design,
and
available
sample
size
will
yield
generalizable

Environment
informa#on
with
sufficient
precision
to
be
useful
and
reliable

for
pa#ents,
their
caregivers,
and
clinicians?

•  Efficient
Use
of

Resources

Merit
Review
Criteria

•  Impact
of
the

Condi#on

•  Innova#on/

Poten#al
for

Criterion
6:
Inclusiveness
of
Different
Popula?ons

Improvement

•  Impact
on

§  Does
the
research
include
diverse
popula#ons
with
respect
to

Healthcare
age,
gender,
race,
ethnicity,
geography,
or
previously

Performance
understudied
popula#ons
for
whom
effec#veness
informa#on
is

par#cularly
needed?
Is
the
study
popula#on
representa#ve
of
the

•  Pa#ent-‐
full
popula#on
of
interest?

Centeredness

•  Rigorous
Research
§  How
does
the
proposed
research
enable
a
more
personalized

Methods
approach
to
decision-‐making
based
on
a
pa#ent’s
unique

biological,
clinical,
or
socio-‐demographic
characteris#cs?

of

Different
§  Does
the
study
provide
sample
size
calcula#ons
that
will
describe

Popula4ons
the
power
available
to
evaluate
possible
differences
in

•  Team
and
effec#veness
in
different
groups,
or
the
precision
available
for

Environment
es#ma#ng
effec#veness
in
a
specific
previously
understudied

popula#on?

•  Efficient
Use
of

Resources

Merit
Review
Criteria

•  Impact
of
the

Condi#on

•  Innova#on/

Poten#al
for

Criterion
7:
Team
and
Environment

Improvement

•  Impact
on

§  Are
the
inves#gators
appropriately
trained
and
experienced
to

Healthcare
carry
out
the
planned
studies?
Is
the
work
proposed
appropriate

Performance
to
the
experience
level
of
the
principal
inves#gator?

•  Pa#ent-‐ §  Does
the
study
team
have
complementary
and
integrated

Centeredness
exper#se;
is
their
leadership
approach,
governance,
and

•  Rigorous
Research
organiza#onal
structure
appropriate
for
the
project?

Methods
§  Are
relevant
pa#ents
and
other
key
stakeholders
of
the
study

of
informa#on
appropriately
included
on
the
team?

Different

Popula#ons
§  Do
the
experiments
proposed
take
advantage
of
unique
features

of
the
scien#fic
environment
or
employ
useful
collabora#ve

•  Team
and
arrangements?

Environment

•  Efficient
Use
of

§  Is
there
evidence
of
ins#tu#onal
or
other
support?

Resources

Merit
Review
Criteria

•  Impact
of
the

Condi#on

•  Innova#on/

Poten#al
for

Criterion
8:
Efficient
Use
of
Research
Resources

Improvement

•  Impact
on

§  Does
the
budget
appear
to
be
reasonable
in
rela#on
to
the

Healthcare
poten#al
contribu#on
of
the
research?

Performance

§  Does
the
jus#fica#on
address
the
efficiency
with
which
PCORI

•  Pa#ent-‐ resources
would
be
used?
Are
there
opportuni#es
to
make
the

Centeredness
study
more
efficient?

•  Rigorous
Research
§  Are
there
addi#onal
benefits
to
a
PCORI
investment
in
this
study

Methods
through
the
crea#on
of
common
data
or
infrastructure
that
could

of
support
future
research?

Different

Popula#ons

•  Team
and

Environment

•  Efficient
Use
of

Resources

The
Focus
on
Impact

PCORI
Deﬁni4on

Reviewers
will
provide
an
overall
impact

score
that
considers
the
following:

•  Does
the
project
have
poten4al
to
The
assessment
of
impact
is
par4cularly

change
clinical
prac4ce
or
pa4ent
informed
by
three
of
the
eight
PCORI

behavior
in
ways
that
will
create
and
Merit
Review
Criteria.

sustain
improvement
in
outcomes

and
the
health
of
pa4ents?
Use
criteria
2,
4,
and
7
to
evaluate
an

applica4on’s
answer
to
these
ques4ons

•  How
quickly
can
the
results
of
the

project
be
disseminated
and
applied

(from
the
assessment
of

dissemina4on
and
implementa4on

poten4al)?

The
8
Merit
Review
Criteria:
Phase
II
Focus

Phase
II
is
centered
around
impact,
focusing

on
the
following
of
the
Merit
Review
Criteria:

The
8
Merit
Review
Criteria:

Innova4on/

1.  Impact
of
the
Condi#on
Poten4al
for
Pa4ent
Team
and

Improvement
Centeredness
Environment

2.  Innova4on/Poten4al
for

•  Innova#on
–
in
•  Focus
on
•  Inves#gators

Improvement

ways
that
are
ques#ons
and
trained

3.  Impact
on
Healthcare
likely
to
change
outcomes
of
•  Study
team

prac#ce?
specific
interest
exper#se

Performance

•  Poten#al
for
to
pa#ents
and
•  Plan
for
leadership

improvement
their
caregivers
and
governance

4.  Pa4ent-‐Centeredness

(will
findings
•  Robust
pa#ent

5.  Rigorous
Research
Methods
improve
pa#ent
and
stakeholder

well-‐being
or
engagement
plan

6.  Inclusiveness
of
Different
quality
of
care?)
of

different

Popula#ons

popula#ons

7.  Team
and
Environment
•  Ins#tu#onal
or

other
relevant

8.  Efficient
Use
of
Resources
organiza#onal

support

Phase
II
Key
Focus
Areas

Criterion
2:
Innova?on
and
Poten?al
for

Improvement
Through
Research

•  “Is
there
uncertainty?”

–  Varia#on
in
prac#ce,
systema#c
reviews
have
iden#fied
as
such,
or
pa#ent/clinician
groups
have

specifically
called
for
this
informa#on

•  How
will
the
research
influence
current
prac#ce
and
lead
to
meaningful
improvement
in
pa#ent
health,

well-‐being,
or
quality
of
care?

•  Does
the
research
involve
a
novel
interven#on
or
employ
an
innova#ve
approach
in
terms
of
analy#cs,

study
popula#on,
or
research
team
that
makes
it
more
likely
to
change
prac#ce?

•  Does
preliminary
data
suggest
that
the
comparison
will
show
large
differences
in
effec#veness?

•  Does
the
research
ques#on
address
a
cri#cal
gap
in
current
knowledge?
Has
it
been
iden#fied
as
important

by
pa#ent,
caregiver,
or
clinician
groups?
Have
other
agencies
iden#fied
this
topic
as
a
priority?

•  How
quickly
could
posi#ve
findings
be
disseminated
to
affect
changes
in
current
prac#ce?
How
will
the

research
findings
support
improved
decision-‐making
for
pa#ents?

–  PCORI
is
interested
in
funding
studies
with
a
high
likelihood
that
results
will
be
disseminated
and

incorporated
into
prac#ce
immediately
or
within
a
short
period
of
#me
(3-‐
5
years).

Please
refer
to

the
dissemina#on
and
implementa#on
assessment
in
the
applica#on
for
detail
and
clarifica#on,
if

necessary.

Phase
II
Key
Focus
Areas

Criterion
4:
Pa?ent
Centeredness

•  Is
the
proposed
research
focused
on
ques#ons
and
comparisons
that
have
relevance
and

specific
interest
to
pa#ents
and
their
caregivers?
Pa4ent-‐centeredness
is
a
perspec4ve
on

health
that
is
derived
from
and
directly
relevant
to
the
pa4ent’s
experience
of
illness
and
of

care.

•  Does
the
research
fit
with
one
or
more
of
the
key
ques#ons
men#oned
in
PCORI’s
defini#on
of

pa#ent-‐centered
outcomes
research?

•  Are
the
outcomes
proposed
of
importance
to
pa#ents?
Is
the
absence
of
any
par#cularly

important
outcomes
discussed?

•  Note:
Pa#ent
engagement
in
the
research
team
is
dis#nct
and
discussed
in
Criterion
7,
Team

and
Environment.

Phase
II
Key
Focus
Areas

Criterion
7:
Team
and
Environment

•  Are
the
inves#gators
appropriately
trained
and
experienced
to
carry
out
the
planned
studies?

Is
the
work
proposed
appropriate
to
the
experience
level
of
the
principal
inves#gator?

•  Does
the
study
team
have
complementary
and
integrated
exper#se;
is
their
leadership

approach,
governance,
and
organiza#onal
structure
appropriate
for
the
project?

•  Are
relevant
pa#ents
and
other
key
stakeholders
in
the
study
informa#on
appropriately

included
on
the
team?

•  Do
the
proposed
experiments
take
advantage
of
unique
features
of
the
scien#ﬁc
environment,

or
employ
useful
collabora#ve
arrangements?

•  Is
there
evidence
of
ins#tu#onal
support?

Phase
II
Preliminary
Scoring

Assignments
Released
Preliminary
Scoring

•  Assignments
are
released
in
•  Use
Phase
I
cri#ques
to
assign

October
preliminary
score
of
1-‐9

–  You
will
have
access
to
the
full

•  In
PCORI
Online
applica#on,
but
please
use
only
to

reference

•  Login
to
access
your
applica#ons

–  Ensure
no
conflicts
of
interest,
•  Provide
substan#ve,
produc#ve

and
your
qualifica#on
to
review
comments
as
well
as
specific

strengths
and
weaknesses
to

•  All
applica#ons
re-‐released
and
ul#mately
help
answer
the
ques#on:

assigned
to
Phase
II
Reviewers
–  “How,
and
to
what
extent,
will
the

were
top
scorers
in
Phase
I
proposed
research
plan
impact

–  Have
been
ve9ed
for
basic,
hard
pa'ents
in
the
next
3
to
5
years?”

science
and
programma#c

review
•  Enter
comments
and
numerical
score

in
PCORI
Online

Reviewer
Guidance
&
Scoring
Chart

For
the
preliminary
Phase
II
impact
score,
the
far
right
column
in
the
scoring
chart
below

provides
a
descrip4ve
guide
of
how
strengths
and
weaknesses
are
considered
in
a
ra4ng:

Impact Score Descriptor
1 Exceptional
High 2 Outstanding
3 Excellent
4 Very
Good
Medium 5 Good
6 Satisfactory
7 Fair
Low 8 Marginal
9 Poor

General
Logis4cs
for
Phase
II
Panel
Reviewers

Date/Loca4on
Details

Date
of
Phase
II
Panels:
Thursday,
November
15,
2012

Loca4on:
Hya9
Regency
Washington
on
Capitol
Hill,
Washington
D.C.

Time:

8:00am
to
3:00pm

Number
of
Panels:

Five

Ø  Panel
par#cipants
will
reﬂect
the
fact
that
each
applica#on
has
2
scien#sts,
one

stakeholder,
and
one
pa#ent
assigned
to
provide
commentary
and
preliminary
scores

Phase
II
Panel
Process

Introduction Presentation

Overview and Triage Co-Chair Presents Application Synopsis

•  Chair
Introduc#on
•  SRO
captures
panel
discussion

‒  Stress
confiden#ality
and
focus
on
impact

•  Individual
review
and
scoring
on
PCORI
Online
on

•  Triage
process
to
eliminate
lowest-‐ranked
personal
laptops

applica#ons
from
panel
discussion

Review Scoring

Discussion of Application Open to Discussion

•  Provides
their
preliminary
impact
score
and
•  All
panelists
free
to
discuss,
Chair
moderates
if

assessment

and
its
poten#al
for
significant
needed

outcomes/impact

•  If
no
discussion
–
move
to
final
vote.
Reviewers

score
individually
in
PCORI
Online

Triage
Process

During
Phase
II,
a
triage
process
will
take
place:

Ø  Applica#ons
are
ordered
according
to
ranking
(highest
to
lowest
scores)

Ø  Lowest
scoring
applica#ons
will
be
eliminated
all
at
once
from
in-‐person
panel

discussion

Ø  If
you
as
a
Reviewer
want
to
speciﬁcally
discuss
an
applica#on,
please
come
to

the
panels
prepared
to
do
so

Roles
&
Responsibili4es

Each
panel
will
be
lead
by
a
Chair,
Co-‐Chair,
and
SRO
in

tradi4onal
advisory
roles

Scien4fic
Reviewers
(2)
Pa4ent
Reviewer
Stakeholder
Reviewer

Role
Provide
addi#onal
depth
for

Provides
addi#onal
depth
for

up
to
10
applica#ons
for
all

up
to
10
applica#ons
for
all

par#cipants

par#cipants

Key
Provides
their
preliminary
impact
score
and
assessment

and
its
poten#al
for
significant
outcomes/
Responsibili4es
impact

PCORI
review
is
different
because
there
is
no
primary/secondary/ter#ary
reviewer
structure

•  Also
u#lizes
different,
PCORI-‐unique
merit
review
criteria

Each
reviewer
must
provide
substan#ve
strengths
and
weaknesses
for
his/her
assigned

applica#on(s)

•  Reviewers
must
be
prepared
to
speak
to
these
comments
and
scoring
during
the
in-‐person

review
panels

Time
Breakdown
per
Applica4on

NOTE:
Panels
will
spend
no
more
than
10-‐15
minutes
per

applica4on.
An
example:

Up
to…
Descrip4on

1
minute
Co-‐chair
briefly
introduces
applica#on

2
minutes
Scien#fic
Reviewer
#1:
overview
and
score

2
minutes
Stakeholder
reviewer:
overview
and
score

2
minutes
Scien#fic
Reviewer
#2:
overview
and
score

2
minutes
Pa#ent
Reviewer:
overview
and
score

4
minutes
General
discussion,
if
any

2
minutes
Take
vote
and
enter
scores
in
PCORI
Online

Ø  Some
applica#ons
may
be
reviewed
in
less
than
the
15
minutes
allocated.

Ø  It
is
important
to
understand
the
#me
constraints
and
keep
conversa#on
focused,
pointed,
and

succinct
throughout
the
day
to
ensure
fair
and
proper
scoring
of
all
applica#ons

PCORI
Online:
Confiden4ality
Agreement

The
first
thing
you
will
have
to
do
upon
log-‐in
is
agree
to
the
Confiden4ality
Agreement.
This
applies

to
both
preliminary
scoring,
and
any/all
scores
and
discussion
that
take
place
during
the
in-‐person

review
panel.

Accessing
Your
Assigned
Applica4ons

Next,
you
will
be
able
to
see
your
list
of
assigned
applica4ons
by
selec4ng
“Review
Assignments”
in

the
side
bar
on
the
lep
side
of
your
screen.
The
list
will
appear
similar
to
below:

Note
any
Conﬂicts
of
Interest

Use
the
drop-‐down
box
to
note
any
COIs.

If
there
is
a
COI,
use
the
second
drop-‐down
box
to
indicate
type.

Accessing
and
Scoring

Once
you
have
conﬁrmed
there
are
no
COIs,
three
new
icons
will

appear
to
the
right
of
an
assigned
applica4on:

Ø  The
ﬁrst
symbol
(farthest
to
the
lep),
a
PDF
form,
is
the
complete
applica4on

Ø  The
second
symbol
(in
from
lep)
opens
the
applica4on
abstracts

Ø  The
third,
orange
symbol
is
an
applica4on’s
Phase
I
Reviews
(scores
and
wriben
cri4ques)

Ø  The
last
symbol
on
the
far
right
opens
your
Cri4que
Form

Preliminary
Scoring
in
PCORI
Online

The
PCORI
Online
scoring
screen
for
preliminary
impact
scoring:

Preliminary
Scoring:
Review
and
Submit

Final
screen
once
preliminary
scores
are
submibed:

Wrap-‐Up

This
concludes
today’s
session.
We
hope
you
found
this
training
helpful
and
informa#ve.

Thank
you
again
for
your
commitment
to
PCORI.

If
any
ques'ons
remain
unanswered
at
this
point,
please
email
them
to

reviewers@pcori.org

Final phase ii scientific reviewer training

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (9)

Semelhante a Final phase ii scientific reviewer training

Semelhante a Final phase ii scientific reviewer training (20)

Mais de Patient-Centered Outcomes Research Institute

Mais de Patient-Centered Outcomes Research Institute (20)

Último

Último (20)

Final phase ii scientific reviewer training