Bibhas Roy, an orthopaedic consultant at Trafford Hospital slides about his work with Patient Recorded Outcome Measures (PROMs)

1. PROMs 2.0
Bibhas Roy, Consultant Orthopaedic Surgeon
Trafford General Hospital, CMFT
Clinical IT Lead for Secondary Care, NHS North West

2. Experience & Outcome
Definitions
Measuring Healthcare
Validity
PROMs – National approach
PREMs
Example - PROMs 2.0
2 A Practical Guide to Measuring and 08/03/2012
Monitoring Patient Experience

3. Quality in Healthcare
 “Even though quality cannot be defined, you know what quality
is.”
 Robert M. Pirsig 1928 American philosopher
 “Quality is the degree to which health services for individuals
and populations increase the likelihood of desired health
outcomes and are consistent with current professional
knowledge.”
 Institute of Medicine 1990
 “Quality is a process of meeting the needs and expectations of
patients and health service staff.”
 WHO 2000
 We are guests in our patients' lives; and we are their hosts
when they come to us. Why should they, or we, expect anything
less than the graciousness expected by guests and from hosts
at their very best. Service is quality – Don Berwick
 "The Permanente Journal, Volume 3 No. 1". Kaiser Permanente. Winter 1999.
3 A Practical Guide to Measuring and 08/03/2012
Monitoring Patient Experience

4. Donald M. Berwick
Chief Executive Officer of the
Institute for Healthcare Improvement
 My right knee will probably need to be replaced soon. This has given me
the opportunity to define, in very personal terms, 5 specific dimensions
of ―total quality‖ that I will require from the medical institution that does
my surgery and that every patient has the right to require of their
encounters with the health care system.
- Don‘t kill me (no needless deaths).
- Do help me, and don‘t hurt me (no needless pain).
- Don‘t make me feel helpless.
- Don‘t keep me waiting.
- And don‘t waste resources, mine or anyone else‘s.
 Given my requirements, it is not clear that any health
care institution in the United States will want to take
me on as a patient…‖
4 A Practical Guide to Measuring and 08/03/2012
Monitoring Patient Experience

5. Measuring Healthcare
This has become a multi-million
pound industry fuelled partly by
increasing anxiety by society
(especially its political
representatives) about the
variation in quality and safety of
care—an anxiety heightened as
the results of more
measurements reveal even
more problems. Whenever such
an industry develops rapidly, it is
useful to pause and reflect on
the degree to which it is acting
optimally and in the interests of
society and health.
The healthcare quality measurement industry: time to slow the juggernaut? Professor T A Sheldon
Department of Health Sciences, University of York Qual Saf Health Care 2005
5 A Practical Guide to Measuring and 08/03/2012
Monitoring Patient Experience

6. Prof. Iain Buchan, University of Manchester
6 A Practical Guide to Measuring and 08/03/2012
Monitoring Patient Experience

7. Measures
Institute for Healthcare Improvement (IHI)
Balancing Measures - Are
Outcome Measures – What is Process Measures - changes designed to improve a
the result parts/steps in the system part of the system causing new
problems in other areas
• For access: Number of days • For access: Average daily • For reducing patients' length
to appointment clinician hours available For of stay in the hospital: Make
• For critical care: ICU mortality critical care: Use of adverse sure readmission rates are not
drug event chart review increasing
7 A Practical Guide to Measuring and 08/03/2012
Monitoring Patient Experience

8. Hawthorne effect
Hawthorne Works
• 1924 -1932
• commissioned a
study to see if its
workers would
become more
productive in
higher or lower
levels of light.
8 A Practical Guide to Measuring and 08/03/2012
Monitoring Patient Experience

9. Outcome Measures
Blunt figures
• Mortality / Number of treatments performed
Clinician / Physician reported
• Evolution from broad subjective categories to
validated sensitive quantified tools
• Eg - Constant-Murley Shoulder Score
Patient Reported Measures
9 A Practical Guide to Measuring and 08/03/2012
Monitoring Patient Experience

10. Stratification/Adjusting for risk
Conclusions that
Variation in outcome
outcome differences are Factor not measured or
may be due to
caused by differences in inadequately
differences in
quality will always be measured?
healthcare quality - BUT
tentative.
Type of patient -
age, gender, co-
morbidity, disease
severity, socio-economic
status etc…
Data collection / reporting -
numerator / denominator /
case mix adjustment
definitions
Chance
10 A Practical Guide to Measuring and 08/03/2012
Monitoring Patient Experience

11. PROMs & PREMs
Unidimensional / Condition Desirable
Multidimensional Targeted / Generic Attributes
• Important
• Sound
theoretical basis
• Validated
• Reliable
• Sensitive
• Acceptable
The Point of Care Measures of patients‘ experience in hospital: purpose, methods and
uses - Angela Coulter, Ray Fitzpatrick, Jocelyn Cornwell, July 2009 – The King‘s Fund
11 A Practical Guide to Measuring and 08/03/2012
Monitoring Patient Experience

12. Experience & satisfaction
Experience
•what actually occurred, rather than the patient‘s
evaluation of what occurred.
Satisfaction
• the personal preferences of the patient
• the patient‘s expectations
• response tendencies due to personal characteristics
• the quality of the care received
12 A Practical Guide to Measuring and 08/03/2012
Monitoring Patient Experience

13. PROMs
• standardised validated
instruments (question sets)
• measure patients‘ perceptions of
their health status (impairment),
• their functional status (disability)
• their health-related quality of life
(well-being).
13 A Practical Guide to Measuring and 08/03/2012
Monitoring Patient Experience

14. Spreading the project
People are not passive recipients of innovations. Rather (and to a
greater or lesser extent in different individuals), they seek
innovations out, experiment with them, evaluate them, find (or fail to
find) meaning in them, develop feelings (positive or negative) about
them, challenge them, worry about them, complain about them, ‗work
round‘ them, talk to others about them, develop know-how about
them, modify them to fit particular tasks, and attempt to improve or
redesign them (often through dialogue with other users).
How to Spread Good Ideas, A systematic review of the literature on diffusion, dissemination and
sustainability of innovations in health service delivery and organisation
Trisha Greenhalgh et.al 2004
14

15. Product & Process
 Know your products
 Map processes
 Break it into components
and timeframes
 Discover what works well
 Discover what does not
work well

16. Clinical Engagement?
 Clinician
 a doctor having direct contact with patients rather than
being involved with theoretical or laboratory studies. -
oxforddictionaries.com
 HICAT version (for informatics)
 'Figurehead' Clinicians (Doctors, Nurses, AHPs etc.)
 These Clinicians no longer have clinical responsibility and
are not on the front-line
 Clinicians who work part-time in the Programme.
 These Clinicians work for 40-60% of the time in the
Programme and 40-60% of the time in clinical care
 Fulltime clinicians
 Clinicians who are fully committed to improving patient
care through the use of IT

17. Establishing the Culture and Beliefs to
Deliver Clinical Engagement
1. engagement follows debate at a local level
2. place the patient at the centre of development
3. evidence - ―observability & trialability‖ is required
Rogers & Plsek
4. local clinical leaders are essential – real full time clinicians
5. concept applied successfully in a local context, this ―strength of
evidence‖ is very compelling
6. align objectives between clinician and manager
7. a request to change their working process will produce
dissonance – change management

18. New Double Helix
Approach
Value
Belief
Engagement
Patients Programme
Pathway implementation
Driver Driver

19. Clinical Engagement Escalator
Aligned with Managerial Colleagues Project Plan
Framing as
described in
Large Scale
Change Business as
Usual
Clinical Champions to bring about
Clinical Engagement and Service
New Improvement becoming business Acceptance and
as usual implementation
Vision
Dissemination Evidence on quality
& benefit realisation
Early implementers
Clinical volunteers Evidence on quality
& benefit realisation
Awareness-raising

20. Upscaling the change
Sixty Three NHS Organisations Cancer
network
Trauma
Network Clinical Leaders GP Consortia
Network Network
An Ethos of
Medical
Clinical Expertise
BMA Implementation Quality, Design Directors
At LHC Governance
―Realise the
cultural change‖ Pathology
Mental PEC CHIL
Health Chairs Enabling
Health Informatics Respiratory
LMC
network
Children’s Stroke
Royal AHP Nursing
network network
Colleges network Directors

21. PROMs 2.0
 National PROMs
 PROMs are measures of a patient's health status or
health-related quality of life. They are typically short, self-
completed questionnaires, which measure the patients'
health status or health related quality of life at a single
point in time. – NHS Information Centre
 Must be Validated tools
 Shared decision making with patients
 QIPP principles
 Quality 
 Innovation 
 Productivity 

22. 22 PROMS 2.0 - Patient Generated Data and 26/07/2012
enhancing decisions

23. E-mail to patient

24. Validate the patients into the system
24 PROMS 2.0 - Patient Generated Data and 26/07/2012
enhancing decisions

25. No passwords
25 PROMS 2.0 - Patient Generated Data and 26/07/2012
enhancing decisions

26. Patient Generated data – their responsibilities
26 PROMS 2.0 - Patient Generated Data and 26/07/2012
enhancing decisions

27. List of scores that require completion
27 PROMS 2.0 - Patient Generated Data and 26/07/2012
enhancing decisions

28. Confirmation of Completion
28 PROMS 2.0 - Patient Generated Data and 26/07/2012
enhancing decisions

29. Logic based feedback to guide patients about their care
29 PROMS 2.0 - Patient Generated Data and 26/07/2012
enhancing decisions

30. Birth of PROMs 2.0
Business case Software
for £250,000 £15,000
• rejected
Innovations Dragon‘s Lair
funding (2010) £20,000 award
30

31. Phase II
Adoption bursaries for 10 more organisations
(2011) - £150,000
• 9 Acute Trusts
• 2 CCG
Clinical Leaders Network funding (2012)
• Further 2/3 organisations
Many different PROMs now in system
• Minimal Data set
• EQ5D
31

32. Experience & satisfaction
Experience (PREMs)
•what actually occurred, rather than the patient‘s
evaluation of what occurred.
Satisfaction
• the personal preferences of the patient
• the patient‘s expectations
• response tendencies due to personal characteristics
• the quality of the care received
32

33. Trust Speciality
Salford Royal NHS Foundation Trust Orthopaedics
Norfolk & Norwich University Hospital Orthopaedics
University Hospital of South Manchester Orthopaedics
East Lancashire Hospitals NHS Trust Orthopaedics
East Cheshire Clinical Commissioning Group Pulmonary rehabilitation
Stockport NHS Foundation Trust Orthopaedics
Central Manchester University Hospitals NHS Foundation Trust Anaesthetic, Urology
Wrightington, Wigan and Leigh NHS Foundation Trust Orthopaedics
United League Clinical Commissioning Group ENT
Countess of Chester Hospital Trauma Network
Royal Liverpool and Broadgeen University Hospitals Orthopaedics

34. PROMs 2.0 Team
 All organisations represented
 PROMs & PREMs Mapping
 Governance
 Consent
34 PROMS 2.0 - Patient Generated Data and 26/07/2012
enhancing decisions

35. PROMs 2.0 a success?
 Funding process
 The software development and pilot
 cheap due to the direct relationship between the stakeholders
and the designer
 No elaborate reports necessary to justify funding
 Implementation of product required
 Business requirements analysis to convince early
adoptors
 adoptors were actively chosen
 Future?
 PROMs Summit (6th December Manchester)
35 PROMS 2.0 - Patient Generated Data and 26/07/2012
enhancing decisions

36. Why do we fail?
 Lack of user involvement
 clinical engagement
 Poor requirements
 Long or unrealistic timeframes
 Scope creep- the scope increases insidiously as
the project progresses
 No change control system - especially in
consideration of changing requirements
 Poor testing- testing is not done by those on the
front-line, but by contract workers