This document discusses measurement for quality improvement. It explains that measurement in improvement aims to provide a basis for action to improve processes and outcomes, rather than just estimating parameters. Improvement measures should be simple, specific, and available in real-time. Statistical process control methods are important to separate normal variation from changes resulting from interventions. Examples are provided of run charts measuring improvements in recording BMI for mental health patients and compliance with care bundles. The document advocates making the theories behind improvement efforts more explicit.

1. Providing actionable healthcare analytics at scale:
Understanding improvement measures
Thomas Woodcock, Improvement Science Fellow, Imperial
College London

2. NIHR CLAHRC
Northwest London
Collaboration for Leadership in Applied Health Research and Care
Northwest London
The National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (NIHR CLAHRC) Northwest London
is hosted by Chelsea and Westminster Hospital NHS Foundation Trust and academically led by Imperial College London, in partnership with North West London
Understanding Improvement
Measures
Dr Tom Woodcock
Programme Lead Information Intelligence
Health Foundation Improvement Science Fellow

3. NIHR CLAHRC
Northwest London
To achieve this: Develop a comprehensive systematic approach to
improvement which is transferable and generalizable across the NHS
3
Primary aims
Translate research into practice
Improve health outcomes and patient experience
Deliver value locally and to the wider NHS
Working in collaboration with: Patients, communities, health and care
services, healthcare professionals, academics, universities, third sector,
industry…

4. NIHR CLAHRC
Northwest London
Collaboration for Leadership in Applied Health Research and Care
Northwest London
The National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (NIHR CLAHRC) Northwest London
is hosted by Chelsea and Westminster Hospital NHS Foundation Trust and academically led by Imperial College London, in partnership with North West London
Measurement for Improvement

5. NIHR CLAHRC
Northwest London
Solberg et al 1997
Three reasons for measurement

6. NIHR CLAHRC
Northwest London
Measurement for Improvement
A few simple and specific measures,
linked to your objectives and aims, and
available in real time
to demonstrate whether changes are
making improvements.
NHS Institute for Innovation and Improvement

7. NIHR CLAHRC
Northwest London
Measurement in, and of,
Improvement
• Are changes happening in the way health
care is delivered?
• Are these associated with improvements
for patients, carers & the public?
• Are these changes causally linked?
• How can we reproduce this improvement
elsewhere?

8. NIHR CLAHRC
Northwest London
The measurement cycle
1 Decide aim
2 Choose measures
3 Confirm collection
6 Take appropriate
action
5 Analyse & present
8 Repeat
steps
4-6
4 Collect data
7 Review measures
5 Analyse & present
Adapted from a slide by Mike Davidge

9. NIHR CLAHRC
Northwest London
What to measure?
Aim Concept Measure
Operational
Definitions
Data
Collection
Plan
Data
Collection
Analysis Action
R. Lloyd. Quality Health
Care: A Guide to
Developing and Using
Indicators.

10. NIHR CLAHRC
Northwest London
Aims
Measures
Ideas
(NOT Solutions)
How do we change?
Ideas -> Solutions
The Model for Improvement

11. NIHR CLAHRC
Northwest London
Enumerative vs Analytic Study
On Probability As a Basis For Action, W E Deming,
The American Statistician, Vol. 29 No. 4 1975, pp. 146-152
(and previously in ‘42 and ‘50)
“An enumerative study has for its aim an estimate
of the number of units of a frame that belong to a
specified class.
An analytic study has for its aim a basis for action
on the cause-system or the process, in order to
improve product of the future”

12. NIHR CLAHRC
Northwest London
Enumerative vs Analytic Study
Analytical studies: a framework for quality
improvement design and analysis, Lloyd P Provost
BMJ Qual Saf 2011; 20 (Suppl. 1) doi:10.1136/bmjqs.2011.051557
“Because of the temporal nature of improvement, the
theory and methods for analytical studies are a
critical component of the science of improvement.”
Analogy: pond vs river

13. NIHR CLAHRC
Northwest London
SPC: Origins
• Dr. Walter Shewhart: physicist & engineer,
Western Electric and Bell Laboratories,
1920s. Quality in manufacturing.
• Dr. W. Edwards Deming extended
Shewhart’s work, developing and
explaining applications (U.S. Then Japan
after WWII, worldwide in the 80s and 90s).
• Increasingly seen in healthcare – but
applied with varying degrees of rigor and
success.

14. NIHR CLAHRC
Northwest London
Understanding variation
Everything we can measure will vary
Statistical Process Control separates out Special Cause
from Common cause (routine variation)
– Common: inherent to the process, none dominant
– Special: due to a specific reason, dominant
Distinguishing between the two is crucial for good decision
making
SPC provides tools to make this distinction
– run and control charts

15. NIHR CLAHRC
Northwest London
Collaboration for Leadership in Applied Health Research and Care
Northwest London
The National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (NIHR CLAHRC) Northwest London
is hosted by Chelsea and Westminster Hospital NHS Foundation Trust and academically led by Imperial College London, in partnership with North West London
An Example QI Initiative

16. NIHR CLAHRC
Northwest London
An Example QI Initiative

17. NIHR CLAHRC
Northwest London
Physical health and severe and
enduring mental illness
“Adults with serious mental illness represent the
greatest and least recognized health disparity...
reflected in a 13- to 30-year reduced life expectancy.
The primary cause... is cardiovascular disease
associated with disproportionately high rates of
obesity and tobacco use... the greatest current
challenge to providing appropriate mental health
services is not a knowledge gap, but an
implementation gap.” Bartels, Am J Psych, 2015

18. NIHR CLAHRC
Northwest London
Don’t just screen- intervene

19. NIHR CLAHRC
Northwest London
Linking project aims and activities…

20. NIHR CLAHRC
Northwest London
Untangling systems and processes…

21. NIHR CLAHRC
Northwest London
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Run Chart - Percentage of Admissions with BMI Recorded
A MENTAL HEALTH WARD IN NORTHWEST LONDON
Percentage with BMI recorded
Baseline Median (21%)
Baseline Median - Extended
New process median (61%)
Baseline audit
(247 admissions)
Intervention period
(311 admissions)

22. NIHR CLAHRC
Northwest London
Collaboration for Leadership in Applied Health Research and Care
Northwest London
The National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (NIHR CLAHRC) Northwest London
is hosted by Chelsea and Westminster Hospital NHS Foundation Trust and academically led by Imperial College London, in partnership with North West London
Additional Examples

23. NIHR CLAHRC
Northwest London
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
02-Jan 02-Feb 02-Mar 02-Apr 02-May 02-Jun 02-Jul 02-Aug 02-Sep 02-Oct 02-Nov 02-Dec
COPD Care Bundle - Hospital X
Overall Compliance
Percentage Compliance
Average
Lower Control Limit

24. NIHR CLAHRC
Northwest London
0%
5%
10%
15%
20%
25%
30%
Percentage ED
attendances admitted
Seasonalised
Average
Seasonalised Lower
Process Limit
Seasonalised Upper
Process Limit
Monthly % ED Attendances Admitted; England 2011-2014
0
0.1
0.2
0.3
0.4
0.5
0.9
0.92
0.94
0.96
0.98
1
1.02
1.04
1.06
1 2 3 4 5 6 7 8 9 101112
RangeChart
AverageChart
Month
Average and Range Chart for Monthly
Seasonal Factors
Average
Grand Average
Lower Average
Limit
Upper Average
Limit

25. NIHR CLAHRC
Northwest London
Collaboration for Leadership in Applied Health Research and Care
Northwest London
The National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (NIHR CLAHRC) Northwest London
is hosted by Chelsea and Westminster Hospital NHS Foundation Trust and academically led by Imperial College London, in partnership with North West London
What next?

26. NIHR CLAHRC
Northwest London

27. NIHR CLAHRC
Northwest London
We conclude that
although informal theory is
always at work in
improvement, practitioners
are often not aware of it or
do not make it explicit.
The germane issue for
improvement practitioners,
therefore, is not whether
they use theory but
whether they make explicit
the particular theory or
theories, informal and
formal, they actually use.

28. NIHR CLAHRC
Northwest London
Safer Clinical Systems
Evaluation findings (http://www.health.org.uk/publication/safer-clinical-systems-
evaluation-findings)

29. NIHR CLAHRC
Northwest London
Collaboration for Leadership in Applied Health Research and Care
Northwest London
The National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (NIHR CLAHRC) Northwest London
is hosted by Chelsea and Westminster Hospital NHS Foundation Trust and academically led by Imperial College London, in partnership with North West London
Plan
Explore
Design
Test and Scale
Up
Spread
MeasurementPlanning
SupportTool
Measurementfor
ImprovementSupportTool
SPCPublication
Standards
WISH
MeasureDefinition
Process
MeasurementTraining
(inc.QI4U)
Technicalsupport

30. NIHR CLAHRC
Northwest London
Measurement for Improvement -
Summary
• Goal is improvement – analytic study
• Methodology required is different – SPC
• Not an excuse to sacrifice rigour!
• QI methods support good practice – if used
correctly
• Further research required – how to reliably
achieve good measurement
• More support and guidance is needed for QI
teams