2. Why assess health systems
performance?
“ Performance assessment allows policy-makers, health
providers and the population at large to see themselves
in terms of the social arrangements they have
constructed to improve health. It invites reflection on
the forces that shape performance and the actions that
can improve it.”
Gro Harlem Brundtland
Preface to WHR 2000
3. A wider range of aims in practice
Aims Examples
To improve policy and
planning
To monitor for accountability
To create generalizable
knowledge
To review Kenya’s HSP 94-
2010 so as to inform policy
framework through to 2030
South Africa DHB “tool to
monitor progress towards
strategic health goals”
Thailand, to analyze factors
that contributed to changes
in health system performance
during past 2 decades
4. Three Challenges to Evaluating
Health Systems Performance
How to “benchmark” good performance
Limited knowledge about appropriate values of health
indicators
Deciding what to measure:
Cannot measure everything.
When tracking effects of reforms, choice of indicators likely
to be influenced by logic models
But how predictable are effects?
How to move from information to action:-
For information to be actionable need to understand WHY
results are observed
Need to present evidence in a way that engages users, both
policy-makers and civil society
5.
6. Is first always best?
Comparison of South Africa districts on process indicators
Fezile Dabi Motheo Mopani
Rate Rank Rate Rank Rate Rank
Nurse
clinical
workload
44.2 1 28.9 15 12.5 52
ALOS 2.7 7 4.0 25 4.0 26
Bed
Utilization
rate
79.7 6 69.1 16 59.2 38
Clinical
supervision
rate
36.5 39 45.5 31 56.9 20
7.
8. For example:
Determining Appropriate ALOS
Factors Affecting ALOS
Case Mix
Socio-economic & demographic characteristics of the
population: age, nutritional status
Care provided post-discharge: care in the community,
long term care facilities
Key question: How does ALOS affect patient
outcomes?
9.
10. Frameworks for indicator selection Most health system performance assessments use
conceptual frameworks to guide indicator selection:
Donabedian: inputs, processes, outcomes/impacts
WHR 2000: functions (stewardship, creating resources,
financing, service delivery); outcomes (responsiveness,
fair financing, health)
CHeSS M&E logic model for health systems
performance
11.
12. Health systems as Complex
Adaptive Systems
May be difficult to predict impacts:-
Path dependence: health system interventions may have
different consequences in different contexts
Emergent behavior: spontaneous organization of actors
within the system (eg. organization of informal
providers) can lead to unpredictable effects
Phase transitions/tipping points
Need enhanced understanding of these dynamic
properties of health systems to ensure performance
assessment tracks the right thing
13.
14. Decision makers need to know the “Why?”
Without knowing “why?” difficult to take corrective
action:-
Was the intervention faithful to the original design or
was it adapted by local actors?
Complex interventions (as well as complex systems) –
what component of intervention caused the observed
effects?
Were effects influenced by context: if so what?
15. How to present the evidence?
How frequently do policy and decision makers need to see
evidence?
Are more complex indicators credible, do decision makers
understand them?
How does evidence get packaged? Simple dashboards (but
are they open to misinterpretation? Face-to-face
discussions – but how feasible to do?
Who gets to see it? Publicly accessible? Government
officials only?
How best to link to routine government processes eg.
health sector reviews?
16.
17. What can health systems research
contribute?
Benchmarking of indicators: what are optimal values of
different process indicators, how does this vary by context
(level of facility, resource availability etc)
Enhance understanding of complex adaptive systems:
modelling and empirical studies to better understand
properties
Focused research to understand “Why”: Was
intervention implemented as planned? What causal effects
occurred?
How best to package and communicate evidence from
health systems performance assessments?
18. Benefits to HSR
HSR challenged by lack of routine data on critical
issues:-
HRH availability
Private sector service provision
Health financing trends
Need to build up basic data collection systems (routine
facility surveys, household health expenditure surveys
etc)
Contribute to understanding of contexts, and
enhancing transferability of research findings.
Notas do Editor
Start with your audience….what are these things for? Role of HSR in assessments of health systems perrformance varies (I believe) according to purpose
Appropriate methods and approaches for Health Systems performance assessment will vary according to aim of the assessment: creation of generalizable knowledge is itself a research function – but even for the other two aims, I will argue that HSR can (and often should) play a role
DHB – district health barometer
Extracted from South African District Health Board
Green top tercile of rank, yellow middle, red lowest. This works well for indicators of output and outcome, but much less clear for process indicators – what is a good value for ALOS, how does this vary by context?