1.
Effects of Payment for Performance on
Knowledge, Practice and the Know-Do Gap
Evidence from Pwani,
Tanzania
Josephine Borghi
24th November
RBF – a health systems perspective. White Sands Hotel, Dar es Salaam.

2.
Rationale
• P4P is expected to improve quality and service coverage
through changed health worker behaviour
• Improvements in health worker knowledge and practice is
necessary for optimal health gain
• Knowledge may increase through substitution of health
workers; greater investment in training
• Practice (or application of knowledge) is likely to increase
through a desire to meet targets, and improved resource
availability
• In Rwanda found limited effect on knowledge and
improvements in practice, especially among those with higher
knowledge levels.
• Aim: examine P4P effects on knowledge, practice and
the gap in Tanzania using data from our evaluation in
Pwani

3.
P4P in Tanzania
Aim: A pilot introduced in 2011 focusing on MCH
service coverage to inform a national programme
Location: Pwani region of Tanzania
Implementers: MOHSW and CHAI
Funder: Government of Norway

4.
Scheme Design
Facility level targets:
– ANC: IPT2; % HIV+ women on ART
– Institutional delivery rate
– % of newborns with OPV0 in first 2 weeks
– % infants with Penta 3; measles vaccine
– % of PNC visit w/n 7 days
– CYP
– HMIS reports correctly filled and submitted on
time + use of partograms
District – regional level targets:
– % of maternal/perinatal deaths audited on time
– % of facilities with stock outs

5.
Study Design
• Design: Controlled before and after study design
– 7 intervention districts
– 4 neighbouring control districts
– Comparable poverty, literacy, rate of
institutional deliveries, IMR, pop. per health
facility, no. of children < 1 yr
• Timing:
-Baseline in January-February 2012
-Endline in March-April 2013 (13 months)

6.
7 P4P districts 4 districts with no P4P
150 health facilities, 75 in each
arm incl.
6 hospitals
16 health centres
53 dispensaries
1 facility survey at
each facility
20 interviews with
women who delivered
in past 12 months, from
the catchment area of
each facility
Only include
facilities
eligible for
first cycle
payment
1-2 health
workers at each
facility

7.
Measurement:
Knowledge
• Used a clinical “vignette”: a hypothetical patient case, in this case, a
woman attending her first antenatal visit.
• Derived from the World Bank Impact Evaluation Toolkit
• Presented to health workers who regularly provide ANC
• 45 items from the antenatal clinical guidelines were covered in the
vignettes, with items corresponding to four dimensions:
– patient medical history
– physical examinations
– laboratory investigations
– drugs prescriptions.
• Measure scores for each dimension and in total: number of items
mentioned by the total number of items.

8.
Measurement: Practice
• Procedures performed by the provider on patients (adherence to
protocol).
• Household interviews with women attending ANC during their
current or last pregnancy living within the catchment area of
facilities where health workers were surveyed
• A total of 18 items regarding ANC services, 11 match the 45 items
in the health workers survey
• Dimensions of care:
– physical examinations
– laboratory investigations
– drugs prescription.
• We constructed an additional dimension relating to client
counselling and educational services.

9.
Measurement: Gap
• The knowledge–practice gap measures provider efficiency to
translate knowledge into actual ANC practice.
• Defined as the difference between the knowledge and practice share
of clinical guidelines for an ANC visit.
• Women linked to a given facility by its catchment area are matched
to the health workers’ responses for that same facility.
• Take average value for knowledge measure at facility level where
more than one health worker was surveyed per facility.
• Measured for 11 items across:
– physical examinations
– laboratory investigations
– drugs prescribed.

10.
Analysis
• Used a difference-in-difference identification strategy:
• 𝐾ℎ𝑗𝑡 = 𝛽0 + 𝛽1(𝑃4𝑃𝑗 × 𝛿𝑡) + 𝛽2 𝛿𝑡 + 𝛽3 𝑍ℎ𝑗𝑡 + 𝛾𝑗 + 𝜀ℎ𝑗𝑡
• 𝑌𝑖𝑗𝑡 = 𝛽0 + 𝛽1(𝑃4𝑃𝑗 × 𝛿𝑡) + 𝛽2 𝛿𝑡 + 𝛽3 𝑍𝑗𝑡 + 𝛽4 𝑋𝑖𝑗𝑡 + 𝛾𝑗 + 𝜀𝑖𝑗𝑡
• 𝐾ℎ𝑗𝑡 knowlegde share of ANC guidelines by provider h at health facility j in period t
• 𝑌𝑖𝑗𝑡 is the practice share or the gap
• 𝑃4𝑃𝑗 is a dummy variable taking the value of 0 for comparison facilities and 1 for
intervention facilities;
• 𝛿𝑡 is a year fixed effects dummy taking the value of 0 at baseline and 1 at endline;
• 𝑍ℎ𝑗𝑡 health worker-level characteristics expected to drive programme outcomes
• 𝑋𝑖𝑗𝑡 are household level characteristics expected to drive programme outcomes
• 𝛾𝑗 is a facility fixed effects to control for facility-level time invariant characteristics; and
𝜀ℎ𝑗𝑡 is a random error term
• Assumption: pre-trends in outcomes are parallel

11.
Impact on ANC Knowledge
Variables Baseline Impact
P4P Control Diff % effect of
P4P
Knowledge shares for each
dimension
Medical history taking (% of
items known) 26 items
20.2 34.1 -13.9*** 12.5***
Physical examinations (%) 10
items
25.5 42.7 -17.2*** 11.9***
Lab investigations (%) 7 items 26.7 48.4 -21.6*** 18.5***
Drug prescriptions (%) 2 items 63.8 87.1 -23.3*** 16.2***
Total items known (%) 45 items 24.3 40.6 -16.3*** 13.4***
Total items known – gap (%) 11
items
42.0 64.6 -22.7***
17.6***

12.
Impact on ANC Practice
Variables Baseline Impact
P4P Control Diff % effect of
P4P
Practice shares for each
dimension
Client Counselling (% of items
done) 7 items
78.6 71.8 6.8*** -3.2*
Physical examinations (%) 6 items 89.4 87.7 1.7 0.2
Lab investigations (%) 3 items 86.8 83.2 3.6* 2.2
Drug prescriptions (%) 2 items 71.4 73.6 -2.2 7.1**
Total items done (%) 18 items 82.6 79.1 3.5*** 0.0
Total items done – gap (%) 11
items
85.3 83.9 1.5
2.0*

13.
Impact on ANC Know-Do Gap
Variables Baseline Impact
P4P Control Diff % effect of
P4P
Gap shares for each dimension
Physical examinations (%) 6 items -54.4 -34.7 -19.7*** 9.2*
Lab investigations (%) 3 items -43.1 -6 -28.0*** 18.6***
Drug prescriptions (%) 2 items -2.5 14.9 -17.5*** 2.8
Total items (%) 11 items -39.1 -17.9 -21.2*** 10.1*

14.
Conclusions
• P4P significantly improved health worker knowledge across all
dimensions
• Plausible?
• District managers shifted efficient workers to help those struggling to meet
targets.
• Opportunities to upgrade skills with training increased as a result of P4P from
health worker survey.
• Knowledge indication of ‘intended behaviour’ – may be more responsive to P4P
in the short term than practice
• P4P improves practice in relation to incentivised components of
care: drug prescriptions but no evidence of other
improvements in adherence to care guidelines
• As knowledge increases and practice generally doesn’t –
the inefficiency gap increases

15.
Limitations
• Imbalance in baseline knowledge between intervention and control
• Unable to assess whether trends in knowledge and practice were
parallel prior to P4P; trends in ANC coverage were
• Gap analysis:
– Tools not originally intended to pursue this gap analysis – and only a limited
number of items could be compared in this way
• Consider the one-two health workers interviewed as representative
of ‘practice’ at a given facility which may not be the case
• Assume households went to their nearest facility for ANC
• Concern that practice found to exceed knowledge at baseline
(negative gap)

16.
Acknowledgements
• Josephine Borghi – LSHTM
• Paola Vargas – LSHTM/OPM
• Peter Binyaruka - LSHTM
• Powell-Jackson T - LSHTM
• Patouillard E - LSHTM
• Torsvik G – CMI
• Mayumana I – IHI
• Masuma Mamdani - IHI
• Lange S - CMI
• Maestad O - CMI