1. The science of delivery
Use of administrative data in Health Result
Innovation Trust Fund (HRITF) portfolio
Ha Thi Hong Nguyen | Cape Town, 2014

2. What are administrative data?
• Data on payment to facilities based on verified
performance
– Can compare with reported data
– Typically only available in contracted facilities
• Data reported in the HMIS system
– Can be available for control facilities
• Individual patient records
– Can look at health outcomes and processes of care
– Rarely available in many HRITF countries
• HRITF mostly works with the first 2 categories, and
generally calls them operation data

3. The HRITF OP data portfolio
Country Start date Program areas Catchment population
Benin Mar 2012 8 districts 2.2 million (22%)
Burkina Faso* Dec 2011 3 districts 813 thousand (5%)
Burundi Mar 2010 Countrywide 9.8 million (100%)
Cameroon* Littoral: Apr 2011
3 other: Jul 2012
4 regions 2.8 million (13%)
Kenya* Dec 2011 1 sub-county 200 thousand (0.5%)
Nigeria* Dec 2011 3 LGAs 416 thousand (0.2%)
Zambia Apr 2012 11 districts 1.5 million (11%)
Zimbabwe Mar 2012 18 districts 4.2 million (30%)
Afghanistan April 2009 11 provinces 9.1 million (33%)
Laos Mar 2013 5 provinces 2.2 million (33%)
Sierra Leone Oct 2010 Countrywide 5.9 million (100%)
Total population is for 2012 (WDI)
Note several programs have expanded but OP data are not yet available
3
*Not include recently scaling up areas

4. Why operational data?
• To monitor programs’ progress as basis for further inquiry
and mid-course corrections
– Identifying high and low performing indicators
– Monitoring where money is spent
– Detecting outliers
– Comparing with control areas and watching for
unintended consequences
– Improving implementation design
• To promote transparency and hold providers accountable
for results
• To evaluate the impact of the program

5. Monitoring program progress to facilitate
further inquiries
80
70
60
50
40
30
20
10
0
4 1 2 3 4 1 2 3 4 1
2011 2012 2013 2014
Afghanistan
Benin
Nigeria
Zambia
Zimbabwe
%
Estimated coverage of institutional/SBA deliveries

6. Identifying high and low performance
Zambia: change between Q2 1012 and Q1 2014 in QOC components
Curative Care
100
80
60
40
20
0
ANC
FP
EPI
Delivery
Room
HIV
Community
Participation
HMIS
Management
Supply
General
Management
Q2 2012
Curative
Care
100
80
60
40
20
0
ANC
FP
EPI
Delivery
Room
HIV
HMIS
Supply
Manageme
nt
General
Manageme
nt
Community
Participatio
n
Q1 2014

7. Monitoring where money is spent on
Share of RBF payment for service delivery that went to health center and
lower level
0 20 40 60 80 100
Cameroon
Burundi
Zimbabwe
Benin
Burkina…
Nigeria
Zambia
Kenya
%

8. Monitoring where money is spent on
OP >5
11%
OP <=5
15%
Inst.
Deliver
ies
17%
Others
57%
Burundi
Zambia
Cameroon
Zimbabwe
OP
contact
6%
Inst.
Deliveri
es
35%
Others
19%
FP
40%
OP
contact
35%
Inst.
Deliverie
s
15%
Others
29%
FP
21%
OPC
21%
Hosp.
days
15%
VCT
12%
Others
52%
Figures reported are averages of all quarters to date
8
Three services absorbing largest share of payment

9. Detecting outliers
1600
1280
960
640
320
0
Nigeria: Performance on institutional deliveries by LGA
4 1 2 3 4 1 2 3 4 1
2011 2012 2013 2014
Adamawa
Ondo
Nasarawa

10. Assessing relative progress and watching out for
negative spillover
Afghanistan: number of SBA deliveries in
treatment and control facilities
Zimbabwe: Diarrhea cases among age 5+
(non-incentivized RBF indicator)
25000
20000
15000
10000
5000
0
Control
Treatment
2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1
2010 2011 2012 2013 2014
33
27
31
32
50
45
40
35
30
25
20
15
10
5
0
Mar-11
May-11
Jul-11
Sep-11
Nov-11
Jan-12
Mar-12
May-12
Jul-12
Sep-12
Nov-12
Jan-13
Mar-13
May-13
per 10,000 population
HMIS RBF
HMIS Non-RBF

11. Improving implementation design
Zimbabwe: switching to risk based evaluation based on comparing reported
and verified data
15
10
5
0
-5
-10
-15
Difference Between Declared and Verified 6 Month Totals
Within 5% Difference
HF1 HF2 HF3 HF4 HF5 HF6
Green Category:
• Verified on a quarterly basis
Amber Category
• Verified bi-monthly -
randomly selected 2 months
Red Category
• Verified on a monthly basis
• Also incorporates new
facilities
Difference above 5% but
below or equal to 10%
Difference above 10%
• Model based on three risk levels
• Comparison between declared and
verified values for 6-month totals

12. Promoting transparency and accountability
Burundi
Benin
Nigeria

13. Issues in working with operational data
• Quality of data
• Availability of data outside program (catchment
population)
• Capacity to design and manage a database
• Capacity to analyze data
• Standardized methods and assumption to calculate
coverage
• Practice of sharing data and using results for decision
making
• Integration with country HMIS