This was presented by Rupert Simons from Publish What You Fund at the Impacts of Civic Technology Conference (TICTeC2016) in Barcelona on 27th April. You can find out more information about the conference here: https://www.mysociety.org/research/tictec-2016/
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
Open data for aid and development
1. Open data for aid and
development
Rupert Simons
April 27, 2016
2. Source: World Development Report 2016
We’ve all heard of big data
Cellphone subscriptions
worldwide
billion
0.7
7
2000 2015
Gigabytes of data produced
every day
billion
2.5
2000 2015
<0.1
3. But data is often poor or non-existent in countries
where you need it most
Managing the Ebola crisis
• Where are the patients?
• Where are the
ambulances?
• Have the health workers
been paid?
Source: Nature, FT
Estimating the size of the
economy
• Did Nigeria’s GDP really double
in two years?
• ‘Middle income’ countries pay
more to borrow from the World
Bank, but less in the market
0
1000
2000
3000
4000
Nigeria GDP, USD per capita
4. Publish What You Fund reviewed the quality of
data produced by major aid donors
10
25
65
Activity level data
(28 indicators)
Organisation level
data (8 indicators)
Commitment to
transparency (3
indicators)
We worked with
people from
46agencies in
22 countries
and 36
independent
reviewers and CSOs
100
To assess 39 indicators of
data on aid and
development
Maximum possible
score
Over a data collection
period of 14 weeks
5. What we found: Aid is getting more
transparent
4 7
105
8
8
16
16
16
16
15
726
22
5
2013 Index 2014 Index 2016 Index
Very Poor
Poor
Fair
Good
Very Good
Source: Aid Transparency Index 2013-16
Performance of agencies in the Index
Total 67 68 46
Ten donors met the Busan
commitment in full:
• UNDP
• U.S. – MCC
• UNICEF
• UK – DFID
• The Global Fund
• World Bank – IDA
• IADB
• AsDB
• Sweden
• AfDB
6. 300+
Nearly 400 organisations are now publishing to
IATI
Number of IATI publishers by type
2011-15
Source: IATI annual report 2015
7. The norm is gradually shifting from annual
reporting to quarterly, monthly or real time
Annual reporting
• Companies House/ Charity Commission returns
• OECD-DAC Credit Reporting Statistics (for official donors)
• Annual report on website (usually in PDF)
Regular updating
• Monthly or quarterly publication
• Machine-readable and web formats
• Edit for confidentiality
Real-time data sharing
• Uploaded automatically
• Daily decision-making
• Limited editing
8. Some governments are using the data
NGOs are using it too,
for example:
Source: IATI, Development Gateway
IATI import system
in use
IATI import system
under development
IATI partner country
9. Most people outside government are not
Source: d-portal.org, 2016 Aid Transparency Index
Partly because it’s not
easy to use
Partly because they don’t
know it’s there
Partly because the data
is of poor quality
“Even stakeholders that
regularly search for aid
information, like civil
society watchdog
organizations and
parliamentarians, were not
aware of the available data
sources.”
USAID aid transparency
pilot assessment, July
201513% of activities in
Ghana had a location
specified
10. Meanwhile, civic registration and
administrative data lags far behind
Source: Development Initiatives “Data Revolution in Africa”
82
45
33 29
22
13
9
40
29
13
18
9
9 15
38
58 60
78
Census Household
survey
Poverty
census
Agricultural
census
Birth
statistics
Death
statistics
Recent (last 10
years for census,
last 3-5 years for
survey)
Out of date
Unusable or
unavailable
Quality and availability of data in African countries,
percentage of countries
11. Estimates for closing data gaps range widely,
with $100-200 million per year as the median
Source: Brookings; CGD; SDSN
17-66
Cost of building and maintaining statistical systems to
monitor the Sustainable Development Goals
USD million per year
100-
200
1,100
900- 1,000
>10,000
New money
Existing aid
Own resources
Not specified
12. So what should we do about it?
• A perspective from the INGO Accountability Charter
• What do you think?
• Publish What You Fund perspective
• Data quality
• Awareness
• Tools
Editor's Notes
One of the key objectives of the aid transparency agenda is to provide aid information in a way that can be used by partner country governments, particularly so it can be included in their own budget cycle.
Requires intelligent partnership between governments. Responsibilities lie on both donors and recipients – donors need to give their information in a timely and standardised fashion. Recipients need to make sure that their aid management processes are well linked to their actual budget processes.
In 2007, this took eight months of manual data collection, and another six months to visualise aid flows and the domestic budget in Uganda. In time, IATI will allow us to do this at the click of a button.
Some surprising results: only one half of non-budget support aid was being reported to the govt (and therefore reflected in its budget).
Well over 50% of the health budget was coming from aid project funding.
This should have informed aid programming discussions at sector working groups, but politically sensitive (and at the time, not open data).
This is a domestic process too. Takes international commitments to the domestic agenda.
Since organisations first started publishing in 2011, over 300 have published data to the IATI Standard.
These include International NGOs, bilateral and multi-lateral donors, private sector, foundations and research organisations. NGOs make up a considerable number of publishers, in part due to IATI reporting being made a funding requirement by some donors.
UK, Germany, U.S., WB, AFDB, Sweden, Netherlands, Australia, Canada to name a few.
UK - Department for Environment, Food and Rural Affairs
UK - Department for International Development (DFID)
UK - Department for Work and Pensions
UK - Department of Energy and Climate Change
UK - Department of Health Government United Kingdom
UK - Foreign & Commonwealth Office
UK – Home Office
This means that the standard can accommodate a number of users and publishers. Diversity of those publishing to IATI makes this achievable.
What does it mean to publish to IATI. Not a hugely technical effort but rather extracting data from the source and putting into the common format.
This is a domestic process too. Takes international commitments to the domestic agenda.
One of the key objectives of the aid transparency agenda is to provide aid information in a way that can be used by partner country governments, particularly so it can be included in their own budget cycle.
Requires intelligent partnership between governments. Responsibilities lie on both donors and recipients – donors need to give their information in a timely and standardised fashion. Recipients need to make sure that their aid management processes are well linked to their actual budget processes.
In 2007, this took eight months of manual data collection, and another six months to visualise aid flows and the domestic budget in Uganda. In time, IATI will allow us to do this at the click of a button.
Some surprising results: only one half of non-budget support aid was being reported to the govt (and therefore reflected in its budget).
Well over 50% of the health budget was coming from aid project funding.
This should have informed aid programming discussions at sector working groups, but politically sensitive (and at the time, not open data).
One of the key objectives of the aid transparency agenda is to provide aid information in a way that can be used by partner country governments, particularly so it can be included in their own budget cycle.
Requires intelligent partnership between governments. Responsibilities lie on both donors and recipients – donors need to give their information in a timely and standardised fashion. Recipients need to make sure that their aid management processes are well linked to their actual budget processes.
In 2007, this took eight months of manual data collection, and another six months to visualise aid flows and the domestic budget in Uganda. In time, IATI will allow us to do this at the click of a button.
Some surprising results: only one half of non-budget support aid was being reported to the govt (and therefore reflected in its budget).
Well over 50% of the health budget was coming from aid project funding.
This should have informed aid programming discussions at sector working groups, but politically sensitive (and at the time, not open data).
One of the key objectives of the aid transparency agenda is to provide aid information in a way that can be used by partner country governments, particularly so it can be included in their own budget cycle.
Requires intelligent partnership between governments. Responsibilities lie on both donors and recipients – donors need to give their information in a timely and standardised fashion. Recipients need to make sure that their aid management processes are well linked to their actual budget processes.
In 2007, this took eight months of manual data collection, and another six months to visualise aid flows and the domestic budget in Uganda. In time, IATI will allow us to do this at the click of a button.
Some surprising results: only one half of non-budget support aid was being reported to the govt (and therefore reflected in its budget).
Well over 50% of the health budget was coming from aid project funding.
This should have informed aid programming discussions at sector working groups, but politically sensitive (and at the time, not open data).
One of the key objectives of the aid transparency agenda is to provide aid information in a way that can be used by partner country governments, particularly so it can be included in their own budget cycle.
Requires intelligent partnership between governments. Responsibilities lie on both donors and recipients – donors need to give their information in a timely and standardised fashion. Recipients need to make sure that their aid management processes are well linked to their actual budget processes.
In 2007, this took eight months of manual data collection, and another six months to visualise aid flows and the domestic budget in Uganda. In time, IATI will allow us to do this at the click of a button.
Some surprising results: only one half of non-budget support aid was being reported to the govt (and therefore reflected in its budget).
Well over 50% of the health budget was coming from aid project funding.
This should have informed aid programming discussions at sector working groups, but politically sensitive (and at the time, not open data).