IAC 2024 - IA Fast Track to Search Focused AI Solutions
Open data for development
1. United Nations Development Programme
Open Data
Marc Lepage
Knowledge & Innovation advisor,
UNDP regional center for Africa
2. • What is open data?
• Benefits of open data
• Open data for …
–Government
–Aid efficiency
–Development
–Crisis information management
• Challenges
• What next?
2
3. open data is “data that can be freely used, reused and
redistributed by anyone – subject only, at most, to the
requirement to attribute and sharealike”.
3
4. Open
• data freely available
Accessible
• data easy to use and re-use
Searchable
• data easy to find
4
5. • 1 star: Available on the web (whatever format) but
with an open licence, to be Open Data
• 2 stars: Available as machine-readable structured
data (e.g. excel instead of image scan of a table)
• 3 stars: as (2) plus non-proprietary format (e.g. CSV
instead of excel)
• 4 stars: All the above plus, Use open standards from
W3C (RDF and SPARQL) to identify things, so that
people can point at your stuff
• 5 stars: All the above, plus: Link your data to other
people‟s data to provide context
5
6. • Part of a broader move, including new Access to
Information Policy
• Builds on global Open Data initiatives
• Aim is to stimulate use of development data to help solve
development problems:
–economic benefits (growth and job
creation),
–improved public services and
–more transparent and accountable
government
6
7. Before…
After…
Subscriptions
Free!
Custom query tool
New website, downloads, API
Restrictions on use Minimal restrictions on use
English
+Spanish, French, Arabic,
Chinese
Not searchable
Searchable
Scattered datasets
Data catalog
7
8. • Users are encouraged to use the data.
–extract, download, and make copies
–Share with third parties.
• Attribution
• No Endorsement
• No Association
• No Warranties
8
10. There are several flavors of open data,
for instance open government data
helps to make data held by
governments accessible to both
applications and end-users.
10
11. Major examples of open
government data include transport
data (which has given cities like
New York applications to show
fastest routes on any day etc) and
finance data (giving citizens
visibility into how their money is
spent).
11
12. 1. Government should do the least possible and act as a
Facilitator – Build a platform and let others add to it
2. Engage entrepreneurs in developing applications using
government data through workshops and forums
3. Develop Application Programme Interface (API) to support
web developers for your data
4. When more people are working on the data, innovation
happens
12
13. 1. Citizens are becoming data producers as well as data
consumers.
2. Citizens are changing the political landscape by being more
involved in the decision making process
3. Citizens are engaging other citizens to act through social
networking tools
13
16. • A client government wants to download and use HDR
data into internal systems
–In real time
–Inexpensively
–Without significant human overheads
• An NGO in Botswana wants to follow UNDP‟s project
portfolio in democratic governance
• A researcher wants to combine UNDP‟s HRD data with
data from a country‟s planning commission to analyze
trends
16
17. • A blogger wants to substantiate the opinion in her blog
with data based charts (that update in real time) from the
UNDP‟s project database
• A UNDP staff member answers questions about UNDPs
commitment to transparency
• An independent application developer in Georgia wants
to create a mobile application that tracks the flow of donor
money to monitor election and help local civil society
access the grants (case study here)
17
37. Lack of overall strategy
The “Top Down” approach / culture
Cultural barriers
Legislation gap
Context gap
37
38. • Suggested approach: pilot under 3 criteria
–Low cost (under $1000/month)
–Fast
–Low effort
• 2 options:
a) custom development, piggybacking on the
HDR‟s API
b) open data platform
38
39. • Start with one data set we feel comfortable with
• Test with stakeholders
• Confirm features/functions/scalability
• Learn from the experience
• Identify more potential data sets
• Continue evaluating other options
(custom/platform)
• Get ready for a larger launch
39
40. • The pilot will only include one data set
• (If needed) the pilot will include some basic design work required to
make the pilot site attractive and user-friendly
• The pilot can be live within a week of final approval to proceed
• The pilot should ideally run for no more than 2-3 months
• One of the goals of the pilot will be to understand what additional
features/tools must be bundled with the service to maximize the value
of the service to clients
• The pilot will also help the team understand the technical parameters
required to fully launch the service
• The pilot will also consider what communication/community
approaches will be most effective during the larger launch
40
41. • OUTCOMES
– Establish the governance processes and structures to
support the larger launch
– Resolve security, legal, data confidentiality, and other
issues before the larger launch
– Gain client/other stakeholder buy-in for the service
– Establish the full bouquet of services that should be
bundled with the final product
– Work through a variety of „what‟s in it for me‟ client
scenarios
– Select a platform for the final launch
41
43. ISSUES
STEPS
Who is going to take the lead?
KIC group to assist & offer for pilot
What data can be included in this service (immediately, and in the long term)
KIC group to consult internally
What impact will the initiative have on our client relationships? Are there any
pre-launch steps we need to take
This will depend on the data sets we start with
Can all open data services at UNDP be consolidated through the API that is
being developed for HDR?
HRD team and KIC group to consult
What security clearances are required to launch this service
Depends on the data set identified. Could be
none. Double check with legal dept
Who should manage the service? Which parts of UNDP should be involved
KIC group
What data management steps do we need to take to support the data
requirements of the initiative (catalog, metadata, quality, more)
Pilot will clarify
Are there disclosure/copyright issues we should take into account?
Depending on data sets. Consult with legal
What about the quality of our data?
Pilot will shed light into the possibility of
feedback channel to get quality control from
users
Is there a resource constraint in terms of dealing with queries once the data is
released?
Use the pilot to get insights
43
44. This is just the beginning…….
Availability of
data
Accessibility
Use of data
Engage /
change
44
46. • http://www.od4d.org
• http://openforchange.info
• http://www.slideshare.net/yajitha
• Open data group on Teamworks
• Mark Cardwell, open data evangelist at UNDP
• Giuilio Quaggiotto, all things innovation at UNDP
46
Notas do Editor
UN 2.0 – A transparent, open organization
Data reuse means using data for purposes other than those it was collected for, including combining the data with other datasets. Redistribution means making the data available to other people. The data should be available in a machine-readable format, and available either free or at minimal cost (to cover the costs of reproduction etc if needed). Attribute means to acknowledge the original provider of the data, and sharealike means that any copy or adaptation of the data must be provided under the same or similar license as the original.
Open data is also released in different formats, some of which are more difficult to use than others (believe me, scraping pdfs is *hard*). Tim Berners-Lee created the 5-star system for open data, to encourage data providers to release easier-to-use data:
Here is USAID’s presentation of funding flows, not as current or as well-presented, but quite detailed.
Our key partners are pushing hard for open reporting of financial information. Here is the UK’s aid agency showing where the money is going. Notice the reporting is delayed weeks, not months or years.
Here is a visualization of UK development assistance. Notice the U.N. agencies.
Notice where UNDP is on the scale 55m of a 5.6 billion spend.
The MDTF has made a great step in showing funding flows….
Call on donors to fund, CSO to do it. Virtuous circle better access, better data…….pretty soon we will have 80% of ODA