Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
The University of Waterloo Open Data API
1. THE UNIVERSITY OF WATERLOO OPEN
DATA API
Colin Bell – EnterpriseArchitect, IST
KartikTalwar – Client Services, IST
NathanVexler – Client Services, IST
2. AGENDA
1. What is Open Data?
2. Making Open Data Work
3. TheValue of Open Data
4. The API approach
5. Current State of Open Data at University of Waterloo
6. Open Data by Example
7. How do we get to where we need to be?
8. Tell us your stories!
30. WHY API?
• Do the ETL work once, do it well.
• Reduce duplication of efforts.
• Cut down on costs because it comes in a ‘usable’ form.
• Referential links are moved around as URIs. (context)
• Access can be monitored and controlled.
32. Fall 2009
• 1st Presentation at
STAC about
Portal/Open Data
March 2010
• Presentation at
UCIST/CTSC
Fall 2010
• JeffVerkoeyen’s
uwdata.ca becomes
official
Aug 2011
• KartikTalwar
creates
api.youwaterloo.ca
Feb 2012
• api.uwaterloo.ca
becomes official
Open Data @ uwaterlooTimeline
33. 2012
• Nathan Joins IST
full time
2013
• V2 launched
2014
• Mandate to bring
api.uwaterloo.ca
deeper into the
enterprise
Open Data @ uwaterlooTimeline
Open Data and the 5 Star approach (Nathan Vexler)
Why we have taken the API approach (Colin Bell)
How Open Data Is Useful (examples)
Stuff students have built with Open Data (Kartik Talwar)
Stuff IST has built with Open Data – Food Services (Kartik Talwar)
Current State of Open Data at uWaterloo within context at IST/City/Region and nearby/Province/Canada/World (Nathan Vexler)
History (slide with timeline)
Status Quo
Future
Tell us your stories!
So again… very important… Open Data does NOT include Personally Identifiable Data
It’s important that we safeguard privacy. But we can do it in a really intelligent way.
We definitely want to avoid anything that Is Not Open Data. In this case… on the venn diagram, <this region> is Personally Identifiable Data. Completely off limits.
But… how about <this region over here>: personal data that is de-identified? Let’s do a thought experiment.
Take for example de-identified performance data for 1st year courses. With some minor effort, anyone could create algorithms that pattern match current students with patterns from archival data as a means to predict their performance. With such technology we can engineer early-stage interventions and help students before its too late. Such efforts would dovetail strongly with SSO’s desire
It’s important that we safeguard privacy. But we can do it in a really intelligent way.
We definitely want to avoid anything that Is Not Open Data. In this case… on the venn diagram, <this region> is Personally Identifiable Data. Completely off limits.
But… how about <this region over here>: personal data that is de-identified? Let’s do a thought experiment.
Take for example de-identified performance data for 1st year courses. With some minor effort, anyone could create algorithms that pattern match current students with patterns from archival data as a means to predict their performance. With such technology we can engineer early-stage interventions and help students before its too late. Such efforts would dovetail strongly with SSO’s desire
<todo>
Links to http://bit.ly/ipc_pbd and http://bit.ly/ipc_abd
Routine disclosure
Open Data
Access By Design
Open Data comes in a spectrum
Syllabus Data… Office Hours
Open Data is about releasing human-targeted information in ways that are increasingly easier and more efficient to manipulate by 3rd parties.
In your hands, you have a plan on how we will get there. It focuses on two elements.
Community: co-design with stakeholders
In your hands, you have a plan on how we will get there. It focuses on two elements.
Community: co-design with stakeholders
API restructuring so that we’re in line with widely-adopted industry-standards
Fixing of significant bugs that caused much of the community to shy away from key parts of the API. The community logged bugs and Kartik made it a mission to fix them!
The api restructuring is based on current standards in the industry… most notably the well regarded White House Web API Standards.
The key bugs caused developers to develop their own workarounds. Now with the latest release, we have renewed confidence from developers.
With the release of V2, we are continuing to actively mitigate nagging policy-type issues:
Specifically, regarding the risk of integrity…
We implemented an assortment of warnings and disclaimers to complement the existing disclaimers within the Open Data License.
The origin of the data and method of obtaining the data is explicitly documented on the doc site.
That the origin of the data and method of obtaining the data is properly documented.
That warnings and disclaimers are not sufficiently visible to developers.
Open data at Waterloo in context of all the other open data initaitves
How we’re using github
What does api look like
How it’s currently used
Log into portal
Open Data comes in a spectrum
Syllabus Data… Office Hours
Open Data is about releasing human-targeted information in ways that are increasingly easier and more efficient to manipulate by 3rd parties.
Scraping is vulnerable to go down; being poorly mapped
Scraping is vulnerable to go down; being poorly mapped
Together we enable the Waterloo community member who wants to make information available so that another can make that information available in a novel way.
Together we can build a community that values Privacy and Access by Design.
Together we can coral the gains that we’ve made together thus far into something sustainable, something that we can say was a lasting legacy of this generation of IST’s leadership.