Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Wisconsin SCO Virtual Data Integration
1. VIRTUAL INTEGRATION OF
WISCONSIN PARCEL DATA
Howard Veregin Martin Goettl
Geography & Anthropology
Brenda Hemstead UW – Eau Claire
State Cartographer’s Office
UW-Madison
2. A RESEARCH PROJECT
Research focus: virtual data integration
Exploring feasibility of the approach
• Advantages
• Limitations
• Challenges
• Best practices
3. WHAT WE HAVE DONE
Combined existing county Web map services
into single multi-county online parcel map
Counties supplied REST service endpoints
• No physical data exchange
• No edgematching or
common data model
• Web map points to
published county dataset
(most current data)
4. WHAT ARE WEB SERVICES?
It depends on who you ask
Technology perspective
• Software and standards supporting
computer interaction over the Internet
User perspective
• Web apps offering data, product, or
service delivered over Web
5. WHAT ARE WEB MAP SERVICES?
Technically
• Web Map Service = WMS
• OGC standard for online maps display
• One of a family of standards
Generically
• Way to deliver map content over Web
• Also, feature and geoprocessing
services…
6. OUR GOALS
Project goals broadened over time
Initial goal: create multi-county parcel
Web map
Our focus now is on the research side
• Not just parcels, but any theme
• Limitations, challenges, best practices
• Alignment with other GIS/IT trends
7. OUTLINE OF PRESENTATION
Background (Howard)
Project status (Brenda)
Main challenges (Howard)
• Performance
• Availability of map services
• Resymbolization and map appearance
• Distributed analysis and querying
• Legal issues
9. GEOSPATIAL DATA INTEGRATION
“Combining data from different sources to
provide a unified view of data for users”
A mashup combines different geospatial
themes over one area.
Data integration combines the same
theme over different (adjacent) areas.
10.
11. ADVANTAGE #1
“Well-behaved” problem
Once source datasets are delivered, no
further external inputs are needed
Relatively easy to allocate personnel and
budget to the task
12. ADVANTAGE #2
True integration is possible (in theory)
Rubber sheeting and edgematching
eliminate spatial “gaffs” (slivers, overlaps)
Common data model facilitates analysis
and querying
14. DISADVANTAGE #1
Data producers provide copy of dataset
Issues:
• Data sharing arrangements
• Loss of local control over data
• Possible quality decline due to spatial
and/or attribute adjustments
15. DISADVANTAGE #2
Integrated data may not be current
Depends on time it takes to perform
integration, plus volatility of the layer
Re-integrating updated data is frequently
costly (so often never happens)
16. DISADVANTAGE #3
Curating integrated datasets is
increasingly impractical
Large data volumes (“big data”)
More frequent updates, data collection
17. Need a solution that
incorporates distributed
access, visualization,
analysis and querying
20. PROCESS
Dear <<LIO>>,
This email is to inform you of a joint research project by the State Cartographers Office
(SCO) at UW-Madison and the Department of Geography and Anthropology at UW-Eau
Claire. The purpose of our project is to investigate the feasibility of using Web map
services to create a statewide view of geospatial data collected and maintained at the county
level.
Supported and Our initial goal is to incorporate existing map services showing tax parcels into a single
facilitated by LION
online map application. We have chosen parcel data for this initial test because this type of
data is of interest to a wide range of users. However, our project has a broader purpose,
namely to explore how Web map services can be used to support virtual data integration for
simple mapping and display. Specific research questions we will be investigating include:
- How much flexibility exists to resymbolize map services from different counties? Can we
Contacted stakeholders apply a common set of map symbols to different services to make the integrated map look
more consistent?
- What are the limitations on querying and analysis, given that data models and attributes
(county LIOs) via email are not consistent from county to county? What implications does this have for data
standards?
- How much of an issue is performance? Are there performance improvements that can be
implemented?
- Can we integrate commercial and open-source services in a single Web application?
Gathered REST URL Since our approach makes use of existing Web map services, we do not need to ask for
copies of your geospatial datasets. Instead, we will simply be tapping into existing services
(or other URL) that are already available on the Web. This also means that any map that includes your
county's data will be up-to-date and accurate.
We do have one request. We would greatly appreciate your assistance by providing the
URL of your parcel data Web service. For most counties, this will be an ArcGIS Server
REST endpoint of the form:
Followed up in person https://<host>/ArcGIS/rest/services
or by phone We are also interested in other types of services -- such as ArcIMS, WMS, etc. If you have
such a service, we would appreciate your help identifying its URL.
To provide us with your URL, simply reply to this email or to one of the email addresses
listed below. So that we can keep on schedule for our project, we would greatly appreciate
a response by May 1, 2012.
Please let us know in your email response if you have any conditions or terms associated
21. REST ENDPOINTS
Representational State Transfer
A communications protocol for the Web
http://example.wi.us/arcgis/rest/services
Once a service is published through ArcMap
to ArcGIS Server it is visible at the REST URL
29. PERFORMANCE
Many county REST services designed for
internal access or occasional citizen use
Impact on these users if site usage
increased significantly?
One of the reasons we are not releasing a
public-access viewer at this time
30. SERVICE AVAILABILITY
Services can be added, removed, or
modified
Affects where the map viewer points to
the data
Services may also go off-line
Virtually no control over these changes
33. RESYMBOLIZATION
Need to control symbol color and size
Also need to control zoom levels where
features appear and disappear
Relatively easy to
do, but little
consistency or
standardization
between counties
34. DISTRIBUTED QUERYING
How to query across multiple datasets
with different schemas
A big research issue (“semantic web”)
“Distributed query” or “federated search”
user query subqueries to participating
data sources aggregation of query results
35.
36. LEGAL ISSUES
Opens up new legal issues
Data readily incorporated into any
mashup for any purpose
Loss of control over data
How will local governments
react to this new technology?
38. WHO BENEFITS FROM INTEGRATED DATA?
Geospatial professionals
• Demonstrate ROI for cost of geospatial services
• Model a successful data integration scenario
Counties
• Active participants in an effort that will have
impacts beyond county boundaries
Citizens and taxpayers
• Maximizing utilization of taxpayer investments
• Obtaining greater benefits
• More effective and efficient governance!
39. BENEFITS OF STUDY
Education about GIS Web services
Forward-looking way of thinking about data integration
Challenges and solutions to this new approach
Standards and best practices for Web services