From Oracle to the Web - Automating Spatial Data Updates
1. From Oracle to the Web -
Automating Spatial Data Updates
David Smith, FRS Program Manager, EPA
Amy Ramsdell, GIS Manager, Blue Raster
April 8
2. Overview
The Need: Automate spatial data refresh
The Solution: Technologies used
ArcGIS Server (AGS) map service final output
FME workspaces that automate refresh process
File geodatabase (fgdb) creation
Metadata refresh
Python shutdown script in workspace
FME Server web services used
EPA future plans for FME
Questions
3. The Need:
Automate refresh of map service serving spatial data
Operational database system
The Facility Registry Service (FRS) is a centrally managed
database that identifies facilities, sites or places subject to
environmental regulations or of environmental interest.
Disconnected static file for map service
Fgdb faster performance in AGS
Lessen activity on operational database
Monthly manual refresh
5. ESRI ArcGIS Server Final Output
Web server hosting GIS web
services
REST endpoint for map service
http://geodata.epa.gov/ArcGIS/rest/services/OEI
/FRS_INTERESTS/MapServer
Underlying data: file gdb
28 layers (tables in fgdb)
6. Data Refresh FME Workspace
Oracle Spatial reader
ESRI File Geodatabase
writer
Whole bunch of filtering
in-between to create
separate tables in fgdb
Could build the filtering
in Oracle before it
reaches FME
7. Oracle Spatial Type Reader
Required for spatial object:
Table, materialized view, or view
Shape field is SDO_GEOMETRY
CREATE TYPE sdo_geometry AS OBJECT ( SDO_GTYPE NUMBER, SDO_SRID
NUMBER, SDO_POINT SDO_POINT_TYPE, SDO_ELEM_INFO
SDO_ELEM_INFO_ARRAY, SDO_ORDINATES SDO_ORDINATE_ARRAY);
USER_SDO_GEOM_METADATA table entry
Metadata for spatial tables owned by schema
8. ESRI File Geodatabase API Writer
Non-ArcObjects access
to fgdb
C++ API
Does not require an ArcGIS license on FME
Server or Desktop
Works in ArcGIS Server 10x
18. FME Notification Service - Topics
Create a Success and a Fail notification topic to perform some action
based on the data refresh workspace execution status
19. FME Notification Service - Subscriptions
2 subscribers
Email service to notify when the workspace fails
Metadata workspace runs if data refresh workspace completes successfully
20. Metadata about Map Service
FGDC – Compliant metadata required in XML format
XML file updated for date of fgdb refresh
https://edg.epa.gov/
22. EPA Future Plans for FME
Kick off workspace from Oracle instead of manually in FME Server’s
Services interface – good for more frequent refresh cycles than monthly
using FME’s Job Submitter service
http://servername.com/fmejobsubmitter/Samples/workspacename.fmw?FMESer
verHost=http://servername.com&SuccessTopic=InternetFGDB&Token=4fab5b76
c6yufed2c4731s3d0387bfb366569949&FailTopic=InternetFGDBFail&UserName=a
dmin&Password=password&opt_showresult=false&opt_servicemode=sync
Possibly enhance FRS Query Tools by adding spatial selections and making
more file formats available
Integrate into other EPA program map service refresh flows
Research FME’s ability to translate Resource Description Framework (RDF),
a semantic web data format
24. Thank You!
Questions?
For more information:
David Smith, EPA Facility Registry Service
Smith.DavidG@epamail.epa.gov
@DruidSmith
Judith Doherty, FRS Project Manager,
Indus Corporation
jdoherty@induscorp.com
Amy Ramsdell, Blue Raster
aramsdell@blueraster.com
www.blueraster.com/blog
@AmyRams
Notas do Editor
FME REST services for database extractsUsed in front end Clip and Ship applicationOn-demand requests to services for custom appsScheduled data refreshes