Getting Intergraph and Esri to Play Nicely Using FME
1. Getting Intergraph and Esri to
Play Nicely Using FME
Lou Manglass
Systems Analyst II
9 APR 2013
2. Introducing The Kids
ESRI
ArcGIS Server (Planning, Public Works)
Includes Feature-Linked Annotation
Shapefiles (UGA, Assessor)
File/Personal Geodatabases (various annotations)
Intergraph
Proprietary .map format (.dgn v7) for each agency
Proprietary .ply format (text polygon output)
SQL/Excel (non-spatial sources)
Various spatial information in non-spatial formats
3. Intergraph’s Needs
Creating a 911 map isn’t just dumping out a file
Three customer groups require different .maps
ACCPD/UGA Dispatchers, ACCPD/UGA mobiles, Fire
Modern formats hard to convert to DGN v7
Text files (.ply) defining dispatch zones/beats
SQL Database
Special Address tables maintain verified addresses
and location of access for parcels
Segment and Node tables manage road network and
routing information
Code Tables (e.g. street) provide LUTs
4. A Quick Detour: DGN v7
File system Database
DGN v7 files .map file SQL Database Spatial Data
MSlink
Tabular Data
63 layer limit Spatial info in UORs
32 MB limit (DGN) MSlink relates map
Retired in 1999 Funky legacy issues
6. Setting The Play Date
GeoMedia Pro is cumbersome to use with I/CAD,
little room for automation and no data validation
FME provides validation and conversion paths for
all of our data sources
Data is synthesized, checked for changes and
verified despite political barriers
ESRI Annotations converted letter-by-letter to DGN
Tabular sources can be converted to spatial data
And vice-versa
And we can version product files
17. How FME Returned Our Investment
(TL;DL: Very Well)
License for GeoLabel Pro: $1,995 / $399 p.a.
Time saved through automation: 2 days/roll ($480)
Data collection, validation, conversion automation
4 weeks to 4 hours! (FME gets half credit: $2,340)
That’s per roll, by the way
Fewer errors: ≈60 output layers, all automated
Version stamps also reduce testing confusion
Customers much happier with consistent product
Target 4 map rolls a year, but now could do 5 or 6
18. Where Do We Go Next?
More scripting
Continue to logically group sets of workspaces
Feature Addition
Add custom fields, functions to dispatch workflow
FME performs ETL, validation, QA, logging
FME Server
Offload processing from map development station
Schedule workspace groups for nightly builds
Provide daily graphic-side updates for .map file
19. Thank You!
Questions?
For more information:
Lou Manglass:
Louis.Manglass@AthensClarkeCounty.com
Athens-Clarke County Police Department
Notas do Editor
Feature-Linked Annotation was one of the biggest reasons for going with FME, because it was able to move that to DGN v7.
Each .map file shares the same layer configuration to streamline production and provide interoperability between agencies, but determines if and when those layers turn on and off as required by agency/application. We used to have more maps, but we’ve been able to reduce them to three groups to minimize variations and effort.Intergraph’s I/CAD is not capable of dynamic labeling, so all labels must be created and be capable of being shoehorned into the limited abilities of a DGN v7. That’s a shame, because we’re an Esri organization for our source data, and we already had Maplex. Maplex is much better than GeoMedia Pro, even better than third-party labeling engines in some ways. However, most software packages (such as ESRI) don’t even support writing to DGN v7 anymore. A key feature that attracted us to FME was the ability to format shift so we could make good use of our software investments.SQL database has (mostly) A and B map tables which can be edited independently so a new map can be staged while the system is live, and both sets of tables are independent of the rest of the dispatch system (procedures/searches link up data, not keys, so the system doesn’t miss a beat if all map keys change instantly).sp_ad,co_pl track business or common names for points, and sp_ad is used to validate addresses that are entered into the system in both CAD and RMS. For parcels, sp_adx,y locations can show point of access as well.Code tables, such as st_cd, provide look-up tables and establish referential integrity for naming (in the case of st_cd, provides referential integrity across segments, addresses, intersections)
Technically there are 64 – 0 is a layer, but it is a special-case layer that is not supported by all DGN v7 implementations and has even more special-case uses in the .map file, so stay away from 0 to avoid headaches!The .map wrapper allows users to wrap all DGNs into one file without the hard file size limits. ACC is not a large Intergraph installation, but because of all the annotations we’re close to 100 MB in size.Funky legacy issues e.g.: bitmasks for routing, right-justified address text fields
The higher the number, the later it is in the draw order. So, layer 1 gets drawn first, then 2, through 63 last.1-12: generally base layer info: cadastre, water bodies, transit base layers (not roads)13-16: UGA-specific base layer info17-19: other base layer info (Fire, ACCPD)20-22: Building footprints23-33: Assorted higher level base layer info34-46: Road layers and common annotations (some missing here, they’re in one DGN with intersections)47-50: Address annotation, grouped by application (some are for mobiles, others for dispatchers)51-58: Road annotations and highway shields (vary depending on application)59: Trails and non-road access information61: Annotation labeling subdivisions62: Railroad/Road intersections, with identifier annotations63: Special Address point layer
So how do we get all the data sources into the map deliverables?An efficient structure for GIS would give personnel access to a pool of resources. Failing that, there would be a structure of replication between resource silos to prevent data duplication. At Athens Clarke County, we prefer to keep our data isolated between departments. We’re probably not the only County to do so. A replication workspace copies down data from various sources at the County and compares it to our existing data. Changes are flagged for review and the deltas live in an “incoming” version.ACCPD uses Feature-Linked Annotations in an ESRI ArcGIS Server to maintain road and address labels. The results look great, much better than the native methods provided in GeoMedia Pro, but they are not natively supported in DGN v7. Safe came to the rescue with an annotation to DGN transformer workspace, which we tweaked to handle our workflow. By breaking each annotation down into individual letters and manipulating them appropriately, we can emulate fancy features (such as following curved lines) that would otherwise not be possible. It does come at a penalty to file size: ours grew from roughly 50 MB to 100 MB, but we also added labels so it’s not a direct comparison. We’re down to nearly 90 MB now after removing some unnecessary/duplicate information, but it’s still not a straight comparison because of the map process merging. Again, FME and Safe software did all the work to get these annotations converted. We basically plugged in the data, hit the run button and got immediate return on our investment.Some data sources, such as fire hydrant locations and some addressing information, exist only in tabular form. There’s Excel, Access, SQL Server…the list goes on. FME allows us to pull that data, extract the location information, and dump out a spatial file. In some steps, we even produce another tabular record but change the coordinate system. FME allows us to make these adjustments on the fly and do our data formatting and validation in-line.And we can go the other way, too: spatial data to raw text. Our dispatch zones are currently held in spatial datasets. Normally, it is a manual process to convert them to polygon (.ply) files in GeoMedia Pro and it requires a special plugin that you had best not misplace. Instead, ACCPD has created a workspace to create the PLY files from their data source. Not only is it faster in FME, but it can be automated through scripts. Any time you can automate repetitive tasks you can minimize the risks of human error, which leads to efficiency gains and a consistently positive user experience. Since this is a special-case format that very few places use, it’s not worth creating a reader/writer. Keep in mind that there are over 300 formats, so if you’re not using Intergraph for dispatch your formats are probably all natively supported. If you do choose to go with I/CAD, feel free to get in touch with me if this workspace is of interest.A nice bonus is that ACCPD has a parameterized workspace in which we can load a verison number and publish date to stamp each map file we create. A map has a major, minor, and patch number that we would integrate with a change management system. This is a huge boon for testing and for verifying that our mobiles are all up to date. As an added bonus, different agencies can be on different versions if operational requirements change, as long as we keep track of the dependencies. It’s a small bonus, but it’s saved us hours if not days of time lost to confusion between testing, production, and agency variants.
So, FME isn’t free, right? How well did FME return our investment? The Too Long; Didn’t Listen is “Very Well”Mid-range Analyst II at our department would take $25/hr in salary. MIT conservatively said that 1.25-1.4x base salary is the total cost of an employee, so that’s $30/hr or $240/day cost to the organization.http://web.mit.edu/e-club/hadzima/how-much-does-an-employee-cost.htmlFirst, Intergraph recommends MapText’sGeoLabel Pro for labeling in GeoMedia Pro. It’s not as good as Esri’sMaplex, so if you are an Esri organization like us and already have a Maplex license then this is probably relevant. It gives us the power of a better labeling engine, feature-linked annotation that’s up to date, and software automation that saves us two days every map roll. Already, FME is looking pretty good. And MapText solves a single problem. FME’s just getting warmed up.http://www.maptext.com/products/geolabel-pro/purchase-geolabel-pro/It also saves us money in time per roll. The target is four rolls per year, load willing. It turns out that with the levels of automation we hit, we’re actually saving so much in time that now testing is our biggest time investment and we could be updating even more often than that. 5 or 6 full rolls a year is possible (a full roll updates all data). That’s 25%-50% of a position if you wanted to measure it that way. But I want to look a bit more into just how efficient it is.Automating our data conversion reduced a lot of manual labor. Now 4 weeks to 4 hours sounds crazy, but let me explain: at Athens-Clarke County, we don’t believe in things like Tier 1 and Tier 2 support. People walk in, and whoever is there fixes whatever problem arises. It’s a disruptive workplace, so manual processes take longer than necessary. We used to allot 4 weeks for map builds, because that’s how long it would take to get the work done with all the other interruptions. But those interruptions are part of the position, so it’s important to measure their impact on work. And it’s a big impact: up to $4,680 per roll in personnel time. Granted, some of that could be saved through better organization policies, but at least half of that is credited to FME. We now use that time to develop new tools and layers to improve the quality of the product instead of standing still on the same old map every time.$240/day, 19.5 days (5 days/wk, 4 wks).And consistency in a 911 map is key. We have roughly 60 layers from dozens of source datasets of varying formats. By automating it in workspaces, we ensure consistency with every run. Try that with manual manipulation through GeoMedia Pro. Consistency cannot be enforced, plus you can’t just get up and walk away from the mouse. There’s time savings in here as well, in both consistent output of layers and the ability to stamp all our products with a version. This saves hours of confusion in testing with all of the products we have, and the customers are much happier in the end. This makes them more willing to report issues on the map, because they know the issues will get fixed, and suggest new features. What price do you put on customer satisfaction? But no need to enumerate at this point, I think the benefits are clear.(Can we discuss pricing? Reference: $3350/3370 with shipping, 20% maintenance is $670 p.a.
And that’s not even all FME has to offer us. We’re in our second year with the software and still finding more ways to use the software with no additional licensing costs!Schedule workspaces – more employee savings, built datasets are ready to deploy, faster turnaround for layer changes and complete map rolls. We already hit our big efficiencies here, but there’s still a fair amount of automation we can accomplish that, if we up the frequency of our rolls, will add up.