1.
Using FME Cloud to produce
real time data transit update
streams (GTFS Realtime)

2.
FME
User
Conference
20
22
Tino Miegel
Software Architect

3.
20
22
FME
User
Conference
Stadtwerke Münster
• Münster’s public utility company
• Up to 150k passengers a day
• 100k stops per day
• 22 Lines at day, 6 lines at night
• ~120 vehicles on the road at prime time
• 10 million kilometers a year
• Münster has no subway or streetcars,
but >300k residents

4.
20
22
FME
User
Conference
Goal: Opening up a
closed can of data
• Digital signage at
bus stops
• Internal fleet
controlling
• Dozens of sensors
in vehicles
• But no public API
☹

5.
20
22
FME
User
Conference
How it started…
<?xml version="1.0" encoding="UTF-8"?>
<DatenAbrufenAntwort>
<Bestaetigung Zst="2022-06-17T10:24:17+02:00" Ergebnis="ok"
Fehlernummer="0"></Bestaetigung>
<WeitereDaten>false</WeitereDaten>
<AUSNachricht AboID="2">
<IstFahrt>
<LinienID>6</LinienID>
<RichtungsID>870</RichtungsID>
<FahrtRef>
<FahrtID>
<FahrtBezeichner>170622_100035_8_6_23</FahrtBezeichner>
<Betriebstag>2022-06-17</Betriebstag>
</FahrtID>
<FahrtStartEnde>
<StartHaltID>4201002</StartHaltID>
<Startzeit>2022-06-17T10:35:00+02:00</Startzeit>
<EndHaltID>4722802</EndHaltID>
<Endzeit>2022-06-17T11:37:00+02:00</Endzeit>
</FahrtStartEnde>
</FahrtRef>
<Komplettfahrt>false</Komplettfahrt>
<PrognoseMoeglich>true</PrognoseMoeglich>
<FahrzeugTypID>2_1553</FahrzeugTypID>
</IstFahrt>

6.
20
22
FME
User
Conference
How it ended…

7.
20
22
FME
User
Conference
Project history
• ~2014
• Initial evalutation of FME Desktop to produce GTFS data from
various source formats available for export
• Mapping stops with D-GPS in the field and via referenced arial
photographs
• ~ 2015 Going „live“ on Google with static trip schedules
• Adding optional shapes = vehicle travel paths
• ~2019
• First prototype of a public realtime API with FME as a connector
between internal XML API and a public REST API and websocket
• 2020 Google Realtime feed goes live

8.
20
22
FME
User
Conference
Existing real time interface
● ITCS = Intermodal Transport Control
system = rechnergestütztes
Betriebsleitsystem (RBL)
● XML based subscription model
● Designed to interchange trip schedule data
between transportation agencies
● Designed to connect a handful clients
● HTTP
● Only available in intraweb

9.
20
22
FME
User
Conference
Connecting FME Cloud
IVU System
Stadtwerke
MS
FME Cloud
Instance
EC2 Image
AWS Cloud
Local ICTS
IPSec Tunnel
HTTP HTTPS & Authentication
Traefik
Reverse
Proxy

10.
20
22
FME
User
Conference
Scale up the system
Web
Socket
Rest
Service
Live Public Transport
Service
FME Cloud
Instance
AWS Cloud Public Access

11.
20
22
FME
User
Conference
2 Subscriptions
• VIS = Visualisierungsdienst = Visual
• Real time vehicle positions
• Fetch data every 10 seconds
• AUS = Ist Daten Auskunft = Trip info timetables
• Trips
• Time horizon 120 minutes
• Trip delays
• Trip cancellations
• Vehicle Arrival forecasts
• Fetch data every 15 seconds

12.
20
22
FME
User
Conference
ITCS Subscriptions Alive handling
• „Verliert der Server seine Abonnement Daten, so ist dies
zunächst vom Client nicht feststellbar.“
• Server does not persist subscriptions
• The client has to regularly query the server about his
uptime
• If the server has been restarted since the client
subscribed, it is the client‘s duty to renew his subscription.

13.
20
22
FME
User
Conference
Workspace design handling
subscriptions

14.
20
22
FME
User
Conference
Data workflow

15.
20
22
FME
User
Conference
Data workflow
• 20% XMLFragmenter and
AttributeManager
• 80% DateTimeConverter and
DateTimeCalculator
• UTC with out without time zone,
local times, Unix Epochs…
• Relative time („2 minutes delay“) to
absolute time („arrives at 15:03“)

16.
20
22
FME
User
Conference
GTFS Realtime
• FME has a GTFS reader/writer but none
for GTFS-RT
• Google Protocol Buffers based binary
format
• PythonCaller
• HTTPCaller to push data into Google
Transit Partner Portal

17.
20
22
FME
User
Conference
GTFS Realtime
• All data has to be referenced to the
static feed data
• Separate Feeds / endpoints for
• Vehicle positions
• Trip Updates
• Additional trips
• Cancelled trips
• Delays
• (Alerts, e.g. constructions, medical
alerts...)

18.
20
22
FME
User
Conference
Python – Initiliaze Feed
from google.transit import gtfs_realtime_pb2
def __init__(self):
self.newfeed = gtfs_realtime_pb2.FeedMessage()
self.newfeed.header.gtfs_realtime_version = "2.0"
self.newfeed.header.incrementality =
gtfs_realtime_pb2.FeedHeader.Incrementality.FULL_DATASET
self.newfeed.header.timestamp = int(time.time())

19.
20
22
FME
User
Conference
Python – Add entities to feed
def input(self, feature):
vehicle_ent = self.newfeed.entity.add()
vehicle_ent.id = str(self.counter)
vehicle_ent.vehicle.vehicle.id = feature.getAttribute("fahrzeugid")
vehicle_ent.vehicle.position.latitude = feature.getAttribute("Latitude")
vehicle_ent.vehicle.position.longitude = feature.getAttribute("Longitude")

20.
20
22
FME
User
Conference
Python – Write ASCII and Binary Feed
def close(self):
fmeobjects.FMELogFile().logMessageString("Position Feed has been initialized:
{}".format(self.newfeed.IsInitialized()))
with open(join(fme.macroValues['FME_SHAREDRESOURCE_TEMP'], "vehicles.pbfa"), "w") as f:
f.write(text_format.MessageToString(self.newfeed))
with open(join(fme.macroValues['FME_SHAREDRESOURCE_TEMP'], "vehicles.pbf"), "wb") as f:
f.write(self.newfeed.SerializeToString())
fmeobjects.FMELogFile().logMessageString({} vehicle positions to send".format(self.counter))
f = fmeobjects.FMEFeature()
self.pyoutput(f)

21.
20
22
FME
User
Conference
Pushing Data to Google / Archive
Storage

22.
20
22
FME
User
Conference
Google Transit Portal

23.
FME
User
Conference
20
22

24.
20
22
FME
User
Conference
Outlook
• Enhance VPN reverse proxy to
handle bi-directional traffic
• Switch from „pull data“ to „push
data“
• Reduce traffic at nighttime and
on weekends
• Open question
• Best way to handle expired
subscriptions
• „Webhook not triggered for x
seconds“ anyone? ;)

25.
20
22
FME
User
Conference
Live action
• API
https://api.busradar.conterra.de/
• https://netzplan-muenster.de/
• FIS
• Single Stop View
• Multi View

26.
Thank You!
t.miegel@conterra.de