Transforming Data at the
Speed of Streams!

FME
User
Conference
20
22
Mark Warren
Technical Support
Specialist, FME Server
Presenters
Stewart Harper
Technical Director,
Cloud Group

20
22
FME
User
Conference
MUST-HAVE
FME IS A
FOR EASILY DEPLOYING
STREAM PROCESSING WORKFLOWS.

20
22
FME
User
Conference
Real-time data is becoming
one of the biggest aspects of
location intelligence. FME
reduces barriers of entry into
stream processing.

20
22
FME
User
Conference
Agenda
FME is one of the easiest ways to build and deploy
stream processing workflows while harnessing the
power of spatial analysis.
● What are data streams?
● Common challenges when working with streams
● How to work with streams in FME
● Deploying stream workspaces to FME Server

20
22
FME
User
Conference
What are data streams?

20
22
FME
User
Conference
Data streams are continuously generated datasets from
disparate sources such as IoT devices, sensors, etc.

20
22
FME
User
Conference
Characteristics of Data Streams
● Data records are small in size.
● Data volumes can be extremely high.
● Data distribution can be inconsistent with quiet and
busy periods.
● Data can arrive out of sequence compared to when
the event happened.

20
22
FME
User
Conference

20
22
FME
User
Conference
Common stream challenges
1. Very high data volumes
2. Inconsistent distribution of data over time
3. Data can arrive later than the actual event time
4. Difficulty grouping a continuous dataset
5. Many stream processing solutions use custom code
How can we tackle these obstacles?

20
22
FME
User
Conference
Solution: A small workspace like this!

20
22
FME
User
Conference
Streaming Processing with FME
The Solution
The Problem The Result
How do you filter
and spatially
enhance incoming
stream data on a
continuous basis?
Build a workspace
running in stream
mode, and deploy it
to FME Server to run
continuously.
Significantly reduced
data volumes
Spatially enhanced into
point data
Snapped the points to a
road network

20
22
FME
User
Conference
Deploying streams on FME Server

20
22
FME
User
Conference
The Results
• Not a single line of code!
• Reduced storage requirements
by 85%
• Spatially enabled and enriched
the incoming data before
committing to disk
• Grouped data based on time
with the TimeWindower, and
handled out of order events

20
22
FME
User
Conference
FME will equip you with
the necessary tools to
easily get the most value
out of your real-time data!

20
22
FME
User
Conference
Resources
FME and Stream Processing (Landing Page)
Desktop Tips for Working with Continuous Data Streams
How to use the FME Server Streams Interface
Windowing Data Streams in FME

20
22
FME
User
Conference
Got streams? Jump right in!
FME eliminates some of the biggest
hurdles of stream processing, so what
are you waiting for?
Got a stream data challenge? We’d love
to talk to you and get your feedback!

Thank You!
mark.warren@safe.com
stewart.harper@safe.com