This presentation tells the story, and FME solutions of a Dutch Utility company for the automatic exchange of data containers containing RDF Linked data, BIM, and documents.
The presentation will focus on the non-traditional representation of RDF Linked Data and how this integrates with FME through SPARQL, Apache Jena, and a few customer-built transformers in FME.
This FME solution also uses my Excel switch-based method of directing the data flow (my presentation during the FME World Fair).
3. FME
User
Conference
20
22
Oscar Timothy Willems
FME Certified Specialist
BSc. Architecture, Urbanism & Building Sciences
(2015)
@ Delft University of Technology, The Netherlands
MSc. in Geomatics (2017)
@ Delft University of Technology, The Netherlands
Erasmus+ (2016-2017) @ KTH Royal Institute of
Technology, Stockholm, Sweden
FME Specialist (2018 - present)
@ Vicrea Solutions B.V., The Netherlands
4. FME
User
Conference
20
22
Oscar Timothy Willems
Published Research
Exploring a pure landmark-based approach for
indoor localization
@ Delft University of Technology
DOI: 10.13140/RG.2.2.29510.32329
Temporal and spatial dynamics of falls among
older pedestrians in Sweden
@ Journal of Applied Geography
DOI: 10.1016/j.apgeog.2018.12.007
www.vicrea.nl
5. 20
22
FME
User
Conference
Agenda
The challenge of RDF-Linked Data
• Project context and requirements
• Introduction to RDF Linked Data
• FME Solution for BIM containers and
RDF-Linked Data
• Closing statement and lessons learned
7. 20
22
FME
User
Conference
Project context and requirements
Complex project:
• Large scale IT Infrastructure
• Sensitive timelines
• Mission critical infrastructure
• Difficult stakeholder
management
Linked Data
• Single point of truth, no
duplication
• Multi organisational use
• Transparency
• Improved use for reporting
Scale of construction projects
+ €4.000.000.000,- in project
+ 400 project sites
+ 10 years of construction
+ 10 contractors
8. 20
22
FME
User
Conference
Project
Requirements
• Information exchange standard
for asset redevelopment
• Relatics
• Cloud Database Management
System
• Triple Store + query endpoint
• OTL (Object Type Library)
• Predefined object/property
definitions and code lists
• Classic ETL challenge?
Commissioning
Party
Contractor
RDF-Turtle
Documents
Drawings
IFC
BIM-Assets
9. 20
22
FME
User
Conference
Proof of concept requirements
• Be able to read all the content of a data container
• Be able to validate the content of a data container
• Be able to process the RDF-Turtle format files
• Be able to run SPARQL and process the results
• Be able to integrate with the API of Relatics
• Be able to self-serve via a Web App
• Be able to read all the content of a data container
• Be able to validate the content of a data container
• Be able to process the RDF-Turtle format files
• Be able to run SPARQL and process the results
• Be able to integrate with the API of Relatics
• Be able to self-serve via a Web App
11. 20
22
FME
User
Conference
• Source: 5-star deployment scheme for Open Data (Tim Berners-Lee) https://5stardata.info/en/
Introduction to RDF Linked Data
• RDF-Linked Data
• OL = Open License
• RE = Machine readable
• OF = Open Format
• URI = Uniform Resource
Identifier
• LD = Linked Data
12. 20
22
FME
User
Conference
Introduction to RDF Linked Data
• RDF-Linked Data
• Triples in graph data structure
• Subject-Predicate-Object triples
Source: https://www.w3.org/TR/rdf11-primer/
<Bob> <is a> <person>
<Bob> <is a friend of> <Alice>
<Bob> <is born on> <the 4th of July 1990>
<Bob> <is interested in> <the Mona Lisa>
<the Mona Lisa> <has Wikipedia> <http://www.Wikipedia.org/entity/Q12418>
<the Mona Lisa> <was created by> <Leonardo da Vinci>
<the video 'La Joconde à Washington'> <is about> <the Mona Lisa>
13. 20
22
FME
User
Conference
Introduction to RDF Linked Data
• RDF-Linked Data
• Triples in graph data structure
• Subject-Predicate-Object triples
<Bob> <is a> <person>
<Bob> <is a friend of> <Alice>
<Bob> <is born on> <the 4th of July 1990>
<Bob> <is interested in> <the Mona Lisa>
<the Mona Lisa> <has Wikipedia> <http://www.Wikipedia.org/entity/Q12418>
<the Mona Lisa> <was created by> <Leonardo da Vinci>
<the video 'La Joconde à Washington'> <is about> <the Mona Lisa>
Source: https://www.w3.org/TR/rdf11-primer/
14. 20
22
FME
User
Conference
Introduction to RDF Linked Data
• RDF-Linked Data
• Triples in graph data structure
• Subject-Predicate-Object triples
• RDF-Turtle format: Resource Description Framework (Semantic Web W3C)
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
@prefix rel: <http://www.perceive.net/schemas/relationship/> .
<#green-goblin>
rel:enemyOf <#spiderman> ;
a foaf:Person ; # in the context of the Marvel universe
foaf:name "Green Goblin" .
<#spiderman>
rel:enemyOf <#green-goblin> ;
a foaf:Person ;
foaf:name "Spiderman", "Человек-паук"@ru .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
@prefix rel: <http://www.perceive.net/schemas/relationship/> .
<#green-goblin>
rel:enemyOf <#spiderman> ; # URI-refence
a foaf:Person ; # in the context of the Marvel universe
foaf:name "Green Goblin" . # literal value
<#spiderman>
rel:enemyOf <#green-goblin> ;
a foaf:Person ;
foaf:name "Spiderman", "Человек-паук"@ru .
15. 20
22
FME
User
Conference
Introduction to RDF Linked Data
• RDF-Linked Data
• Triples in graph data structure
• Subject-Predicate-Object triples
• RDF-Turtle format: Resource Description Framework (Semantic Web W3C)
• SPARQL: “SPARQL Protocol And RDF Query Language” (W3C standard)
• Query Language for NoSQL Graph databases
Endpoint: https://bag.basisregistraties.overheid.nl/sparql/now
prefix bag: <http://bag.basisregistraties.overheid.nl/def/bag#>
SELECT ?x WHERE {
?x a bag:Building.
?x bag:originalYearOfConstruction ?yearBuilt
FILTER (?yearBuilt = 1923)
}
limit 100
17. 20
22
FME
User
Conference
Proof of concept requirements
• Be able to read all the content of a data container
• Be able to validate the content of a data container
• Be able to process the RDF-Turtle format files
• Be able to run SPARQL and process the results
• Be able to integrate with the API of Relatics
• Be able to self-serve via a Web App
18. 20
22
FME
User
Conference
Architecture to processing BIM containers
• FME Server managed by Vicrea
• Apache Jena + Fuseki RDF Triple
store + SPARQL-endpoint
• Customer’s systems are remotely
accessible through API’s
• FME Server Web apps authorized
for each project
19. 20
22
FME
User
Conference
FME solution to processing BIM containers
Pre-flight checks
Contract Container
Validate
container
name
Validate supported
formats
Upload RDF to
Apache Jena /
Fuseki
Run
SPARQL
Read
SPARQL
Validate
content of
container
based on
SPARQL
results
Export to:
- Relatics
- Document Management
- Asset Management
HTML Validation /
Export report
Export?
20. 20
22
FME
User
Conference
RDF Linked Data and FME
• Container has naming
convention
• Container is for a specific project
• Read and validate content of
data container
21. 20
22
FME
User
Conference
RDF Linked Data and FME
Read and validate content of
data container
• Container has naming
convention
• Container is for a specific project
• Container is an archive (.zip)
• Contains multiple folders
• Alphanumeric
• Documents
• Geometric
Directory & Filepath reader
unzips an archive with path
features to the files and folders
within the zip
22. 20
22
FME
User
Conference
RDF Linked Data and FME
Read and validate content of
data container
• Container has naming
convention
• Container is for a specific project
• Container is an archive (.zip)
• Contains multiple folders
• Alphanumeric
• Documents
• Geometric
• Restrictions on file types per
folder
Collect validation errors or
processing statistics per step in
a (HTML) report
23. 20
22
FME
User
Conference
RDF Linked Data and FME
Read and process RDF-Turtle
data format
• RDF-Turtle is text based
• Non-traditional data format
• No native support in FME
• Apache Jena: Graph Database /
SPARQL + API
• Apache Fuseki: Web interface
to RDF data + API
24. 20
22
FME
User
Conference
RDF Linked Data and FME
Read and process RDF-Turtle
data format
• RDF-Turtle is text based
• Non-traditional data format
• No native support in FME
• Apache Jena: Graph Database /
SPARQL + API
• Apache Fuseki: Web interface
to RDF data + API
25. 20
22
FME
User
Conference
RDF Linked Data and FME
Run SPARQL query, then parse,
process and validate result
• Heartbeat for Apache Jena/Fuseki:
Graph + SPARQL API endpoint
Doing ‘pre-flight checks’ and
having heartbeat monitoring,
saves you from surprises
26. 20
22
FME
User
Conference
RDF Linked Data and FME
Run SPARQL query, then parse,
process and validate result
• Heartbeat for Apache Jena/Fuseki:
Graph + SPARQL API endpoint
• Encode Query for HTTP-request
• Result is ‘traditional table view’ of data
27. 20
22
FME
User
Conference
RDF Linked Data and FME
Run SPARQL query, then parse,
process and validate result
• Heartbeat for Apache Jena/Fuseki:
Graph + SPARQL API endpoint
• Encode Query for HTTP-request
• Result is ‘traditional table view’ of data
• Determine interpretation of result
Using logic based ‘switches’
you can route features to each
be processed differently
28. 20
22
FME
User
Conference
RDF Linked Data and FME
Integrate with Relatics ingestion endpoint
• SOAP-based endpoint
• Base64 encoded XML body
• SPARQL Export
• SOAP Envelope with project information
Don’t forget your cleanup
steps!
29. 20
22
FME
User
Conference
RDF Linked Data and FME
Self-service FME Server Web App
• Workspace is published as a Workspace App
per project
• Project specific (hidden) parameters are set
by admin
• User can upload a data container
• Pick validation & export or validation only
• User can set project specific parameters
• Collated in one FME Server gallery app
• Authorization per project for personalized FME
Server gallery app tiles
30. 20
22
FME
User
Conference
FME solution to processing BIM containers
Pre-flight checks
Contract Container
Validate
container
name
Validate supported
formats
Upload RDF to
Apache Jena /
Fuseki
Run
SPARQL
Read
SPARQL
Validate
content of
container
based on
SPARQL
results
Export to:
- Relatics
- Document Management
- Asset Management
HTML Validation /
Export report
Export?
32. 20
22
FME
User
Conference
Closing statement
• RDF-Linked Data is fundamentally different
• FME can also be used to orchestrate and
shuttle data files back and forth
• With standard FME functionality you can make
some cool mechanics
Collaboration with Safe Software on support for
RDF is in progress (no timeline available yet)
34. 20
22
FME
User
Conference
Please Share Your Session Feedback
Log in & navigate to the
agenda. Select the
session you attended
Scroll down to “Session
Feedback”
Download EventMobi app
Event code: fmeuc
Click “Launch App”
Fill out the survey
and submit