This training camp teaches you how FIWARE technologies and iSHARE, brought together under the umbrella of the i4Trust initiative, can be combined to provide the means for creation of data spaces in which multiple organizations can exchange digital twin data in a trusted and efficient manner, collaborating in the development of innovative services based on data sharing and creating value out of the data they share. SMEs and Digital Innovation Hubs (DIHs) will be equipped with the necessary know-how to use the i4Trust framework for creating data spaces!
3. NGSI-LD Specific Headers
NGSI-v2 headers
fiware-service
fiware-servicepath
NGSI-LD headers
NGSILD-Tenant - equivalent to fiware-service
NGSILD-Scope ??? - not defined in the NGSI-LD specification
Context brokers are implicitly multi-tenant. The default NGSILD-Tenant is blank.
Data from separate tenants is held in separate databases for legal reasons.
2
4. Content-Type Header
Supported Content-Types
application/json
application/ld+json
Default is application/json, in which case the
@context must be supplied in a Link header
see: https://developer.mozilla.org/en-US/
docs/Web/HTTP/Headers/Link
Link Header is to be preferred as it reduces the size of
the payloads
Follow JSON-LD best practices.
see https://w3c.github.io/json-ld-bp
3
{
"@context": [
"https://fiware.github.io/data-models/context.jsonld",
"https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld"
],
"id": "http://dbpedia.org/resource/John_Lennon",
"type": "Person",
"name": {"type": "Property", "value": "John Lennon"},
"born": {"type": "Property", "value": "1940-10-09"},
"spouse": {"
type": "Relationship",
"object": "http://dbpedia.org/resource/Cynthia_Lennon"
}
}
{
"id": "http://dbpedia.org/resource/John_Lennon",
"type": "Person",
"name": {"type": "Property", "value": "John Lennon"},
"born": {"type": "Property", "value": "1940-10-09"},
"spouse": {"
"type": "Relationship",
"object": "http://dbpedia.org/resource/Cynthia_Lennon"
}
}
'Link: <http://.../path-to-my-public-server/ngsi-context.jsonld>; rel="http://www.w3.org/ns/json-
ld#context"; type="application/ld+json"'
5. Accept Header for GET /entities and Subscription
payloads
Supported Accept Types
application/json - @context is returned in a Link header
application/ld+json - @context is returned in the payload body
application/geo+json - GeoJSON response for GET /entities and subscriptions
see https://tools.ietf.org/html/rfc7946
The fallback for error messages is application/json
Common NGSI-LD Formats
options=normalized
options=keyValues
Custom Formats may be supported by selected context brokers:
options=x-ngsiv2-normalized
options=x-ngsiv2-keyValues
options=x-ngsiv2-keyValues-compacted
Custom NGSI-LD Formats should be used connection to microservices only
Do not use them for data exchange
4
6. GeoJSON request example
Give me all Animal entities which are pigs inCalf to be found within 2km of 13.364°N 52.52°E
… and return the data as key-value pairs in GeoJSON format without an @context attribute
5
curl -G 'http://localhost:1026/ngsi-ld/v1/entities/'
-d 'georel=near;maxDistance==2000'
-d 'geometry=Point'
-d 'coordinates=%5B13.364,52.52%5D'
-d 'q=species==%22pig%22;reproductiveCondition==%22inCalf%22'
-d 'type=Animal'
-d 'options=keyValues'
-H 'NGSILD-Tenant: openiot'
-H 'Accept: application/geo+json'
-H 'Prefer: body=json'
-H 'Link: <http://.../path-to-my-public-server/ngsi-context.jsonld>;
rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json'
Use Prefer=ld+json to return in GeoJSON-LD format
see https://geojson.org/geojson-ld/
7. GeoJSON response example
▪ Since entities typically have a location they
can be plotted onto a map.
▪ GeoJSON is used as an output format only.
▪ Any GeoJSON Feature and/or FeatureCollection
can be easily digested by any GIS system.
6
{
"type": "FeatureCollection",
"features": [
{
"id": "urn:ngsi-ld:Animal:pig016",
"type": "Feature",
"properties": {
"type": "Animal",
"heartRate": 62,
"phenologicalCondition": "femaleAdult",
"reproductiveCondition": "inCalf",
"name": "Tango",
"legalID": "F-sow016-Tango",
"sex": "female",
"species": "pig",
"location": {
"type": "Point",
"coordinates": [13.355, 52.523]
}
},
"geometry": {
"type": "Point",
"coordinates": [ 13.355, 52.523]
}
},
...etc
]
}
8. NGSI-LD Temporal interface
Give me the last 5 readings about a single entity and return in default (normalized) format:
7
curl -G -X GET 'http://localhost:8080/temporal/entities/urn:ngsi-ld:Animal:cow001’
-d 'lastN=5'
-H 'NGSILD-Tenant: openiot'
-H 'Link: <http://.../path-to-my-public-server/ngsi-context.jsonld>;
rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"'
▪ Temporal endpoints are found under /temporal/entities
▪ Temporal endpoints are optional - not supported by all context brokers
▪ Gives a context broker a “memory” at the cost of data storage and maintenance.
▪ Expect a performance hit - don’t run as DEBUG
Sample docker-compose:
https://github.com/FIWARE/tutorials.Short-Term-History/blob/NGSI-LD/docker-compose/orion-ld.yml
10. Temporal Queries on attributes without observedAt
Give me the last 5 readings about all female Animals, and return them 2 at a time
9
curl -G -X GET 'http://localhost:8080/temporal/entities'
-d 'type=Animal'
-d 'pageSize=2'
-d 'lastN=5'
-d 'q=sex==%22female%22'
-d 'timeproperty=modifiedAt'
-d 'options=count'
-H 'NGSILD-Tenant: openiot'
-H 'Link: <http://.../path-to-my-public-server/ngsi-context.jsonld>;
rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"
▪ Default temporal attribute is observedAt.
▪ static attributes are usually not observed - cannot be queried in the q parameter directly
▪ Use timeproperty=modifiedAt to query static properties
11. Temporal Response including modifiedAt
10
{ "id": "urn:ngsi-ld:Animal:cow003",
"type": "Animal",
"heartRate": [
{
"type": "Property",
"value": 51.0,
"observedAt": "2021-04-26T09:36:36.577Z",
"modifiedAt": "2021-04-26T09:38:09.579Z",
"instanceId": "urn:ngsi-ld:attribute:instance:627f4202-a673-11eb-89a1-0242ac120106",
"unitCode": "5K",
"providedBy": {
"object": "urn:ngsi-ld:Device:cowcollar003",
"type": "Relationship",
"modifiedAt": "2021-04-26T09:38:09.579Z",
"instanceId": "urn:ngsi-ld:attribute:instance:62816672-a673-11eb-89a1-0242ac120106"
}
}
… etc
▪ modifiedAt is returned in the response.
▪ There may be a significant lag between observedAt and modifiedAt
▪ modifiedAt identifies the last confirmed value, not necessarily the last change of value
12. Pagination options
Query Parameters
▪ lastN - limits the number of returned Attributes
▪ pageSize - limits the number of returned Entities
▪ pageAnchor - id of the first returned Entity
▪ options=count - includes the number of entities
as a header in the response
Relevant Headers in response
▪ Content-Range -
date-time 2021-04-26T09:41:15.752-2021-04-26T09:29:10.834/5
▪ NGSILD-Results-Count - 174
▪ Page-Size - 2
▪ Next-Page - urn:ngsi-ld:Animal:cow004
11
curl -G -X GET
'http://localhost:8080/temporal/entities'
-d 'type=Animal'
-d 'pageSize=2'
-d 'lastN=5'
-d 'q=sex==%22female%22'
-d 'timeproperty=modifiedAt'
-d 'options=count'
-d pageAnchor=urn:ngsi-ld:Animal:cow004
...etc
13. Time limiting and Geofencing Temporal Queries
Give me the heartRate, location and controlledAsset attributes of all Device entities,
found within 800m of 13.364°N 52.52°E and return all readings taken since 8:30 a.m on 22nd
April, returning them 2 devices at a time and in temporal values format
12
curl -L -g -X GET 'http://localhost:8080/temporal/entities'
-d 'type=Device'
-d 'attrs=location,controlledAsset'
-d 'options=temporalValues'
-d 'georel=near%3BmaxDistance==800'
-d 'geometry=Point'
-d 'coordinates=[13.364,52.52]'
-d 'timerel=after'
-d 'timeAt=2021-04-22T08:33:51.255Z'
-d 'pageSize=2'
-H 'NGSILD-Tenant: openiot'
-H 'Link: <http://.../path-to-my-public-server/ngsi-context.jsonld>;
rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"'
-H 'Accept: application/json'
14. Temporal Values Response
13
[
{
"id": "urn:ngsi-ld:Device:pigcollar001",
"type": "Device",
"heartRate": {
"type": "Property",
"values": [
[ 61.0, "2021-04-26T08:55:56.100Z"]
...etc
]
},
"location": {
"type": "GeoProperty",
"values": [
[{"type": "Point", "coordinates": [13.355, 52.516, 0.0]},"2021-04-26T08:55:56.100Z"],
...etc
]
},
"controlledAsset": {
"type": "Relationship",
"objects": [
["urn:ngsi-ld:Animal:pig001", "2021-04-26T08:55:56.100Z"],
... etc
]
}
},
… etc
]
● The response holds an array of
attribute value-time stamp pairs for each
observed reading.
● Properties are held in values arrays,
Relationships use objects
15. NGSI-LD Language Maps
14
{
"id": "urn:ngsi-ld:Vehicle:A4567",
"type": "Vehicle",
"brandName": {
"type": "Property",
"value": "Mercedes"
},
"street": {
"type": "LanguageProperty",
"languageMap": {
"fr": "Grand Place",
"nl": "Grote Markt"
}
},
"isParked": {
"type": "Relationship",
"object": "urn:ngsi-
ld:OffStreetParking:Downtown1",
"observedAt": "2017-07-29T12:00:04Z",
"providedBy": {
"type": "Relationship",
"object": "urn:ngsi-ld:Person:Bob"
}
}
}
NGSI-LD inherits concepts from JSON-LD
▪ NGSI-LD Entity id and Relationship object is
defined as a JSON-LD @id
@id is used to uniquely identify node objects that
are being described in the JSON-LD document
▪ NGSI-LD Entity type is defined as a JSON-LD
@type
@type is used to set the type of a node or the datatype of a
typed value
▪ NGSI-LD value is defined as an JSON-LD @value
@value is used to specify the data that is associated with
a particular property in the graph
JSON-LD also defines @language- used to specify the
language for a particular string value
▪ Each Property languageMap is defined as a JSON-
LD @language used for multi-language support of
simple string values
Park a car on the Street known as Grand
Place in French and Grote Markt in Dutch
16. lang follows the same rules as the Accept-Language Header
▪ lang="en" - English only
▪ lang="fr-CH,fr" - Either Swiss French or French
▪ lang="*" - Wildcard
▪ lang="fr-CH,fr;q=0.9,en;q=0.8,*;q=0.5"- Quality value ranking
Swiss French or French with no ranked preference, fallback to English as a second choice and finally fallback to
any other supported language.
Which street is urn:ngsi-ld:Vehicle:A4567 parked on? - return the name in French
Language Maps attributes can be retrieved as a value in a single language
using the lang parameter
15
curl -L -g -X GET 'http://localhost:1026/ngsi-ld/v1/entities/urn:ngsi-
ld:Vehicle:A4567'
-d 'attrs=street'
-d 'lang=fr'
-H 'Link: <http://.../path-to-my-public-server/ngsi-context.jsonld>;
rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"'
-H 'Accept: application/json'
{
"id": "urn:ngsi-ld:Vehicle:A4567",
"type": "Vehicle",
"street": {
"type": "Property",
"value": "Grand Place",
"lang": "fr",
}
}
17. Language Maps are limited to simple strings
16
{
"id": "urn:ngsi-ld:Event:bonjourLeMonde",
"type": "Event",
"name": {"type": "Property", "value": "Bonjour le Monde"
},
"description": {
"type": "Property",
"value": "«Bonjour le monde» sont les mots traditionnellement
écrits par un programme informatique simple"
},
"inLanguage": {"type": "Property", "value": "fr"},
"sameAs": [
{
"type": "Relationship", "datasetId": "urn:ngsi-ld:Relationship:1",
"object": "urn:ngsi-ld:Event:helloWorld",
"inLanguage": {"type": "Property", "value": "en"}
},
{
"type": "Relationship", "datasetId": "urn:ngsi-ld:Relationship:2",
"object": "urn:ngsi-ld:Event:halloWelt",
"inLanguage": {"type": "Property","value": "de"}
}
]
}
Well-defined properties, which already
have semantic meaning can be used
to internationalize complex entities
▪ schema.org/inLanguage
▪ schema.org/sameAs
This helps to keep the underlying data
models simple and facilitates reuse,
since not all data model users will need
internationalization
18. Expansion and Compaction
17
function translateRequest(req, res) {
const headers = req.headers;
headers.Accept = 'application/json';
const options = {
url: BASE_PATH + req.path,
method: req.method,
headers,
qs: req.query,
json: true
};
request(options)
.then(async function (cbResponse) {
cbResponse['@context'] = coreContext;
const expanded = await jsonld.expand(cbResponse);
const compacted = await jsonld.compact(expanded, alternate);
delete compacted['@context'];
return res.send(compacted);
})
.catch(function (err) {
return res.send(err);
});
}
Since NGSI-LD is an extended subset
of JSON-LD, you can use standard
JSON-LD libraries to perform
expansion and compaction operations.
Expansion and compaction can
operate on normalized or key-values
payloads
This could be used to support @vocab
elements as properties
Be careful, the resultant payload is
usually not valid NGSI-LD
20. Context data is data for exchange. To facilitate data interchange, strings are always Unicode Strings,
Dates are always ISO 8601 dates etc. Data models shouldn’t hold additional formats unnecessarily. The
context provider and/or the receiver should be able to manipulate the payload themselves if necessary.
● Display opening and closing hours in French
● Sort street names in an accentless fashion in Spanish
● Accept alternate spellings (e.g. “ö” = “oe”) in German
If absolutely necessary use metadata properties-of-properties to describe and query the context data
new Intl.DateTimeFormat('new Intl.DateTimeFormat('fr-FR', { dateStyle: 'full', timeStyle: 'long' }).format(date)
fr-FR', { dateStyle: 'full', timeStyle: 'long' }).format(date)
Natural Language Collation Support
19
str.normalize("NFD").replace(/[u0300-u036f]/g,
"").toLower()
or what context-brokers don’t do directly
new Intl.DateTimeFormat('fr-FR', { dateStyle: 'full', timeStyle: 'long' }).format(date)
/ngsi-ld/v1/entities/?type=Building&q=name.collate==%22schoene%20gruesse%22
str.toLower().replace('ö','oe').replace('ä','ae').replace('ü','ue').replace('ß','ss'
)
21. Context Entities hold a snapshot of the state of an entity representing a thing in the real world:
So how to:
● Store Images when there is no BLOB type
● Create short term predictions
● Create medium term predictions
Answer: don’t use a context broker for this. Use links to data storage, databases, actuations of external
services or chron-jobs where relevant. The real work is done by other microservices.
Remember : Context data is just data.
More context broker anti-patterns
20
or what context-brokers don’t do at all
22. When navigating the knowledge graph, only retrieve what
you really need:
● type - see also /types endpoint
● attrs - should be identifiable from the data model
● id - only guaranteed within a broker federation -
consider using an externally defined legalId Property
(or equivalent in use in your domain)
Each of these parameters can take a comma separated list. The short names for type and attrs
are defined using the @context
● Use simple JSON keyValues to minimize payloads internally
● Use full NGSI-LD normalized when initiating data exchange between clients
Filtering entity queries
21
let productsList = await ngsiLD.listEntities(
{
type: 'Shelf',
options: 'keyValues',
attrs: 'stocks,numberOfItems',
id: furniture.join(',')
},
headers
);
24. ● geometry - any supported GeoJSON type
● coordinates
● georel
○ near;maxDistance
○ near;minDistance
○ within
○ contains
○ intersects
○ equals
○ disjoint
○ overlaps
● geoproperty - Optional default is
location
The geoQ parameters
23
?georel=near;maxDistance==2000
&geometry=Point
&coordinates=[8,40]
&geoproperty=observationSpace
?georel=within&
geometry=Polygon&
coordinates=[[[100.0,0.0],[101.0,0.0],
[101.0,1.0],[100.0,1.0],[100.0,0.0]]]&
geoproperty=location
25. ● timeAt - any DateTime
● endTimeAt - any DateTime
● timerel
○ before
○ after
○ between
● timeproperty - Optional default is
observedAt
The temporalQ parameters
24
?timerel=before&
timeAt=2020-04-13T14:20:00Z&
timeproperty=modifiedAt
?timerel=between&
timeAt=2021-04-26T09:00:00Z&
endTimeAt=2021-05-21T14:40:00Z&
timeproperty=observedAt
26. Imagine the following scenario:
● A farm has Pigs and Cows tracked with Animal Collars
● The veterinary practice holds status records for the same Pigs and Cows
● A weather service can provide detailed weather conditions for locations on the farm
Who are the data providers?
What data does the farmer own/purchase?
Which common data models should be used?
How to ensure data from other sources refers to the correct entity?
Connecting Data Providers
25
27. ● Animal Data Model
https://github.com/smart-data-models/dataModel.Agrifood/tree/master/Animal
● Field Data Model
https://github.com/smart-data-models/dataModel.Agrifood/tree/master/AgriParcel
● Animal Collar Data Model
https://github.com/smart-data-models/dataModel.Device/tree/master/Device
● Weather Observed Data Model
https://github.com/smart-data-models/dataModel.Weather/tree/master/WeatherObserved
Farmer and Vet share Animal, WeatherObserved is used by the WeatherService
Data Models
26
28. 27
{
"@context": "https://..path-to-context/ngsi-context.jsonld",
"id": "urn:ngsi-ld:Animal:cow006",
"type": "Animal",
"species": {"type": "Property", "value": "dairy cattle"},
"name": {"type": "Property", "value": "Twilight" },
"sex": {"type": "Property", "value": "female"},
"phenologicalCondition": {"type": "Property", "value": "femaleAdult"},
"reproductiveCondition": {"type": "Property", "value": "active"},
"legalID": {"type": "Property", "value": "F-cow006-Twilight" },
"heartRate": {
"type": "Property", "value": 52, "unitCode": "5K",
"observedAt": "2021-05-03T09:06:51.051Z",
"providedBy": {
"type": "Relationship",
"object": "urn:ngsi-ld:Device:cowCollar006"
}
},
"locatedAt": {"type": "Relationship", "object": "urn:ngsi-ld:AgriParcel:field001",
"weatherConditions": {
"weatherType": "Raining",
"temperature": 25,
... etc
}
},
"location": {
"type": "GeoProperty", "value": {"type": "Point", "coordinates": [13.41, 52.47]},
"observedAt": "2021-05-03T09:06:51.051Z",
"providedBy": {
"type": "Relationship",
"object": "urn:ngsi-ld:Device:cowCollar006"
}
}
}
Following the standard Animal
model:
id":"urn:ngsi-ld:Animal:cow006"
is unique to the Farmer’s system,
but not a globally shared identifier.
● legalId is a globally shared
between Farmer and Vet
● phenologicalCondition and
reproductiveCondition are
provided by Vet
● weatherConditions is a
property-of-a relationship copied
here for convenience. The
AgriParcel entity must hold
sufficient information to be able to
request the weather conditions.
29. 28
curl -L -X POST 'http://localhost:1026/ngsi-ld/v1/subscriptions/'
-H 'Content-Type: application/ld+json'
-H 'NGSILD-Tenant: openiot'
--data-raw '{
"description": "Notify me of Veterinary Requests",
"type": "Subscription",
"entities": [{"type": "Animal"}],
"watchedAttributes": ["filling"],
"notification": {
"attributes": ["legalId", "refreshVetData"],
"format": "keyValues",
"endpoint": {
"uri": "http://i4trust-app/veterinary-practice",
"accept": "application/json"
}
},
"@context": "https://..path-to-context/ngsi-context.jsonld"
}'
Subscribe to changes on an attribute to trigger
a refresh of data.
Ensure all relevant data is passed to the
subscription then make a GET request to the
Vet’s context broker.
Additional Business logic to manipulate
response (e.g. expansion/compaction) and
upsert the result back into the Farmer’s context
broker.
Option 1 - Provide a common agreed identifier such as
legalId
curl -L -X PATCH 'http://localhost:1026/ngsi-ld/v1/entities/urn:ngsi-ld:Animal:cow006/attrs/refreshVetData'
-H 'NGSILD-Tenant: openiot'
-H 'Content-Type: application/json'
-H 'Link: <https://..path-to-context/ngsi-context.jsonld>; rel="http://www.w3.org/ns/json-ld#context";
type="application/ld+json"'
-d '{ "type": "Property", "value": "phenologicalCondition, reproductiveCondition"}'
30. 29
{
"@context": "https://..path-to-context/ngsi-context.jsonld",
"id": "urn:ngsi-ld:AgriParcel:field001",
"type": "AgriParcel",
"location": { "type": "GeoProperty", "value": {
"type": "Polygon",
"coordinates": [[[100, 0], [101, 0],
[101, 1], [100, 1], [100, 0]]]
}
},
"area": { "type": "Property", "value": 200},
"description": { "type": "Property", "value": "Pasture”},
"category": { "type": "Property", "value": "grassland"},
"relatedSource": { "type": "Property", "value": [
{
"application": "urn:ngsi-ld:AgriApp:weather001",
"applicationEntityId": "app:ExternalWeatherStation"
}
]
},
"weatherConditions": { "type": "Property", "value": {
"weatherType": "Raining",
"temperature": 25,
... etc
}
"observedAt": "2021-05-03T09:06:51.051Z",
"providedBy": {
"type": "Relationship",
"object": "urn:ngsi-ld:AgriApp:weather001"
}
}
}
Following the standard AgriParcel model:
id":"urn:ngsi-ld:AgriParcel:field001" is
unique to the Farmer’s system, but not a
globally shared identifier.
The relatedSource attribute holds the
Weather Station identifier within the external
System
Additional weatherConditions attribute
within AgriParcel. This is able to hold
additional information which is not required on
each of the Animal entities.
Option 2 - use relatedSource
for linking to External Applications
31. 30
curl -L -X PATCH 'http://localhost:4041/ngsi-ld/v1/entities/urn:ngsi-ld:AgriApp:weather001/attrs/update'
-H 'NGSILD-Tenant: openiot'
-H 'Content-Type: application/json'
-H 'Link: <https://..path-to-context/ngsi-context.jsonld>; rel="http://www.w3.org/ns/json-ld#context";
type="application/ld+json"'
-d '{ "type": "Property", "value": " " }'
Subscribe to changes on an attribute to trigger
a refresh of data.
Ensure all relevant data is passed to the
subscription then make a GET request to the
Weather provider’s context broker.
Additional Business logic to cascade batch
upsert the result back into the Farmer’s context
broker AgriParcel and Animal entities
Connecting to External Data Providers
curl -L -X POST 'http://localhost:1026/ngsi-ld/v1/subscriptions/'
-H 'Content-Type: application/ld+json'
-H 'NGSILD-Tenant: openiot'
--data-raw '{
"description": "Notify me of Weather Requests",
"type": "Subscription",
"entities": [{"type": "Weather"}],
"watchedAttributes": ["update"],
"notification": {
"format": "keyValues",
"endpoint": {
"uri": "http://i4trust-app/weather-provider",
"accept": "application/json"
}
},
"@context": "https://..path-to-context/ngsi-context.jsonld"
}'
32. Strictly speaking, Animal shouldn’t have weatherConditions at all.
You can navigate the knowledge graph based on the locatedAt relationship
But what if you want to determine do cows lie down in the rain?
● Each cow can be moved to a separate field at different times.
● Each field could experience different weather conditions.
The data may be duplicated for ease of calculations but:
● More data storage required
● Potential reduction in interoperability - reuse common attribute names.
Usable or Ontologically Correct?
31
33. ● A Push Model is typically used by Devices connected IoT Agents
POST /ngsi-ld/v1/entityOperations/upsert/
● Registration may be used for either federated environments or actuations but not both
● Subscriptions can also be used for actuation
Note that true federation implies a greater degree of trust than i4Trust
Upsert, Registration or Subscription
32