An experience on empirical research about rdf stream

Dipartimento di
Elettronica, Informazione e
Bioingegneria
An Experience on Empirical
Research about RDF Stream
Processing
Daniele Dell’Aglio – daniele.dellaglio@polimi.it
Joint work with: Jean-Paul Calbimonte, Marco Balduini, Oscar Corcho
and Emanuele Della Valle

Dipartimento di Elettronica, Informazione
e Bioingegneria
RDF Stream Processing in a nutshell
 Continuous queries over RDF streams - infinite
sequences of time-stamped RDF statements (RDF
streams)
 Bring together DSMS/CEP and Semantic Web research
fields
 Several prototypes – with similar models – are available
today
 Trend on evaluation and comparison of the existing
systems
26 May 2014 - EMPIRICAL@ESWC2014
DanieleDell'Aglio-ExperimentalresearchaboutRSPs
2

e Bioingegneria
The CQL model for RSPs
 Transform a set of mappings in another set of
mappings
 SPARQL 1.0/1.1 queries
 Each set of mapping produced by the R2R operator
is transformed and appended to the output
stream
 Operators: RStream, DStream, IStream
 Converts the infinite stream of RDF elements in a
finite set of mappings
 The window operators: time-based, tuple-based, …
S2R
operator
R2R
operator
R2S
operator
Input stream
Output stream
3

e Bioingegneria
R2R operator
S2R - Time-based sliding window
S3
S4 S5
S6
S7
S8
S9 S10
S11
S12
S
S1
S2
W(ω,β)
β
ω
t
widthslide
4

e Bioingegneria
Implementations (oversimplified!)
 C-SPARQL
– RDF Store + Stream processor


RDF Store
Stream
processor
Continuous
query
continuous
results
translator
5

e Bioingegneria
 C-SPARQL
 CQELS:
– Implemented from scratch. Focus on performance

RDF Store
Stream
processor
Continuous
query
continuous
results
Native RSP
Continuous
query
continuous
results
translator
5

e Bioingegneria
 C-SPARQL
 CQELS:
– Implemented from scratch. Focus on performance
 SPARQLstream:
– Ontology-based stream query answering
RDF Store
Stream
processor
Continuous
query
continuous
results
Native RSP
Continuous
query
continuous
results
translator
DSMS/CEP
Continuous
query
continuous
results
rewriter
R2RML mappings
5

e Bioingegneria
Same inputs, different outputs…
 And the continuous
query:
– Where are Alice and
Bob, when they are
together?
– With a tumbling
window W(ω=β=5)
Execution 1° answer 2° answer
1 :hall [6] :kitchen [11]
4 - [7] - [12]
S1 S2 S3 S4S
t3 6 91
:alice :isIn :hall
:bob :isIn :hall
:alice :isIn :kitchen
:bob :isIn :kitchen
width
slide
 After 4 executions:
 Let’s consider the following stream:
8

e Bioingegneria
The first hypothesis
 All the three systems show similar behaviours
 Intuition: there are one or more parameters that are not
taken into account by the model
 As consequence, the implementations can output
different correct answers
9

e Bioingegneria
The first hypothesis
 HP1: it is possible to have a unique correct answer if we
can control the time instant on which the sliding window
operator starts to work (t0)
S1 S2 S3 S4S
t3 6 91
:bob :isIn :hall :bob :isIn :kitchen
t0=0
:alice :isIn :hall :alice :isIn :kitchen
t0=1
t0=2
10

e Bioingegneria
The experiment
 We work on the difference between the time
instant on which the stream starts (ts) and the
query registration time (tq)
– At each execution, we check the result
– We estimated the delay between tq and t0
tq
ts
 Black box approach
– we work on inputs/outputs
– the source code of all the systems
RSP
11
t0

e Bioingegneria
Observation and explanation
 As result, for each system
– We identified the value of the t0 parameter
– We are able to produce the different results for each t0
value
 Is it enough to claim that hypothesis 1 holds?
Exec 1° answer 2° answer
4 - [7] - [12]
Window 1° answer 2° answer
t0=0 :hall [5] :kitchen [10]
t0=1 :hall [6] :kitchen [11]
t0=2 - [7] - [12]
12

e Bioingegneria
Some consideration on the experiment
 Comparison:
– We ran the experiment multiple times to collect
instances and check them
 Reproducibility: can other researchers reproduce the
experiment?
– We released both the code and the data used for the
experiment (see
http://streamreasoning.org/Benchmarks/)
 Repeatability: is the result universally valid?
– We changed inputs (streams and queries) and
OS/JVM to verify if the hypothesis holds
– We repeated the experiment with different
implementations (C-SPARQL, CQELS, etc.)
13

e Bioingegneria
Something more on repeatability…
 We made some assumptions on the setting
14
S2R
R2R R2SS2R
S2R
From single
to multi
window
From single to
multi stream
Reasoning
q2
Static
knowledge
Multiple
queries

e Bioingegneria
 As “side effect” of the first experiment, we
discovered that results of different systems are
not the same:
 Intuition: t0 is not the only parameter our model
lacks
A more complex problem…
4 - [7] - [12]
2 No answers
4 No answers
C-SPARQL CQELS
15

e Bioingegneria
R2R operator
The SECRET framework
S3
S4 S5
S6
S7
S8
S9 S10
S11
S12
S
S1
S2
W(ω,β)
β
ω
t0: When does the
window start?
(internal window
param)
TICK: When are
data stream
elements added to
the window?
Triple-based vs
graph-based
REPORT: When is the window content
made available to the R2R operator?
Non-empty content, Content-change,
Window-close, Periodic
t
16

e Bioingegneria
SECRET and RSPs
 HP2: given an input stream, a query, the value of t0 and
description of the RSP w.r.t. SECRET, we can determine
the answer that will be provided by the system
 To investigate it, we built a software that evaluates in
batch the answer and matches it with the RSP one
17

e Bioingegneria
Observation and analysis
 We prepared a set of seven
queries (to stress different part of
the sliding window)
 We run each query multiple times
 Most of the times, we can foresee the
answer that will be provided
CQELS
C-SPARQL
SPARQLstream
Q1
Q2
Q3
Q4
Q5
Q6
Q7
18

e Bioingegneria
Observation and analysis
 We investigated the observations where there is
not a match, and we discovered that they were
errors in the implementations, such as:
– Initialization
– Slide parameter
– Window contents
– Internal timestamp management
 Conclusion: HP2 seems to be valid in the
considered setting
19

e Bioingegneria
CSR-bench
 The main outcome of our experience is CSR-bench, an
extension of the CSR benchmark
– More info at http://www.w3.org/wiki/CSRBench
 Two main components:
– A common model for the RDF stream processor
operational semantics
– An oracle (an automatic correctness validator),
available at https://github.com/dellaglio/csrbench-
oracle
– A test suite
20

e Bioingegneria
References
 Daniele Dell'Aglio, Marco Balduini, Emanuele Della Valle. On the need to
include functional testing in RDF stream engine benchmarks. 1st
International Workshop on Benchmarking RDF Systems (BeRSys2013)
 Daniele Dell'Aglio, Jean-Paul Calbimonte, Marco Balduini, Óscar Corcho,
Emanuele Della Valle: On Correctness in RDF Stream Processor
Benchmarking. International Semantic Web Conference (2) 2013: 326-342
 Barbieri, D.F., Braga, D., Ceri, S., Della Valle, E., Grossniklaus, M.: C-
SPARQL: A continuous query language for RDF data streams. IJSC 4(1)
(2010) 3–25
 Calbimonte, J.P., Jeung, H., Corcho, O., Aberer, K.: Enabling Query
Technologies for the Semantic Sensor Web. IJSWIS 8(1) (2012) 43–63
 Le-Phuoc, D., Dao-Tran, M., Xavier Parreira, J., Hauswirth, M.: A native and
adaptive approach for unified processing of linked streams and linked data.
In: ISWC. (2011) 370–388
21

e Bioingegneria
Thank you! Questions?
An Experience on Empirical Research about
RDF Stream Processing
Daniele Dell’Aglio
(DEIB, Politecnico di Milano)
daniele.dellaglio@polimi.it
22

An experience on empirical research about rdf stream

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a An experience on empirical research about rdf stream

Semelhante a An experience on empirical research about rdf stream (20)

Mais de Daniele Dell'Aglio

Mais de Daniele Dell'Aglio (20)

Último

Último (20)

An experience on empirical research about rdf stream