BDE concept and platform as presented during the 3rd BDE SC5 online hangout.

1. BIG DATA EUROPEBIG DATA EUROPE:
CONCEPT AND PLATFORM
NCSR Demokritos20 December 2016

2. Big Data Europe (CSA: 2015-17)
 Show societal value of Big Data
o Across all societal challenges addressed by Horizon 2020
 Lower barrier for using big data technologies
o Effort and resources to convert tools and workflows
o Skills and expertise
H l bli h d l h i Help establish data value chains
o Across languages, organizations and domains
31-janv.-17www.big-data-europe.eu

3. Consortium
NCSR
DEMOKRITOS

4. Stakeholder Engagement
Stakeholder engagement workshops:g g p
 Present the action and its
showcase deployments
 Raise awareness about BDE results
and what they mean for
stakeholders
 Collect requirements to drive
further development
31-janv.-17www.big-data-europe.eu

5. SocietalSoc e a
Challenges
31-janv.-17www.big-data-europe.eu

6. Architecture
 Big Data Integrator (BDI):
o The prototype developed by BDE
 Main points of the architecture Main points of the architecture
o Dockerization
Support layer including integrated UIo Support layer, including integrated UI
o Semantification layer
31-janv.-17www.big-data-europe.eu

7. Docker containers
 Docker offers lightweight virtualization
o Docker containers can be shared to be provisioned on different
Linux variations and versions
 Identical base sys
not required
 All BDI components:
Docker containers
31-janv.-17www.big-data-europe.eu
Docker containers

8. BDI components
 Processing and storage components
o Re-used existing docker containers where available
o Dockerized by BDE where not
o Ensured all can be provisioned through Docker Swarm
 Components by BDE:
o Support Layer
o Semantic Layer
31-janv.-17www.big-data-europe.eu

9. BDE Docker Containers
D t i HDFS C d 4 t Data serving: HDFS, Cassandra, 4store,
PostGIS, Strabon, Elastic Search, Hive,
SemagrowSemagrow
 Processing: Spark, Flink, Sansa
 Stream ingestion middleware: Flume,
Kafka
31-janv.-17www.big-data-europe.eu

10. Semantic layer tools
 BDE tooling for Semantic Data Lake:g
o Swagger: Semantics of RESTful APIs
o Semantic Analytics Stack (SANSA):y ( )
Distributed data processing for large-
scale RDF data
o Semagrow: SPARQL perspective over
Big Data stores
31-janv.-17www.big-data-europe.eu

11. Questions?
 BigDataEurope Web site:
https://www.big-data-europe.eu
 Thank you for your attention!
www.big-data-europe.eu