SlideShare uma empresa Scribd logo
1 de 23
Baixar para ler offline
BigData
Architectures
Daan Gerits
Dasos
Volume
IOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOIOOO
OIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOI
OIOIOIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOII
IOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIII
OIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIOII
OIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOIOOOOIOII

We already have that:
- NAS/SAN
- High Performance Computing
Variety

IOII IOIIIOIIIOII

IOII
IOII
IOII

IOII

IOII

We already have that:
- Meta-modeling
- NAS/SAN
Velocity

OIII
IOII
IOII OO
OIII
We already have that:
- Complex Event Processing
But do you have all of that in 1
platform?
But How??
Architectures

(Thx Nathan Marz!)
Analytical Big Data

Analysis Oriented
Optimize
Non-intrusive
Delta
Apps
Dashboards
Distributed
Database

Data
Sources
Ingestion
Engine

Enrich

Data
Systems
Delta
Impala,
Hive, ...

Apps
Dashboards

Distributed
Database

Data
Sources
Flume,
Sqoop,
Scribe, ...

MR, Pig,
Crunch,
Mahout, ...

MR, Pig,
Crunch, ...

Data
Systems
Delta

Analytical Big Data
architecture for enriching mostly
structured data with the goal to
optimize business processes.
Delta
Apps
Dashboards
Distributed
Database

Data
Sources
Ingestion
Engine

Enrich

Overload!

Data
Systems
Delta
Be
write-heavy
or
read-heavy
NOT both!
Operational Big Data
Focussed on Day-today business
Innovate
(Non-)intrusive
(Thx Nathan Marz!)
Lambda
Realtime
View A
Realtime
Processing

Apps

Realtime
View B

Dashboard

Realtime
View C

Data
Sources

Batch
View A
Fact
Store

Just In Time
Combiner

Batch
View B
Batch
View C

Reports
Lambda
Cassandra*
Storm

Apps

Cassandra*

Dashboard

Cassandra*
Custom
Code*

Data
Sources
ElephantDB
HDFS

ElephantDB
ElephantDB

Reports
Lambda

Operational Big Data
architecture for storing and processing

multi-structured and
immutable data with the goal to
Innovate business
Technologies to use

Pick your
stack!
Advice
Pilots, PoC, PoT, … do them!
Be pragmatic, start skinny
In Belgium: Variety > Volume
Be prepared to pivot on technologies
Questions?
Thoughts?
Ideas?
Disagreements?
...

daan.gerits@dasos.be
www.dasos.be
@daangerits

All images are used merely for illustrational means. In no
way was it my purpose to violate any rights by using
BigData
Architectures
Backup
Slides
Variety

Velocity

Volume
Lambda
Multistructured

Unstructured

Restructured

Mais conteúdo relacionado

Destaque

Getting more out of your big data
Getting more out of your big dataGetting more out of your big data
Getting more out of your big data
Nathan Bijnens
 

Destaque (8)

Real time Analytics with Apache Kafka and Apache Spark
Real time Analytics with Apache Kafka and Apache SparkReal time Analytics with Apache Kafka and Apache Spark
Real time Analytics with Apache Kafka and Apache Spark
 
Getting more out of your big data
Getting more out of your big dataGetting more out of your big data
Getting more out of your big data
 
Apache storm vs. Spark Streaming
Apache storm vs. Spark StreamingApache storm vs. Spark Streaming
Apache storm vs. Spark Streaming
 
Big Data
Big DataBig Data
Big Data
 
Kafka and Storm - event processing in realtime
Kafka and Storm - event processing in realtimeKafka and Storm - event processing in realtime
Kafka and Storm - event processing in realtime
 
Big Data & Hadoop Tutorial
Big Data & Hadoop TutorialBig Data & Hadoop Tutorial
Big Data & Hadoop Tutorial
 
Hadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm ArchitectureHadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm Architecture
 
Big data ppt
Big  data pptBig  data ppt
Big data ppt
 

Semelhante a Big data architectures

Semelhante a Big data architectures (20)

Big data - Apache Hadoop for Beginner's
Big data - Apache Hadoop for Beginner'sBig data - Apache Hadoop for Beginner's
Big data - Apache Hadoop for Beginner's
 
Hadoop Tutorial | What is Hadoop | Hadoop Project on Reddit | Edureka
Hadoop Tutorial | What is Hadoop | Hadoop Project on Reddit | EdurekaHadoop Tutorial | What is Hadoop | Hadoop Project on Reddit | Edureka
Hadoop Tutorial | What is Hadoop | Hadoop Project on Reddit | Edureka
 
Keedio Stack: BDaaS deployment for dummies by Alessio Comisso at Big Data Spa...
Keedio Stack: BDaaS deployment for dummies by Alessio Comisso at Big Data Spa...Keedio Stack: BDaaS deployment for dummies by Alessio Comisso at Big Data Spa...
Keedio Stack: BDaaS deployment for dummies by Alessio Comisso at Big Data Spa...
 
Introduction to Big Data by Manouj Bongirr
Introduction to Big Data by Manouj BongirrIntroduction to Big Data by Manouj Bongirr
Introduction to Big Data by Manouj Bongirr
 
IRJET- Systematic Review: Progression Study on BIG DATA articles
IRJET- Systematic Review: Progression Study on BIG DATA articlesIRJET- Systematic Review: Progression Study on BIG DATA articles
IRJET- Systematic Review: Progression Study on BIG DATA articles
 
Why Blancco ?
Why Blancco ?Why Blancco ?
Why Blancco ?
 
Big data introduction
Big data introductionBig data introduction
Big data introduction
 
Birst Cloud BI Data Sheet
Birst Cloud BI Data SheetBirst Cloud BI Data Sheet
Birst Cloud BI Data Sheet
 
Seagate Solution Brief Highlights axle ai Integration with Seagate Lyve Cloud
Seagate Solution Brief Highlights axle ai Integration with Seagate Lyve CloudSeagate Solution Brief Highlights axle ai Integration with Seagate Lyve Cloud
Seagate Solution Brief Highlights axle ai Integration with Seagate Lyve Cloud
 
Geode is Not a Cache, it's an Analytics Engine
Geode is Not a Cache, it's an Analytics EngineGeode is Not a Cache, it's an Analytics Engine
Geode is Not a Cache, it's an Analytics Engine
 
big data
big databig data
big data
 
BigData Analytics
BigData AnalyticsBigData Analytics
BigData Analytics
 
Comparing Scalable Predictive Analysis using Spark XGBoost Platforms
Comparing Scalable Predictive Analysis using Spark XGBoost PlatformsComparing Scalable Predictive Analysis using Spark XGBoost Platforms
Comparing Scalable Predictive Analysis using Spark XGBoost Platforms
 
IoT and BigData
IoT and BigDataIoT and BigData
IoT and BigData
 
Red Hat Middleware Solutions for SAP Systems
Red Hat Middleware Solutions for SAP SystemsRed Hat Middleware Solutions for SAP Systems
Red Hat Middleware Solutions for SAP Systems
 
Big Data: Its Characteristics And Architecture Capabilities
Big Data: Its Characteristics And Architecture CapabilitiesBig Data: Its Characteristics And Architecture Capabilities
Big Data: Its Characteristics And Architecture Capabilities
 
Copy Data Management & Storage Efficiency - Ravi Namboori
Copy Data Management & Storage Efficiency - Ravi NambooriCopy Data Management & Storage Efficiency - Ravi Namboori
Copy Data Management & Storage Efficiency - Ravi Namboori
 
T9
T9T9
T9
 
Big data insights with Red Hat JBoss Data Virtualization
Big data insights with Red Hat JBoss Data VirtualizationBig data insights with Red Hat JBoss Data Virtualization
Big data insights with Red Hat JBoss Data Virtualization
 
Building a Big Data Analytics Platform- Impetus White Paper
Building a Big Data Analytics Platform- Impetus White PaperBuilding a Big Data Analytics Platform- Impetus White Paper
Building a Big Data Analytics Platform- Impetus White Paper
 

Último

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Último (20)

DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu SubbuApidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 

Big data architectures