Enviar pesquisa
Carregar
Database ,18 Current Issues
•
Transferir como PPTX, PDF
•
2 gostaram
•
1,916 visualizações
A
Ali Usman
Seguir
Tecnologia
Denunciar
Compartilhar
Denunciar
Compartilhar
1 de 59
Baixar agora
Recomendados
Database , 15 Object DBMS
Database , 15 Object DBMS
Ali Usman
Database ,16 P2P
Database ,16 P2P
Ali Usman
Database , 17 Web
Database , 17 Web
Ali Usman
Database , 6 Query Introduction
Database , 6 Query Introduction
Ali Usman
Database , 5 Semantic
Database , 5 Semantic
Ali Usman
Database ,10 Transactions
Database ,10 Transactions
Ali Usman
141060753008 3715302
141060753008 3715302
ITM Universe - Vadodara
Practical MySQL
Practical MySQL
Indus Khaitan
Recomendados
Database , 15 Object DBMS
Database , 15 Object DBMS
Ali Usman
Database ,16 P2P
Database ,16 P2P
Ali Usman
Database , 17 Web
Database , 17 Web
Ali Usman
Database , 6 Query Introduction
Database , 6 Query Introduction
Ali Usman
Database , 5 Semantic
Database , 5 Semantic
Ali Usman
Database ,10 Transactions
Database ,10 Transactions
Ali Usman
141060753008 3715302
141060753008 3715302
ITM Universe - Vadodara
Practical MySQL
Practical MySQL
Indus Khaitan
MySQL Cluster page management (2014)
MySQL Cluster page management (2014)
Frazer Clement
Heterogenous Persistence
Heterogenous Persistence
Jervin Real
Challenges of Implementing an Advanced SQL Engine on Hadoop
Challenges of Implementing an Advanced SQL Engine on Hadoop
DataWorks Summit
In-Memory Logical Data Warehouse for accelerating Machine Learning Pipelines ...
In-Memory Logical Data Warehouse for accelerating Machine Learning Pipelines ...
Gianmario Spacagna
Pagereplacement algorithm(computional concept)
Pagereplacement algorithm(computional concept)
Siddhi Viradiya
9 virtual memory management
9 virtual memory management
Dr. Loganathan R
Page replacement
Page replacement
Davin Abraham
Transaction conccurency
Transaction conccurency
Esraa Farrag
Virtual memory and page replacement algorithm
Virtual memory and page replacement algorithm
Muhammad Mansoor Ul Haq
Case Study: A Multi-Source Time Variant Data warehouse
Case Study: A Multi-Source Time Variant Data warehouse
tarun kumar sharma
SHADOW PAGING and BUFFER MANAGEMENT
SHADOW PAGING and BUFFER MANAGEMENT
ramya marichamy
Introduction to mongo db by zain
Introduction to mongo db by zain
KenAndTea
NoSQL Evolution
NoSQL Evolution
Abdul Manaf
Caching in drupal
Caching in drupal
Vivek Panicker
40 demand paging
40 demand paging
myrajendra
Five Lessons in Distributed Databases
Five Lessons in Distributed Databases
jbellis
Page replacement alg
Page replacement alg
V.V.Vanniapermal College for Women
IMC Summit 2016 Breakout - Pandurang Naik - Demystifying In-Memory Data Grid,...
IMC Summit 2016 Breakout - Pandurang Naik - Demystifying In-Memory Data Grid,...
In-Memory Computing Summit
Scalability truths and serverless architectures
Scalability truths and serverless architectures
Regunath B
UPD_OP_SQL
UPD_OP_SQL
Valapet Badri
Database , 4 Data Integration
Database , 4 Data Integration
Ali Usman
Cisco Packet Tracer Overview
Cisco Packet Tracer Overview
Ali Usman
Mais conteúdo relacionado
Mais procurados
MySQL Cluster page management (2014)
MySQL Cluster page management (2014)
Frazer Clement
Heterogenous Persistence
Heterogenous Persistence
Jervin Real
Challenges of Implementing an Advanced SQL Engine on Hadoop
Challenges of Implementing an Advanced SQL Engine on Hadoop
DataWorks Summit
In-Memory Logical Data Warehouse for accelerating Machine Learning Pipelines ...
In-Memory Logical Data Warehouse for accelerating Machine Learning Pipelines ...
Gianmario Spacagna
Pagereplacement algorithm(computional concept)
Pagereplacement algorithm(computional concept)
Siddhi Viradiya
9 virtual memory management
9 virtual memory management
Dr. Loganathan R
Page replacement
Page replacement
Davin Abraham
Transaction conccurency
Transaction conccurency
Esraa Farrag
Virtual memory and page replacement algorithm
Virtual memory and page replacement algorithm
Muhammad Mansoor Ul Haq
Case Study: A Multi-Source Time Variant Data warehouse
Case Study: A Multi-Source Time Variant Data warehouse
tarun kumar sharma
SHADOW PAGING and BUFFER MANAGEMENT
SHADOW PAGING and BUFFER MANAGEMENT
ramya marichamy
Introduction to mongo db by zain
Introduction to mongo db by zain
KenAndTea
NoSQL Evolution
NoSQL Evolution
Abdul Manaf
Caching in drupal
Caching in drupal
Vivek Panicker
40 demand paging
40 demand paging
myrajendra
Five Lessons in Distributed Databases
Five Lessons in Distributed Databases
jbellis
Page replacement alg
Page replacement alg
V.V.Vanniapermal College for Women
IMC Summit 2016 Breakout - Pandurang Naik - Demystifying In-Memory Data Grid,...
IMC Summit 2016 Breakout - Pandurang Naik - Demystifying In-Memory Data Grid,...
In-Memory Computing Summit
Scalability truths and serverless architectures
Scalability truths and serverless architectures
Regunath B
UPD_OP_SQL
UPD_OP_SQL
Valapet Badri
Mais procurados
(20)
MySQL Cluster page management (2014)
MySQL Cluster page management (2014)
Heterogenous Persistence
Heterogenous Persistence
Challenges of Implementing an Advanced SQL Engine on Hadoop
Challenges of Implementing an Advanced SQL Engine on Hadoop
In-Memory Logical Data Warehouse for accelerating Machine Learning Pipelines ...
In-Memory Logical Data Warehouse for accelerating Machine Learning Pipelines ...
Pagereplacement algorithm(computional concept)
Pagereplacement algorithm(computional concept)
9 virtual memory management
9 virtual memory management
Page replacement
Page replacement
Transaction conccurency
Transaction conccurency
Virtual memory and page replacement algorithm
Virtual memory and page replacement algorithm
Case Study: A Multi-Source Time Variant Data warehouse
Case Study: A Multi-Source Time Variant Data warehouse
SHADOW PAGING and BUFFER MANAGEMENT
SHADOW PAGING and BUFFER MANAGEMENT
Introduction to mongo db by zain
Introduction to mongo db by zain
NoSQL Evolution
NoSQL Evolution
Caching in drupal
Caching in drupal
40 demand paging
40 demand paging
Five Lessons in Distributed Databases
Five Lessons in Distributed Databases
Page replacement alg
Page replacement alg
IMC Summit 2016 Breakout - Pandurang Naik - Demystifying In-Memory Data Grid,...
IMC Summit 2016 Breakout - Pandurang Naik - Demystifying In-Memory Data Grid,...
Scalability truths and serverless architectures
Scalability truths and serverless architectures
UPD_OP_SQL
UPD_OP_SQL
Destaque
Database , 4 Data Integration
Database , 4 Data Integration
Ali Usman
Cisco Packet Tracer Overview
Cisco Packet Tracer Overview
Ali Usman
Lec4
Lec4
Saad Gabr
Processor Specifications
Processor Specifications
Ali Usman
Modul 04 ta1_ metodologi penelitian
Modul 04 ta1_ metodologi penelitian
Fokgusta
Media ajarelektronik
Media ajarelektronik
Fokgusta
Model pembelajaran yang efektif
Model pembelajaran yang efektif
Fokgusta
Carpooling management system ver 2.0
Carpooling management system ver 2.0
Martinus Benjamin
Model pemb-2005
Model pemb-2005
Fokgusta
Kalimat aktifpasif-kalimat-langsung-taklangsung
Kalimat aktifpasif-kalimat-langsung-taklangsung
Fokgusta
Carpool db diagram
Carpool db diagram
Larry Jennings
Kata kajian-dan-kata-populer
Kata kajian-dan-kata-populer
Fokgusta
Carpooling Network Presentation
Carpooling Network Presentation
sineadc96
1.studi tentang organisasi
1.studi tentang organisasi
Fokgusta
Gsm (Part 1)
Gsm (Part 1)
Ali Usman
Muslim Contributions in Astronomy
Muslim Contributions in Astronomy
Ali Usman
Processor Specifications
Processor Specifications
Ali Usman
Database ,7 query localization
Database ,7 query localization
Ali Usman
Database, 3 Distribution Design
Database, 3 Distribution Design
Ali Usman
Database ,11 Concurrency Control
Database ,11 Concurrency Control
Ali Usman
Destaque
(20)
Database , 4 Data Integration
Database , 4 Data Integration
Cisco Packet Tracer Overview
Cisco Packet Tracer Overview
Lec4
Lec4
Processor Specifications
Processor Specifications
Modul 04 ta1_ metodologi penelitian
Modul 04 ta1_ metodologi penelitian
Media ajarelektronik
Media ajarelektronik
Model pembelajaran yang efektif
Model pembelajaran yang efektif
Carpooling management system ver 2.0
Carpooling management system ver 2.0
Model pemb-2005
Model pemb-2005
Kalimat aktifpasif-kalimat-langsung-taklangsung
Kalimat aktifpasif-kalimat-langsung-taklangsung
Carpool db diagram
Carpool db diagram
Kata kajian-dan-kata-populer
Kata kajian-dan-kata-populer
Carpooling Network Presentation
Carpooling Network Presentation
1.studi tentang organisasi
1.studi tentang organisasi
Gsm (Part 1)
Gsm (Part 1)
Muslim Contributions in Astronomy
Muslim Contributions in Astronomy
Processor Specifications
Processor Specifications
Database ,7 query localization
Database ,7 query localization
Database, 3 Distribution Design
Database, 3 Distribution Design
Database ,11 Concurrency Control
Database ,11 Concurrency Control
Semelhante a Database ,18 Current Issues
6-Query_Intro (5).pdf
6-Query_Intro (5).pdf
JaveriaShoaib4
Database ,14 Parallel DBMS
Database ,14 Parallel DBMS
Ali Usman
Database , 12 Reliability
Database , 12 Reliability
Ali Usman
JUG SF - Introduction to data streaming
JUG SF - Introduction to data streaming
Nicolas Fränkel
CDC patterns in Apache Kafka®
CDC patterns in Apache Kafka®
confluent
1 introduction
1 introduction
Amrit Kaur
SCALE - Stream processing and Open Data, a match made in Heaven
SCALE - Stream processing and Open Data, a match made in Heaven
Nicolas Fränkel
Apache Big Data 2016: Next Gen Big Data Analytics with Apache Apex
Apache Big Data 2016: Next Gen Big Data Analytics with Apache Apex
Apache Apex
Gcp dataflow
Gcp dataflow
Igor Roiter
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
In-Memory Computing Summit
BruJUG - Introduction to data streaming
BruJUG - Introduction to data streaming
Nicolas Fränkel
WaJUG - Introduction to data streaming
WaJUG - Introduction to data streaming
Nicolas Fränkel
Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...
Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...
DataStax
2014 01-23-eranea-apalia-private-cloud
2014 01-23-eranea-apalia-private-cloud
Didier Durand
Streaming analytics state of the art
Streaming analytics state of the art
Stavros Kontopoulos
Database , 13 Replication
Database , 13 Replication
Ali Usman
Data Stream Management
Data Stream Management
k_tauhid
SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...
SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...
ScyllaDB
Scaling systems using change propagation across data stores
Scaling systems using change propagation across data stores
Jagadeesh Huliyar
Real-Time Streaming: Move IMS Data to Your Cloud Data Warehouse
Real-Time Streaming: Move IMS Data to Your Cloud Data Warehouse
Precisely
Semelhante a Database ,18 Current Issues
(20)
6-Query_Intro (5).pdf
6-Query_Intro (5).pdf
Database ,14 Parallel DBMS
Database ,14 Parallel DBMS
Database , 12 Reliability
Database , 12 Reliability
JUG SF - Introduction to data streaming
JUG SF - Introduction to data streaming
CDC patterns in Apache Kafka®
CDC patterns in Apache Kafka®
1 introduction
1 introduction
SCALE - Stream processing and Open Data, a match made in Heaven
SCALE - Stream processing and Open Data, a match made in Heaven
Apache Big Data 2016: Next Gen Big Data Analytics with Apache Apex
Apache Big Data 2016: Next Gen Big Data Analytics with Apache Apex
Gcp dataflow
Gcp dataflow
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
BruJUG - Introduction to data streaming
BruJUG - Introduction to data streaming
WaJUG - Introduction to data streaming
WaJUG - Introduction to data streaming
Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...
Webinar: Dyn + DataStax - helping companies deliver exceptional end-user expe...
2014 01-23-eranea-apalia-private-cloud
2014 01-23-eranea-apalia-private-cloud
Streaming analytics state of the art
Streaming analytics state of the art
Database , 13 Replication
Database , 13 Replication
Data Stream Management
Data Stream Management
SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...
SAS Institute on Changing All Four Tires While Driving an AdTech Engine at Fu...
Scaling systems using change propagation across data stores
Scaling systems using change propagation across data stores
Real-Time Streaming: Move IMS Data to Your Cloud Data Warehouse
Real-Time Streaming: Move IMS Data to Your Cloud Data Warehouse
Mais de Ali Usman
Islamic Arts and Architecture
Islamic Arts and Architecture
Ali Usman
Database , 8 Query Optimization
Database , 8 Query Optimization
Ali Usman
Database ,2 Background
Database ,2 Background
Ali Usman
Database , 1 Introduction
Database , 1 Introduction
Ali Usman
Fifty Year Of Microprocessor
Fifty Year Of Microprocessor
Ali Usman
Discrete Structures lecture 2
Discrete Structures lecture 2
Ali Usman
Discrete Structures. Lecture 1
Discrete Structures. Lecture 1
Ali Usman
Muslim Contributions in Medicine-Geography-Astronomy
Muslim Contributions in Medicine-Geography-Astronomy
Ali Usman
Muslim Contributions in Geography
Muslim Contributions in Geography
Ali Usman
Ptcl modem (user manual)
Ptcl modem (user manual)
Ali Usman
Nimat-ul-ALLAH shah wali
Nimat-ul-ALLAH shah wali
Ali Usman
Muslim Contributions in Mathematics
Muslim Contributions in Mathematics
Ali Usman
Osi protocols
Osi protocols
Ali Usman
Mais de Ali Usman
(13)
Islamic Arts and Architecture
Islamic Arts and Architecture
Database , 8 Query Optimization
Database , 8 Query Optimization
Database ,2 Background
Database ,2 Background
Database , 1 Introduction
Database , 1 Introduction
Fifty Year Of Microprocessor
Fifty Year Of Microprocessor
Discrete Structures lecture 2
Discrete Structures lecture 2
Discrete Structures. Lecture 1
Discrete Structures. Lecture 1
Muslim Contributions in Medicine-Geography-Astronomy
Muslim Contributions in Medicine-Geography-Astronomy
Muslim Contributions in Geography
Muslim Contributions in Geography
Ptcl modem (user manual)
Ptcl modem (user manual)
Nimat-ul-ALLAH shah wali
Nimat-ul-ALLAH shah wali
Muslim Contributions in Mathematics
Muslim Contributions in Mathematics
Osi protocols
Osi protocols
Último
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
Khem
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
sudhanshuwaghmare1
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Igalia
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Drew Madelung
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
The Digital Insurer
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
Results
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
HampshireHUG
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
Principled Technologies
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
Pixlogix Infotech
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
naman860154
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Rafal Los
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Malak Abu Hammad
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
Delhi Call girls
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
Delhi Call girls
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
Delhi Call girls
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
wesley chun
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
Delhi Call girls
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc
Último
(20)
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
Database ,18 Current Issues
1.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/1 Outline • Introduction • Background • Distributed Database Design • Database Integration • Semantic Data Control • Distributed Query Processing • Multidatabase Query Processing • Distributed Transaction Management • Data Replication • Parallel Database Systems • Distributed Object DBMS • Peer-to-Peer Data Management • Web Data Management • Current Issues ➡ Data Stream Management ➡ Cloud Data Management
2.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/2 Inputs & Outputs • Inputs: One or more sources generate data continuously, in real time, and in fixed order ➡ Sensor networks – weather monitoring, road traffic monitoring, motion detection ➡ Web data – financial trading, news/sports tickers ➡ Scientific data – experiments in particle physics ➡ Transaction logs – telecom, point-of-sale purchases ➡ Network traffic analysis (IP packet headers) – bandwidth usage, routing decisions, security • Outputs: Want to collect and process the data on-line ➡ Environment monitoring ➡ Location monitoring ➡ Correlations across stock prices ➡ Denial-of-service attack detection • Up-to-date answers generated continuously or periodically
3.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/3 Traditional DBMS Persistent data - stored until deleted by user or application Transient queries - issued once, then forgotten
4.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/4 Data Stream Management System (DSMS) Persistent queries - generate up-to-date answers as time goes on Transient data - deleted as window slides forward
5.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/5 DSMSs – Novel Problems • Push-based (data-driven), rather than pull-based (query-driven) computation model ➡ New data arrive continuously and must be processed ➡ Query plans require buffers, queues, and scheduling mechanisms ➡ Query operators must be non-blocking ➡ Must adapt to changing system conditions throughout the lifetime of a query ➡ Load shedding may be required if the system can’t keep up with the stream arrival rates
6.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/6 DSMS Implementation Choices • Application on top of a relational DBMS ➡ Application simulates data-driven processing ➡ Inefficient due to the semantic gap between the DBMS and the DSMS-like application • Use advanced features of the DBMS engine ➡ Triggers, materialized views, temporal/sequence data models ➡ Still based upon query-driven model, triggers don’t scale and are not expressive enough • Specialized DSMS ➡ Incorporate streaming semantics and data-driven processing model inside the engine
7.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/7 Abstract System Architecture
8.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/8 Stream Data Models • Append-only sequence of timestamped items that arrive in some order. • More relaxed definitions are possible ➡ Revision tuples ➡ Sequence of events (as in publish/subscribe systems) ➡ Sequence of sets (or bags) of elements with each set storing elements that have arrived during the sameunit of time. ➡ … • Possible models ➡ Unordered cash register ➡ Ordered cash register ➡ Unordered aggregate ➡ Ordered aggregate
9.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/9 Processing Model • Stream-in-stream-out • Problem ➡ Streams have unbounded length (system point of view) ➡ New data are more accurate/interesting (user point of view) • Solution ➡ Windows Data Stream System
10.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/10 Windows • Based on direction of movement of endpoints ➡ Two endpoints can be fixed, moving forward, or moving backward ➡ Nine possibilities, interesting ones ✦ Fixed window ✦ Sliding window ✦ Landmark window • Based on direction of window size ➡ Logical (or time-based) window ➡ Physical (or count-based) window ➡ Predicate window • Based on windows within windows ➡ Elastic window ➡ N-of-N window • Based on window update interval ➡ Jumping window ➡ Tumbling window
11.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/11 Stream Query Languages • Queries are persistent • They may be monotonic or non-monotonic ➡ Monotonic: result always grows ✦ If Q(t) is the result of a query at time t, given two executions at time ti and tj, Q(ti) Q(tj) for all ti> tj ➡ Non-monotonic: deletions from the result are possible • Monotonic query semantics: ➡ • Non-monotonic query semantics: ➡
12.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/12 Declarative Languages • Syntax similar to SQL + window specifications • Examples: CQL, GSQL, StreaQuel • CQL ➡ Three types of operators: ✦ Relation-to-realtion ✦ Stream-to-relation ✦ Relation-to-stream ➡ Join of one-minute windows on the a-attribute: SELECT * FROM S1 [RANGE 1 min], S2 [RANGE 1 min] WHERE S1.a=S2.a ➡ ROWS for count-based windows, RANGE for time-based windows
13.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/13 Declarative Languages (cont’d) • GSQL ➡ Input and output are streams (composability) ➡ Each stream should have an ordering attribute (e.g., timestamp) ➡ Subset of operators of SQL (selection, aggregation with group-by, join) ➡ Stream merge operator ➡ Only landmark windows, sliding windows may be simulated • StreaQuel ➡ SQL syntax ➡ Query includes a for-loop construct with a variable t that iterates over time ➡ Sliding window over stream S with size 5 that should run for 50 time units: for(t=ST; t<ST+50; t++) WindowIs(S, t-4, t)
14.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/14 Object-based Languages • Use abstract data typing and/or type hierarchies • Examples: Tribeca, Cougar • Tribeca ➡ Models stream contents according to a type hierarchy ➡ SQL-like syntax, accepts a stream as input and generates one or more output streams ➡ Operations: projection, selection, aggregation (over the entire input stream or over a sliding window), multiplex and demultiplex (corresponding to union and group-by) • Cougar ➡ Model sources as ADTs ➡ SQL-like syntax + $every() clause to specify re-execution frequency
15.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/15 Procedural Languages • Let the user specify how the data should flow through the system • Example: Aurora • Aurora ➡ Accepts streams as inputs and generates output streams ➡ Static data sets may be incorporated into query plans via connection points ➡ SQuAl algebra ✦ Seven operators: projection, union, map, buffered sort, windowed aggregate, binary band join, resample ➡ Interface includes ✦ Boxes that correspond to operators ✦ Edges that connect boxes that correspond to data flow ✦ User creates the execution plan
16.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/16 Comparison of Languages Language/Sys tem Allowed inputs Allowed outputs Novel operators Supported windows Execution frequency CQL/STREA M Streams and relations Streams and relations Relation-to- stream, stream-to- relation Sliding Continuous or periodic GSQL/ Gigascope Streams Streams Order- preserving union Landmark Periodic StreaQuel/ TelegraphCQ Streams and relations Sequences of relations WindowIs Fixed, landmark, sliding Continuous or periodic Tribeca Single stream Streams Multiplex, demultiplex Fixed, landmark, sliding Continuous SQuAl/Auror a Streams and relations Streams Resample, map, buffered sort Fixed, landmark, sliding Continuous or periodic
17.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/17 Operators over Unbounded Streams • Simple relational operators (selection, projection) are fine • Other operators (e.g., nested loop join) are blocking ➡ You need to see the entire inner operand • For some blocking operators, non- blocking versions exist ➡ Symmetric hash join a a a a b pass or drop S1 a b d a c e f g d b d b f f insert probe generate result S1 S2
18.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/18 Blocking Operators • Alternatives if no non-blocking version exists ➡ Constraints over the input streams ✦ Schema-level ✦ Data-level ✓ Punctuations ➡ Approximation ✦ Summaries ✓ Counting methods ✓ Sketches ➡ Windowed operations
19.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/19 Operators over Sliding Windows • Joins and aggregation may require unbounded state, so they typically operate over sliding windows • E.g., track the maximum value in an on-line sequence over a sliding window of the last N time units 75 53 67 6871 67 73 70 68 65 64 62 61 time Max = 75Max = ?
20.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/20 Operators over Sliding Windows • Issues ➡ Need to store the window so that we “remember what to forget” and when ➡ Need to undo previous results by way of negative tuples
21.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/21 Query Processing • Queuing and scheduling ➡ Queues allow sources to push data into the query plan and operators to pull data when they need them ➡ Timeslicing ➡ Allowing multiple operators to process one or multiple tuples • Tuple expiration ➡ Removing old tuples from their state buffers and (possibly) update answers ➡ Time-based window: simple – when time moves ✦ Join results have interesting expiration times ✦ Negation operator may force tuples to expire earlier ➡ Count-based window: no. of tuples constant overwrite the oldest tuple with the new arriving tuple
22.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/22 Query Processing (cont’d) • Continuous query processing over sliding windows ➡ Negative tuple approach ➡ Direct approach
23.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/23 Negative Tuple Approach • Negative tuples flow through the plan • Corresponding “real” tuples deleted from operator state • Updated answer generated, if necessary • Each tuple is processed twice Stream 1 Stream 2 MAX Index lookup Index lookup Negative tuple
24.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/24 Direct Approach • No negative tuples • Operator states are scanned each time window moves • Updated answer generated, if necessary • Each tuple is processed once, but state maintenance expensive Stream 1 Stream 2 MAX Check timestamps Check timestamps
25.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/25 Periodic Query Evaluation • Generate output periodically rather than continuously • No need to react to every insertion/expiration • E.g., compute MAX over a 10-minute window that slides every minute ➡ Store MAX over each non-overlapping one-minute chunk ➡ Take the max of the MAXes stored in each chunk time max = max(10,17,…,37,32) = 37 10171333352815163732
26.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/26 DSMS Optimization Framework • General idea: similar to cost-based DBMS query optimization • Generate candidate query plans ➡ New DSMS-specific rewritings: selections and time-based sliding windows commute, but not selections and count-based windows • Compute the cost of some of the plans and choose the cheapest plan ➡ New cost model for persistent queries: ✦ per unit time ✦ queries typically evaluated in main memory, so disk I/O is not a concern
27.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/27 Additional DSMS Optimizations – Scheduling • Scheduling • Many tuples at a time: ➡ Each operator gets a timeslice and processes all the tuples in its input queue • Many operators at a time: ➡ Each tuple is processed by all the operators in the pipeline • Choice of scheduling strategy depends upon optimization goal ➡ Minimize end-to-end latency? ➡ Minimize queue sizes?
28.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/28 Additional DSMS Optimizations – Adaptivity • System conditions can change throughout the lifetime of a persistent query ➡ Query workload can change ➡ Stream arrival rates can change • Adjust the query plan on-the-fly ➡ Or do away with the query plan and route tuples through the query operators according to some routing strategy ✦ Eddies approach
29.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/29 Additional DSMS Optimizations – Load Shedding • Random load shedding ➡ Randomly drop a fraction of arriving tuples • Semantic load shedding ➡ Examine the contents of a tuple before deciding whether or not to drop it ➡ Some tuples may have more value than others • Or, rather than dropping tuples: ➡ Spill to disk and process during idle times ➡ Shorten the windows ➡ Update the answer less often
30.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/30 Additional DSMS Optimizations – Multi-Query Processing • DBMS: queries are typically issued individually • DSMS: many persistent queries may be in the system at any given time ➡ Some of them may be similar and could be executed together ➡ E.g., similar SELECT and WHERE clauses, but different window length in the FROM clause ➡ Or, same SELECT and FROM clauses, but different predicate in the WHERE clause
31.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/31 Cloud Computing • The vision ➡ On demand, reliable services provided over the Internet (the “cloud”) with easy access to virtually infinite computing, storage and networking resources • Simple and effective! ➡ Through simple Web interfaces, users can outsource complex tasks ✦ Data mgt, system administration, application deployment ➡ The complexity of managing the infrastructure gets shifted from the users' organization to the cloud provider • Capitalizes on previous computing models ➡ Web services, utility computing, cluster computing, virtualization, grid computing
32.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/32 Great Opportunities for Research! • Technical Grand Challenge ➡ Cost-effective support of the very large scale of the infrastruture to manage lots of users and resources with high QoS • Current solutions are ad-hoc and proprietary ➡ Developed by Web industry giants such as Amazon, Google, Microsoft and Yahoo ✦ E.g. Google File System (GFS) ➡ Specific, simple applications with low consistency needs • But the research community is catching up ➡ Many new conferences and journals on Cloud Computing ✦ Distributed systems, OS, data management communities ➡ Open Source alternatives, e.g. Hadoop HDFS ➡ As the complexity of applications increases, the implication of the research community is needed
33.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/33 Cloud Definition • Working def.: a cloud provides on demand resources and services over the Internet, usually at the scale and with the reliability of a data center • Everything gets delivered as a service ➡ Pay-as-you-go pricing model, whereby users only pay for the resources they consume ➡ Service Level Agreement (SLA) to govern the use of services by customers and support pricing ✦ E.g. the service uptime during a billing cycle (e.g. a month) should be at least 99%, and if the commitment is not met, the customer should get a service credit
34.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/34 Cloud Taxonomy • Infrastructure-as-a-Service (IaaS) • Computing, networking and storage resources, as a service • Provides elasticity: ability to scale up (add more resources) or scale down (release resources) as needed ➡ E.g. Amazon Web Services • Software-as-a-Service (SaaS) ➡ Application software as a service ➡ Generalizes the earlier ASP model with tools to integrate other applications, e.g. developed by the customer (using the cloud platform) ➡ Hosted applications: from simple (email, calendar) to complex (CRM, data analysis or social network) ➡ E.g. Safesforce CRM system • Platform-as-a-Service (PaaS) ➡ Computing platform with development tools and APIs as a service ➡ Enables developers to create and deploy custom applications directly on the cloud infrastructure and integrate them with applications provided as SaaS ➡ Ex. Google Apps
35.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/35 Cloud Benefits • Reduced cost ➡ Customer side: the IT infrastructure needs not be owned and managed, and billed only based on resource consumption ➡ Cloud provider side: by sharing costs for multiple customers, reduces its cost of ownership and operation to the minimum • Ease of access and use ➡ Customers can have access to IT services anytime, from anywhere with an Internet connection • Quality of Service (QoS) ➡ The operation of the IT infrastructure by a specialized, experienced provider (including with its own infrastructure) increases QoS • Elasticity ➡ Easy for customers to deal with sudden increases in loads by simply creating more virtual machines (VMs)
36.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/36 The Main Issue: Security and Privacy • Current solutions ➡ Internal cloud (or private cloud) : the use of cloud technologies but in a private network behind a firewall ✦ Much tighter security ✦ Reduced cost advantage because the infrastructure is not shared with other customers (as in public cloud) ✦ Compromise: hybrid cloud (internal cloud for OLTP + public cloud for OLAP) ➡ Virtual private cloud: Virtual Private Network (VPN) within a public cloud with security services ✦ Promise of a similar level of security as an internal cloud and tighter integration with internal cloud security ✦ But such security integration is complex and requires talented security administrators
37.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/37 • OLTP ➡ Operational databases of average sizes (TB), write-intensive ➡ ACID transactional properties, strong data protection, response time guarantees • Not very suitable for cloud ➡ Requires shared-disk multiprocessors ➡ Corporate data gets stored at untrusted host • OLAP ➡ Historical databases of very large sizes (PB), read-intensive, can accept relaxed ACID properties • Suitable for cloud ➡ Shared-nothing clusters of commodity servers are cost-effective ➡ Sensitive data can be hidden (anonymized) in the cloud
38.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/38 Grid Architecture • Access through Web services to distributed, heterogeneous resources ➡ supercomputers, clusters, databases, etc. • For Virtual Organizations ➡ which share the same resources, with common rules and access rights • Grid middleware ➡ security, database, provisioning, job scheduling, workflow management, etc. Cluster 1 Service Compute nodes nodes User 1 User 2 Cluster 2 Service Compute nodes nodes Cluster 3 Service Storage nodes nodes reserve deploy run clean store clean reserve store WS calls MPI calls WS calls WS calls
39.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/39 Cloud Architecture User 1 User 2 Cluster 1 Service Compute Storage nodes nodes nodes Create VMs start VMs terminate pay WS calls Cluster 2 Service Compute Storage nodes nodes nodes reserve store pay • Like grid, access to resources using Web services ➡ But less distribution, more homogeneity, and bigger clusters • For different customers ➡ Including individuals • Replication across sites for high availability • Scalability, SLA, accounting and pricing essential
40.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/40 Cloud Data Management: why not RDBMS? • RDBMS all have a distributed and parallel version ➡ With SQL support for all kinds of data (structured, XML, multimedia, streams, etc.) • But the “one size fits all” approach has reached the limits ➡ Loss of performance, simplicity and flexibility for applications with specific, tight requirements ➡ New specialized DBMS engines better: column-oriented DBMS for OLAP, DSMS for stream processing, etc. • For the cloud, RDBMS provide both ➡ Too much: ACID transactions, complex query language, lots of tuning knobs ➡ Too little: specific optimizations for OLAP, flexible programming model, flexible schema, scalability
41.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/41 Cloud Data Management Solutions • Cloud data ➡ Can be very large (e.g. text-based or scientific applications), unstructured or semi-structured, and typically append-only (with rare updates) • Cloud users and application developers ➡ In very high numbers, with very diverse expertise but very little DBMS expertise • Therefore, current cloud data management solutions trade consistency for scalability, simplicity and flexibility ➡ New file systems: GFS, HDFS, … ➡ New DBMS: Amazon SimpleDB, Google Base, Google Bigtable, Yahoo Pnuts, etc. ➡ New parallel programming: Google MapReduce (and its many variations)
42.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/42 Google File System (GFS) • Used by many Google applications ➡ Search engine, Bigtable, Mapreduce, etc. • The basis for popular Open Source implementations: Hadoop HDFS (Apache & Yahoo) • Optimized for specific needs ➡ Shared-nothing cluster of thousand nodes, built from inexpensive harware => node failure is the norm! ➡ Very large files, of typically several GB, containing many objects such as web documents ➡ Mostly read and append (random updates are rare) ✦ Large reads of bulk data (e.g. 1 MB) and small random reads (e.g. 1 KB) ✦ Append operations are also large and there may be many concurrent clients that append the same file ✦ High throughput (for bulk data) more important than low latency
43.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/43 Design Choices • Traditional file system interface (create, open, read, write, close, and delete file) ➡ Two additional operations: snapshot and record append. • Relaxed consistency, with atomic record append ➡ No need for distributed lock management ➡ Up to the application to use techniques such as checkpointing and writing self-validating records • Single GFS master ➡ Maintains file metadata such as namespace, access control information, and data placement information ➡ Simple, lightly loaded, fault-tolerant • Fast recovery and replication strategies
44.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/44 GFS Distributed Architecture • Files are divided in fixed-size partitions, called chunks, of large size, i.e. 64 MB, each replicated at several nodes Application GFS client Get chunk location Get chunk data GFS chunk server Linux file system GFS Master GFS chunk server Linux file system
45.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/45 Google Bigtable • Database storage system for a shared-nothing cluster ➡ Uses GFS to store structured data, with fault-tolerance and availability • Used by popular Google applications ➡ Google Earth, Google Analytics, Orkut, etc. • The basis for popular Open Source implementations ➡ Hadoop Hbase on top of HDFS (Apache & Yahoo) • Specific data model that combines aspects of row-store and column-store DBMS ➡ Rows with multi-valued, timestamped attributes ✦ A Bigtable is defined as a multidimensional map, indexed by a row key, a column key and a timestamp, each cell of the map being a single value (a string) • Dynamic partitioning of tables for scalability
46.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/46 A Bigtable Row Row key Contents: Anchor: Language: "google.com" "google.com" "Google" "<html> …<html>""com.google.www" "english" "<html> …<html>" inria.fr t5 t1t2 t3 t4 t1 uwaterloo.ca Row unique id Column family Column key Column family = a kind of multi-valued attribute • Set of columns (of the same type), each identified by a key - Colum key = attribute value, but used as a name • Unit of access control and compression
47.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/47 Bigtable DDL and DML • Basic API for defining and manipulating tables, within a programming language such as C++ ➡ Various operators to write and update values, and to iterate over subsets of data, produced by a scan operator ➡ Various ways to restrict the rows, columns and timestamps produced by a scan, as in relational select, but no complex operator such as join or union ➡ Transactional atomicity for single row updates only
48.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/48 Dynamic Range Partitioning • Range partitioning of a table on the row key ➡ Tablet = a partition corresponding to a row range. ➡ Partitioning is dynamic, starting with one tablet (the entire table range) which is subsequently split into multiple tablets as the table grows ➡ Metadata table itself partitioned in metadata tablets, with a single root tablet stored at a master server, similar to GFS’s master • Implementation techniques ➡ Compression of column families ➡ Grouping of column families with high locality of access ➡ Aggressive caching of metadata information by clients
49.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/49 Yahoo! PNUTS • Parallel and distributed database system • Designed for serving Web applications ➡ No need for complex queries ➡ Need for good response time, scalability and high availability ➡ Relaxed consistency guarantees for replicated data • Used internally at Yahoo! ➡ User database, social networks, content metadata management and shopping listings management apps
50.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/50 Design Choices • Basic relational data model ➡ Tables of flat records, Blob attributes ➡ Flexible schemas ✦ New attributes can be added at any time even though the table is being queried or updated ✦ Records need not have values for all attributes • Simple query language ➡ Selection and projection on a single relation ➡ Updates and deletes must specify the primary key • Range partitioning or hashing of tables into tablets ➡ Placement in a cluster (at a site) ➡ Sites in different geographical regions maintain a complete copy of the system and of each table • Publish/subscribe mechanism with guaranteed delivery, for both reliability and replication ➡ Used to replay lost updates, thus avoiding a traditional database log
51.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/51 Relaxed Consistency Model • Between strong consistency and eventual consistency ➡ Motivated by the fact that Web applications typically manipulate only one record at a time, but different records may be used under different geographic locations • Per-record timeline consistency: guarantees that all replicas of a given record apply all updates to the record in the same order • Several API operations with different guarantees ➡ Read-any: returns a possibly stale version of the record ➡ Read-latest: returns the latest copy of the record ➡ Write: performs a single atomic write operation
52.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/52 MapReduce • For data analysis of very large data sets ➡ Highly dynamic, irregular, schemaless, etc. ➡ SQL or Xquery too heavy • New, simple parallel programming model ➡ Data structured as (key, value) pairs ✦ E.g. (doc-id, content), (word, count), etc. ➡ Functional programming style with two functions to be given: ✦ Map(k1,v1) list(k2,v2) ✦ Reduce(k2, list (v2)) list(v3) • Implemented on GFS on very large clusters
53.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/53 MapReduce Typical Usages • Counting the numbers of some words in a set of docs • Distributed grep: text pattern matching • Counting URL access frequencies in Web logs • Computing a reverse Web-link graph • Computing the term-vectors (summarizing the most important words) in a set of documents • Computing an inverted index for a set of documents • Distributed sorting
54.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/54 MapReduce Processing Map … (k1,v) (k2,v) Group by k Map (k2,v) (k2,v) Map (k1,v) Map (k1,v) (k2,v) (k1,(v,v,v)) (k2,(v,v,v,v)) Reduce Reduce Group by k Inputdataset Outputdataset
55.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/55 MapReduce Example EMP (ENAME, TITLE, CITY) Query: for each city, return the number of employees whose name is "Smith" SELECT CITY, COUNT(*) FROM EMP WHERE ENAME LIKE "%Smith" GROUP BY CITY With MapReduce Map (Input (TID,emp), Output: (CITY,1)) if emp.ENAME like "%Smith" return (CITY,1) Reduce (Input (CITY,list(1)), Output: (CITY,SUM(list(1))) return (CITY,SUM(1*))
56.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/56 Fault-tolerance • Fault-tolerance is fine-grain and well suited for large jobs • Input and output data are stored in GFS ➡ Already provides high fault-tolerance • All intermediate data is written to disk ➡ Helps checkpointing Map operations, and thus provides tolerance from soft failures • If one Map node or Reduce node fails during execution (hard failure) ➡ The tasks are made eligible by the master for scheduling onto other nodes ➡ It may also be necessary to re-execute completed Map tasks, since the input data on the failed node disk is inaccessible
57.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/57 MapReduce vs Parallel DBMS • [Pavlo et al. SIGMOD09]: Hadoop MapReduce vs two parallel DBMS, one row- store DBMS and one column-store DBMS ➡ Benchmark queries: a grep query, an aggregation query with a group by clause on a Web log, and a complex join of two tables with aggregation and filtering ➡ Once the data has been loaded, the DBMS are significantly faster, but loading is much time consuming for the DBMS ➡ Suggest that MapReduce is less efficient than DBMS because it performs repetitive format parsing and does not exploit pipelining and indices • [Dean and Ghemawat, CACM10] ➡ Make the difference between the MapReduce model and its implementation which could be well improved, e.g. by exploiting indices • [Stonebraker et al. CACM10] ➡ Argues that MapReduce and parallel DBMS are complementary as MapReduce could be used to extract-transform-load data in a DBMS for more complex OLAP.
58.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/58 Issues in Cloud Data Management • Main challenge: provide ease of programming, consistency, scalability and elasticity at the same time, over cloud data • Current solutions ➡ Quite successful for specific, relatively simple applications ➡ Have sacrificed consistency and ease of programming for the sake of scalability ➡ Force applications to access data partitions individually, with a loss of consistency guarantees across data partitions • For more complex apps. with tighter consistency requirements ➡ Developers are faced with a very difficult problem: providing isolation and atomicity across data partitions through careful engineering
59.
Distributed DBMS ©
M. T. Özsu & P. Valduriez Ch.18/59 Research Directions in Data Management • Declarative programming languages for the cloud ➡ E.g. BOOM project (UC Berkeley] using Overlog • Parallel OLAP query processing with consistency guarantees wrt concurrent updates ➡ E.g. using snapshot isolation • Scientific workflow management ➡ E.g. with P2P worker nodes • Data privacy preserving query processing ➡ E.g. queries on encrypted data • Autonomic data management ➡ E.g. automatic management of replication to deal with load changes • Green data management ➡ E.g. optimizing for energy efficiency
Notas do Editor
Animated slide
Baixar agora