SlideShare uma empresa Scribd logo
1 de 18
Baixar para ler offline
De praktijk van
Big Data
                            Friso van Vollenhoven
                            fvanvollenhoven@xebia.com
En waarom de huidige
technologie niet (altijd)
voldoet
Big Data
Big Data
Big Data
Big Data




                   Requirement:
           Full table scan, 200GB table
Big Data
Big Data

           Egypte, 27 januari 2011
Big Data




                    Requirement:
           40.000 updates per seconde, 24/7.
Databases




            =   +   +
Databases




              =        +    +
network


                   SAN
                  storage
HDFS en MapReduce


                 bottleneck

                 SELECT SESSION, COUNT(*) FROM
                 WEB_CLICKS GROUP BY SESSION;
                                             CLIENT


storage
network
HDFS en MapReduce




                 SELECT SESSION, COUNT(*) FROM
                 WEB_CLICKS GROUP BY SESSION;
                                             CLIENT


storage
network
            bottleneck
HDFS en MapReduce
HDFS en MapReduce


SELECT * FROM
 WEB_CLICKS;




SELECT * FROM           SELECT * FROM
 WEB_CLICKS;             WEB_CLICKS;
HDFS en MapReduce




                    GROUP BY SESSION
HDFS en MapReduce


               COUNT(*)




               COUNT(*)   COUNT(*)
HDFS en MapReduce

        MAP     REDUCE
SELECT * FROM
                    COUNT(*)
 WEB_CLICKS;




                               SORT/SHUFFLE
                               GROUP BY SESSION




        MAP     REDUCE                            MAP   REDUCE
SELECT * FROM                           SELECT * FROM
                    COUNT(*)                                COUNT(*)
 WEB_CLICKS;                             WEB_CLICKS;
NoSQL




index     A B C D E F G H I   J K L M N O P Q R S T U V W X Y Z

Mais conteúdo relacionado

Mais de Xebia Nederland BV

Mais de Xebia Nederland BV (20)

Creating the right products
Creating the right productsCreating the right products
Creating the right products
 
Videoscribe je agile transitie
Videoscribe je agile transitieVideoscribe je agile transitie
Videoscribe je agile transitie
 
Sketchnote je Product Backlog Items & Sprint Retrospectives
Sketchnote je Product Backlog Items & Sprint RetrospectivesSketchnote je Product Backlog Items & Sprint Retrospectives
Sketchnote je Product Backlog Items & Sprint Retrospectives
 
Why we need test automation, but it’s not the right question
Why we need test automation, but it’s not the right questionWhy we need test automation, but it’s not the right question
Why we need test automation, but it’s not the right question
 
Testen in de transitie naar continuous delivery
Testen in de transitie naar continuous deliveryTesten in de transitie naar continuous delivery
Testen in de transitie naar continuous delivery
 
Becoming an agile enterprise, focus on the test ingredient
Becoming an agile enterprise, focus on the test ingredientBecoming an agile enterprise, focus on the test ingredient
Becoming an agile enterprise, focus on the test ingredient
 
How DUO started with Continuous Delivery and changed their way of Testing
How DUO started with Continuous Delivery and changed their way of TestingHow DUO started with Continuous Delivery and changed their way of Testing
How DUO started with Continuous Delivery and changed their way of Testing
 
Become a digital company - Case KPN / Xebia
Become a digital company - Case KPN / XebiaBecome a digital company - Case KPN / Xebia
Become a digital company - Case KPN / Xebia
 
Building a Docker powered feature driven delivery pipeline at hoyhoy.nl
Building a Docker powered feature driven delivery pipeline at hoyhoy.nlBuilding a Docker powered feature driven delivery pipeline at hoyhoy.nl
Building a Docker powered feature driven delivery pipeline at hoyhoy.nl
 
Webinar Xebia & bol.com
Webinar Xebia & bol.comWebinar Xebia & bol.com
Webinar Xebia & bol.com
 
TestWorks Conf The magic of models for 1000% test automation - Machiel van de...
TestWorks Conf The magic of models for 1000% test automation - Machiel van de...TestWorks Conf The magic of models for 1000% test automation - Machiel van de...
TestWorks Conf The magic of models for 1000% test automation - Machiel van de...
 
TestWorks Conf Serenity BDD in action - John Ferguson Smart
TestWorks Conf Serenity BDD in action - John Ferguson SmartTestWorks Conf Serenity BDD in action - John Ferguson Smart
TestWorks Conf Serenity BDD in action - John Ferguson Smart
 
TestWorks Conf Scalable QA with docker - Maarten van den Ende and Adé Mochtar
TestWorks Conf Scalable QA with docker - Maarten van den Ende and Adé MochtarTestWorks Conf Scalable QA with docker - Maarten van den Ende and Adé Mochtar
TestWorks Conf Scalable QA with docker - Maarten van den Ende and Adé Mochtar
 
TestWorks Conf Robot framework - the unsung hero of test automation - Michael...
TestWorks Conf Robot framework - the unsung hero of test automation - Michael...TestWorks Conf Robot framework - the unsung hero of test automation - Michael...
TestWorks Conf Robot framework - the unsung hero of test automation - Michael...
 
TestWorks Conf Real world automation survival - Alan Richardson
TestWorks Conf Real world automation survival - Alan RichardsonTestWorks Conf Real world automation survival - Alan Richardson
TestWorks Conf Real world automation survival - Alan Richardson
 
TestWorks Conf Performance testing made easy with gatling - Guillaume Corré
TestWorks Conf Performance testing made easy with gatling - Guillaume CorréTestWorks Conf Performance testing made easy with gatling - Guillaume Corré
TestWorks Conf Performance testing made easy with gatling - Guillaume Corré
 
TestWorks conf Dry up your angularjs unit tests using mox - Mike Woudenberg
TestWorks conf Dry up your angularjs unit tests using mox - Mike WoudenbergTestWorks conf Dry up your angularjs unit tests using mox - Mike Woudenberg
TestWorks conf Dry up your angularjs unit tests using mox - Mike Woudenberg
 
TestWorks Conf 2015 Beefing up FitNesse - Arjan Molenaar
TestWorks Conf 2015 Beefing up FitNesse - Arjan MolenaarTestWorks Conf 2015 Beefing up FitNesse - Arjan Molenaar
TestWorks Conf 2015 Beefing up FitNesse - Arjan Molenaar
 
Adriaan de-jonge
Adriaan de-jongeAdriaan de-jonge
Adriaan de-jonge
 
Bitcoin Xebicon2012
Bitcoin Xebicon2012Bitcoin Xebicon2012
Bitcoin Xebicon2012
 

Último

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Último (20)

Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot ModelNavi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 

120412 oracle big data summit