SlideShare uma empresa Scribd logo
1 de 32
Baixar para ler offline
Tools and tech
for Big Data success
Donnie Berkholz, Ph.D.
IT Industry Analyst
@dberkholz
[Imagine cheesy clipart of hammers, silicon, etc.]
Tools and tech — huh?
● Languages
● Infrastructure
● Tooling
● It's not about the what, it's about the how
Thesis: Technology adoption
is increasingly bottom-up
The new kingmakers
The best DX wins
It's all about barriers to entry
Ecosystems matter
The challenges
IT pros delivering data solutions
60/36 IT/biz
Talend, summer 2012
The skills gap
NVP survey of execs, summer 2012
What can we learn from book sales?
What about the data analysts?
KDNuggets, summer 2012
What about the data analysts?
KDNuggets, summer 2012
It's not all about popularity...
The real growth isn't in SQL interfaces
Developers vs. marketers
Google Trends
IT pros delivering data solutions
60/36 IT/biz
Talend, summer 2012
Hadoop distributions: popularity
Google Trends: $VENDOR hadoop
Developers choose what's easiest
How to run: bare metal / DC, private
cloud, public cloud
Storing Data: HDFS, Ceph, Gluster
Open Building Blocks but not Open
Source: EMR on AWS
Realtime: Impala, Druid
In-memory: Redis, Memcached
Streaming: Storm, S4
Other Options: Hadoop YARN,
HPCC, Cassandra, Mongo, Riak
Other Options: Spark/Shark/Mesos
Abstraction: Mortar, Continuuity,
Qubole, Concurrent Lingual, etc.
Conclusion: KISS
Donnie Berkholz, Ph.D.
Analyst, RedMonk
@dberkholz
Disclosure: 10gen, Amazon, Basho, Cloudera,
Continuuity, IBM, MapR, Microsoft, and VMware are
clients.

Mais conteúdo relacionado

Mais procurados

Dataiku - google cloud platform roadshow - october 2013
Dataiku  - google cloud platform roadshow - october 2013Dataiku  - google cloud platform roadshow - october 2013
Dataiku - google cloud platform roadshow - october 2013
Dataiku
 
Online Games Analytics - Data Science for Fun
Online Games Analytics - Data Science for FunOnline Games Analytics - Data Science for Fun
Online Games Analytics - Data Science for Fun
Dataiku
 
Symposium 2018 - Big data transport and collaboration - Gregory Vial
Symposium 2018 - Big data  transport and collaboration - Gregory VialSymposium 2018 - Big data  transport and collaboration - Gregory Vial
Symposium 2018 - Big data transport and collaboration - Gregory Vial
PMI-Montréal
 
Dataiku r users group v2
Dataiku   r users group v2Dataiku   r users group v2
Dataiku r users group v2
Cdiscount
 

Mais procurados (18)

How to Build a Successful Data Team - Florian Douetteau (@Dataiku)
How to Build a Successful Data Team - Florian Douetteau (@Dataiku) How to Build a Successful Data Team - Florian Douetteau (@Dataiku)
How to Build a Successful Data Team - Florian Douetteau (@Dataiku)
 
Neo4j Innovation Lab, Stefan Wendin, Neo4j
Neo4j Innovation Lab, Stefan Wendin, Neo4jNeo4j Innovation Lab, Stefan Wendin, Neo4j
Neo4j Innovation Lab, Stefan Wendin, Neo4j
 
Dataiku - google cloud platform roadshow - october 2013
Dataiku  - google cloud platform roadshow - october 2013Dataiku  - google cloud platform roadshow - october 2013
Dataiku - google cloud platform roadshow - october 2013
 
Is Data Scientist still the sexiest job of 21st century? Find Out!
Is Data Scientist still the sexiest job of 21st century? Find Out!Is Data Scientist still the sexiest job of 21st century? Find Out!
Is Data Scientist still the sexiest job of 21st century? Find Out!
 
Data Culture Series - Keynote - 24th feb
Data Culture Series - Keynote - 24th febData Culture Series - Keynote - 24th feb
Data Culture Series - Keynote - 24th feb
 
From Lab to Factory: Or how to turn data into value
From Lab to Factory: Or how to turn data into valueFrom Lab to Factory: Or how to turn data into value
From Lab to Factory: Or how to turn data into value
 
Online Games Analytics - Data Science for Fun
Online Games Analytics - Data Science for FunOnline Games Analytics - Data Science for Fun
Online Games Analytics - Data Science for Fun
 
Nonprofits + Data: Pathway to Innovation
Nonprofits + Data: Pathway to InnovationNonprofits + Data: Pathway to Innovation
Nonprofits + Data: Pathway to Innovation
 
Data Scientist: Sexiest job of the 21st century
Data Scientist: Sexiest job of the 21st centuryData Scientist: Sexiest job of the 21st century
Data Scientist: Sexiest job of the 21st century
 
Back to Square One: Building a Data Science Team from Scratch
Back to Square One: Building a Data Science Team from ScratchBack to Square One: Building a Data Science Team from Scratch
Back to Square One: Building a Data Science Team from Scratch
 
iTrain Malaysia: Data Science by Tarun Sukhani
iTrain Malaysia: Data Science by Tarun SukhaniiTrain Malaysia: Data Science by Tarun Sukhani
iTrain Malaysia: Data Science by Tarun Sukhani
 
Data science skills and development for the energy sector
Data science skills and development for the energy sectorData science skills and development for the energy sector
Data science skills and development for the energy sector
 
Symposium 2018 - Big data transport and collaboration - Gregory Vial
Symposium 2018 - Big data  transport and collaboration - Gregory VialSymposium 2018 - Big data  transport and collaboration - Gregory Vial
Symposium 2018 - Big data transport and collaboration - Gregory Vial
 
Data Science Salon: Introduction to Machine Learning - Marketing Use Case
Data Science Salon: Introduction to Machine Learning - Marketing Use CaseData Science Salon: Introduction to Machine Learning - Marketing Use Case
Data Science Salon: Introduction to Machine Learning - Marketing Use Case
 
Big Data and HR - Talk @SwissHR Congress
Big Data and HR - Talk @SwissHR CongressBig Data and HR - Talk @SwissHR Congress
Big Data and HR - Talk @SwissHR Congress
 
What is a Data Scientist
What is a Data Scientist What is a Data Scientist
What is a Data Scientist
 
5 ways to get more from data science
5 ways to get more from data science5 ways to get more from data science
5 ways to get more from data science
 
Dataiku r users group v2
Dataiku   r users group v2Dataiku   r users group v2
Dataiku r users group v2
 

Destaque

The limits of sharing craft
The limits of sharing craftThe limits of sharing craft
The limits of sharing craft
Donnie Berkholz
 
The parallel universes of DevOps and cloud developers (GlueCon)
The parallel universes of DevOps and cloud developers (GlueCon)The parallel universes of DevOps and cloud developers (GlueCon)
The parallel universes of DevOps and cloud developers (GlueCon)
Donnie Berkholz
 

Destaque (16)

451’s Berkholz on How DevOps, Automation and Orchestration Combine for Contin...
451’s Berkholz on How DevOps, Automation and Orchestration Combine for Contin...451’s Berkholz on How DevOps, Automation and Orchestration Combine for Contin...
451’s Berkholz on How DevOps, Automation and Orchestration Combine for Contin...
 
Viking reprise: Nordic influences on US tech culture
Viking reprise: Nordic influences on US tech cultureViking reprise: Nordic influences on US tech culture
Viking reprise: Nordic influences on US tech culture
 
Commit yourself already: Contributing to open source
Commit yourself already: Contributing to open sourceCommit yourself already: Contributing to open source
Commit yourself already: Contributing to open source
 
The limits of sharing craft
The limits of sharing craftThe limits of sharing craft
The limits of sharing craft
 
Building with containers: How containers will drive cloud services
Building with containers: How containers will drive cloud servicesBuilding with containers: How containers will drive cloud services
Building with containers: How containers will drive cloud services
 
451 Group on Analyst Relations
451 Group on Analyst Relations451 Group on Analyst Relations
451 Group on Analyst Relations
 
Parallel universes of DevOps and cloud developers, plus a BONUS config manage...
Parallel universes of DevOps and cloud developers, plus a BONUS config manage...Parallel universes of DevOps and cloud developers, plus a BONUS config manage...
Parallel universes of DevOps and cloud developers, plus a BONUS config manage...
 
Is your community healthy? Metrics on the top CM software
Is your community healthy? Metrics on the top CM softwareIs your community healthy? Metrics on the top CM software
Is your community healthy? Metrics on the top CM software
 
The parallel universes of DevOps and cloud developers
The parallel universes of DevOps and cloud developersThe parallel universes of DevOps and cloud developers
The parallel universes of DevOps and cloud developers
 
The parallel universes of DevOps and cloud developers (GlueCon)
The parallel universes of DevOps and cloud developers (GlueCon)The parallel universes of DevOps and cloud developers (GlueCon)
The parallel universes of DevOps and cloud developers (GlueCon)
 
Emerging trends in software development: The next generation of storage
Emerging trends in software development: The next generation of storageEmerging trends in software development: The next generation of storage
Emerging trends in software development: The next generation of storage
 
DevOps 101+: From collaboration to microservices
DevOps 101+: From collaboration to microservicesDevOps 101+: From collaboration to microservices
DevOps 101+: From collaboration to microservices
 
DevOps 101
DevOps 101DevOps 101
DevOps 101
 
How IT will disrupt in 2016: The ITaaS imperative
How IT will disrupt in 2016: The ITaaS imperativeHow IT will disrupt in 2016: The ITaaS imperative
How IT will disrupt in 2016: The ITaaS imperative
 
The New Assembly Line: 3 Best Practices for Building (Secure) Connected Cars
The New Assembly Line: 3 Best Practices for Building (Secure) Connected CarsThe New Assembly Line: 3 Best Practices for Building (Secure) Connected Cars
The New Assembly Line: 3 Best Practices for Building (Secure) Connected Cars
 
China capitulates: Another round of auto stimulus
China capitulates: Another round of auto stimulusChina capitulates: Another round of auto stimulus
China capitulates: Another round of auto stimulus
 

Semelhante a Tools and Tech for Big Data Success

Architecting a Data Platform For Enterprise Use (Strata NY 2018)
Architecting a Data Platform For Enterprise Use (Strata NY 2018)Architecting a Data Platform For Enterprise Use (Strata NY 2018)
Architecting a Data Platform For Enterprise Use (Strata NY 2018)
mark madsen
 

Semelhante a Tools and Tech for Big Data Success (20)

Building a Data Platform Strata SF 2019
Building a Data Platform Strata SF 2019Building a Data Platform Strata SF 2019
Building a Data Platform Strata SF 2019
 
5 Major Trends in Data You Should Know
5 Major Trends in Data You Should Know5 Major Trends in Data You Should Know
5 Major Trends in Data You Should Know
 
Architecting a Data Platform For Enterprise Use (Strata NY 2018)
Architecting a Data Platform For Enterprise Use (Strata NY 2018)Architecting a Data Platform For Enterprise Use (Strata NY 2018)
Architecting a Data Platform For Enterprise Use (Strata NY 2018)
 
Big data and you
Big data and you Big data and you
Big data and you
 
IT In Europe
IT In EuropeIT In Europe
IT In Europe
 
Seminarppt
SeminarpptSeminarppt
Seminarppt
 
Big Data: an introduction
Big Data: an introductionBig Data: an introduction
Big Data: an introduction
 
The Role of Data Wrangling in Driving Hadoop Adoption
The Role of Data Wrangling in Driving Hadoop AdoptionThe Role of Data Wrangling in Driving Hadoop Adoption
The Role of Data Wrangling in Driving Hadoop Adoption
 
Café da manhã - São Paulo - Use-cases and opportunities in BigData with Hadoop
Café da manhã - São Paulo - Use-cases and opportunities in BigData with HadoopCafé da manhã - São Paulo - Use-cases and opportunities in BigData with Hadoop
Café da manhã - São Paulo - Use-cases and opportunities in BigData with Hadoop
 
How to become a data scientist
How to become a data scientist How to become a data scientist
How to become a data scientist
 
Data and data scientists are not equal to money david hoyle
Data and data scientists are not equal to money   david hoyleData and data scientists are not equal to money   david hoyle
Data and data scientists are not equal to money david hoyle
 
Big Data Visualisation with Hadoop and PowerPivot
Big Data Visualisation with Hadoop and PowerPivotBig Data Visualisation with Hadoop and PowerPivot
Big Data Visualisation with Hadoop and PowerPivot
 
SF Data Science: Developing Data Products
SF Data Science: Developing Data ProductsSF Data Science: Developing Data Products
SF Data Science: Developing Data Products
 
You're the New CDO, Now What?
You're the New CDO, Now What?You're the New CDO, Now What?
You're the New CDO, Now What?
 
TBR IT Trends Study
TBR IT Trends StudyTBR IT Trends Study
TBR IT Trends Study
 
Big Data
Big DataBig Data
Big Data
 
The Maturity Model: Taking the Growing Pains Out of Hadoop
The Maturity Model: Taking the Growing Pains Out of HadoopThe Maturity Model: Taking the Growing Pains Out of Hadoop
The Maturity Model: Taking the Growing Pains Out of Hadoop
 
Data scientist the sexiest job of the 21st century by thomas h davenport and ...
Data scientist the sexiest job of the 21st century by thomas h davenport and ...Data scientist the sexiest job of the 21st century by thomas h davenport and ...
Data scientist the sexiest job of the 21st century by thomas h davenport and ...
 
Data In Action: Business Value of Data
Data In Action: Business Value of DataData In Action: Business Value of Data
Data In Action: Business Value of Data
 
2012 IBM Tech Trends Report: Fast track to the future
2012 IBM Tech Trends Report: Fast track to the future2012 IBM Tech Trends Report: Fast track to the future
2012 IBM Tech Trends Report: Fast track to the future
 

Mais de Donnie Berkholz

Mais de Donnie Berkholz (20)

Pricing and Packaging in Covid-19 Times - Heavybit
Pricing and Packaging in Covid-19 Times - HeavybitPricing and Packaging in Covid-19 Times - Heavybit
Pricing and Packaging in Covid-19 Times - Heavybit
 
Open Source & Open Community at a 100-Year-Old Company
Open Source & Open Community at a 100-Year-Old CompanyOpen Source & Open Community at a 100-Year-Old Company
Open Source & Open Community at a 100-Year-Old Company
 
Open Source & Open Community at a 100-Year-Old Company
Open Source & Open Community at a 100-Year-Old CompanyOpen Source & Open Community at a 100-Year-Old Company
Open Source & Open Community at a 100-Year-Old Company
 
Discovering rivers & building bridges: DevOps @ CWT
Discovering rivers & building bridges: DevOps @ CWTDiscovering rivers & building bridges: DevOps @ CWT
Discovering rivers & building bridges: DevOps @ CWT
 
Assholes are killing your project (MDC 2017)
Assholes are killing your project (MDC 2017)Assholes are killing your project (MDC 2017)
Assholes are killing your project (MDC 2017)
 
Reality Check: How much influence do developers really have?
Reality Check: How much influence do developers really have?Reality Check: How much influence do developers really have?
Reality Check: How much influence do developers really have?
 
Cloud Native: A dose of reality
Cloud Native: A dose of realityCloud Native: A dose of reality
Cloud Native: A dose of reality
 
Cloud Native in the Enterprise: Real-World Data on Container and Microservice...
Cloud Native in the Enterprise: Real-World Data on Container and Microservice...Cloud Native in the Enterprise: Real-World Data on Container and Microservice...
Cloud Native in the Enterprise: Real-World Data on Container and Microservice...
 
Mining the Go Developer Community
Mining the Go Developer CommunityMining the Go Developer Community
Mining the Go Developer Community
 
How the rise of DevOps and containers is transforming IT service delivery
How the rise of DevOps and containers is transforming IT service deliveryHow the rise of DevOps and containers is transforming IT service delivery
How the rise of DevOps and containers is transforming IT service delivery
 
The OpenStack Pulse: Containers and Platforms
The OpenStack Pulse: Containers and PlatformsThe OpenStack Pulse: Containers and Platforms
The OpenStack Pulse: Containers and Platforms
 
Microservices 101: From DevOps to Docker and beyond
Microservices 101: From DevOps to Docker and beyondMicroservices 101: From DevOps to Docker and beyond
Microservices 101: From DevOps to Docker and beyond
 
Can we compare communities or are they all unique snowflakes?
Can we compare communities or are they all unique snowflakes?Can we compare communities or are they all unique snowflakes?
Can we compare communities or are they all unique snowflakes?
 
How microservices are redefining modern application architecture
How microservices are redefining modern application architectureHow microservices are redefining modern application architecture
How microservices are redefining modern application architecture
 
DevOps, containers & microservices: Separating the hype from the reality
DevOps, containers & microservices: Separating the hype from the realityDevOps, containers & microservices: Separating the hype from the reality
DevOps, containers & microservices: Separating the hype from the reality
 
Hacking industry analysis: Analyst relations for startups
Hacking industry analysis: Analyst relations for startupsHacking industry analysis: Analyst relations for startups
Hacking industry analysis: Analyst relations for startups
 
From 0 to DevOps in 5 minutes
From 0 to DevOps in 5 minutesFrom 0 to DevOps in 5 minutes
From 0 to DevOps in 5 minutes
 
Commit yourself already: Contributing to Open Source
Commit yourself already: Contributing to Open SourceCommit yourself already: Contributing to Open Source
Commit yourself already: Contributing to Open Source
 
The new stack isn’t a stack: Fragmentation and terraforming 
the service layer
The new stack isn’t a stack: Fragmentation and terraforming 
the service layerThe new stack isn’t a stack: Fragmentation and terraforming 
the service layer
The new stack isn’t a stack: Fragmentation and terraforming 
the service layer
 
Beyond Linus Pauling: Conformation dependence of ideal geometry in proteins
Beyond Linus Pauling: Conformation dependence of ideal geometry in proteinsBeyond Linus Pauling: Conformation dependence of ideal geometry in proteins
Beyond Linus Pauling: Conformation dependence of ideal geometry in proteins
 

Último

IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 

Último (20)

Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 

Tools and Tech for Big Data Success