SlideShare a Scribd company logo
1 of 35
BIGDATA APPS
ON OPENSTACK
thanks:Uri Cohen


V.P. PM @ GigaSpaces
     @uri1803
  github.com/uric
about:me


V.P. R&D @ GigaSpaces
    @g_korland
github.com/gkorland
“Big data and cloud technology
go hand-in-hand. Big data needs
clusters of servers for processing,
which clouds can readily provide.”

                  Ed Dumbill, The O’Reilly Radar
Why OpenStack?

Everyone (well, almost…) is in
HP, RackSpace, Dell, IBM, RedHat
Public / Private cloud interop
Cloud bursting
Portability
The 3 V’s of Big Data
Big Data Is Not Just About
        Elephants
Distributed File System
Map/Reduce
Data Mining
Data Access
Discovery & Orchestration
Real Time Event Processing
HDFS, in Reality
Map/Reduce, in Reality
Data Access, in Reality
Mongo, in Reality
Storm
Storm, In Reality
And What about Other Parts
      of Your App??
And This is What Your Big Data
 App Will Eventually Look Like
Managing All of That In the Cloud
Start VMs
Install and configure components
Monitor
Repair
(Auto) Scale
Burst…
Managing Big   Managing Big
 Data Apps   ≠    Data
Some Existing (incomplete)
        Solutions
Introducing Cloudify
the Open Paas Stack (Pun
When PaaS Met DevOps…
Your App, Your Recipe
How It Works           Install
                                                      Configure
                                  Bootstrap             Start
Application Recipe          Java & Cloudify Agent      Monitor
                                                    & Ensures H/A




mmands
 e Allocation



    Invokes IaaS
                     Compute
        API
                     Services
Demo
Getting Started…




     32
Get Involved!
     Community site
     http://www.cloudifysource.org
     OSS project repo:
     https://github.com/CloudifySource/cloudify
     Jira:
     https://cloudifysource.atlassian.net
     Travis-CI:

33
     http://travis-ci.org/#!/CloudifySource/cloudify
Q&A
Thank You!

 Get it today, for free:
www.cloudifysource.org

More Related Content

What's hot

Building The Modern IT
Building The Modern ITBuilding The Modern IT
Building The Modern IT
Arthur Berezin
 

What's hot (20)

Evolution of container orchestration in Alibaba Cloud
Evolution of container orchestration in Alibaba CloudEvolution of container orchestration in Alibaba Cloud
Evolution of container orchestration in Alibaba Cloud
 
From Docker Straight to AWS
From Docker Straight to AWSFrom Docker Straight to AWS
From Docker Straight to AWS
 
2016-08-25 TechExeter - going serverless with Azure
2016-08-25 TechExeter - going serverless with Azure2016-08-25 TechExeter - going serverless with Azure
2016-08-25 TechExeter - going serverless with Azure
 
Going serverless
Going serverlessGoing serverless
Going serverless
 
Automating AWS with DevOps
Automating AWS with DevOpsAutomating AWS with DevOps
Automating AWS with DevOps
 
Cloud computing overview & Technical intro to Google Cloud
Cloud computing overview & Technical intro to Google CloudCloud computing overview & Technical intro to Google Cloud
Cloud computing overview & Technical intro to Google Cloud
 
Building The Modern IT
Building The Modern ITBuilding The Modern IT
Building The Modern IT
 
Techdays Finland 2019 - Adventures of building a (multi-tenant) PaaS on Micro...
Techdays Finland 2019 - Adventures of building a (multi-tenant) PaaS on Micro...Techdays Finland 2019 - Adventures of building a (multi-tenant) PaaS on Micro...
Techdays Finland 2019 - Adventures of building a (multi-tenant) PaaS on Micro...
 
Our advice on building a scalable, resilient infrastructure with Kubernetes
Our advice on building a scalable, resilient infrastructure with KubernetesOur advice on building a scalable, resilient infrastructure with Kubernetes
Our advice on building a scalable, resilient infrastructure with Kubernetes
 
Tour of Dapr
Tour of DaprTour of Dapr
Tour of Dapr
 
Introduction to Google Cloud Platform (GCP) | Google Cloud Tutorial for Begin...
Introduction to Google Cloud Platform (GCP) | Google Cloud Tutorial for Begin...Introduction to Google Cloud Platform (GCP) | Google Cloud Tutorial for Begin...
Introduction to Google Cloud Platform (GCP) | Google Cloud Tutorial for Begin...
 
Big data on google cloud
Big data on google cloudBig data on google cloud
Big data on google cloud
 
Pyspark
PysparkPyspark
Pyspark
 
How to build virtual assistant like Jarvis (in Ironman) with Google Assistant...
How to build virtual assistant like Jarvis (in Ironman) with Google Assistant...How to build virtual assistant like Jarvis (in Ironman) with Google Assistant...
How to build virtual assistant like Jarvis (in Ironman) with Google Assistant...
 
Top Advantages of Using Google Cloud Platform
Top Advantages of Using Google Cloud PlatformTop Advantages of Using Google Cloud Platform
Top Advantages of Using Google Cloud Platform
 
Google Cloud Platform Solutions for DevOps Engineers
Google Cloud Platform Solutions  for DevOps EngineersGoogle Cloud Platform Solutions  for DevOps Engineers
Google Cloud Platform Solutions for DevOps Engineers
 
Yannis Zarkadas. Enterprise data science workflows on kubeflow
Yannis Zarkadas. Enterprise data science workflows on kubeflowYannis Zarkadas. Enterprise data science workflows on kubeflow
Yannis Zarkadas. Enterprise data science workflows on kubeflow
 
Android x Azure 輕鬆玩 - WebApp / CognitiveService / AI - Android Taipei 2019122...
Android x Azure 輕鬆玩 - WebApp / CognitiveService / AI - Android Taipei 2019122...Android x Azure 輕鬆玩 - WebApp / CognitiveService / AI - Android Taipei 2019122...
Android x Azure 輕鬆玩 - WebApp / CognitiveService / AI - Android Taipei 2019122...
 
Google Cloud Platform: Prototype ->Production-> Planet scale
Google Cloud Platform: Prototype ->Production-> Planet scaleGoogle Cloud Platform: Prototype ->Production-> Planet scale
Google Cloud Platform: Prototype ->Production-> Planet scale
 
Gcp
GcpGcp
Gcp
 

Similar to Open stack bigdata NY cloudcamp

Similar to Open stack bigdata NY cloudcamp (20)

Big Data Apps on OpenStack
Big Data Apps on OpenStackBig Data Apps on OpenStack
Big Data Apps on OpenStack
 
The Future of Cloud Innovation, featuring Adrian Cockcroft
The Future of Cloud Innovation, featuring Adrian CockcroftThe Future of Cloud Innovation, featuring Adrian Cockcroft
The Future of Cloud Innovation, featuring Adrian Cockcroft
 
OpenSlava 2014 - CloudFoundry inside-out
OpenSlava 2014 - CloudFoundry inside-outOpenSlava 2014 - CloudFoundry inside-out
OpenSlava 2014 - CloudFoundry inside-out
 
Cloud Computing
Cloud ComputingCloud Computing
Cloud Computing
 
MongoDB in the Clouds
MongoDB in the CloudsMongoDB in the Clouds
MongoDB in the Clouds
 
[Capitole du Libre] #serverless -  mettez-le en oeuvre dans votre entreprise...
[Capitole du Libre] #serverless -  mettez-le en oeuvre dans votre entreprise...[Capitole du Libre] #serverless -  mettez-le en oeuvre dans votre entreprise...
[Capitole du Libre] #serverless -  mettez-le en oeuvre dans votre entreprise...
 
Spring Boot & Spring Cloud on PAS- Nate Schutta (1/2)
Spring Boot & Spring Cloud on PAS- Nate Schutta (1/2)Spring Boot & Spring Cloud on PAS- Nate Schutta (1/2)
Spring Boot & Spring Cloud on PAS- Nate Schutta (1/2)
 
Cloud Architecture - Multi Cloud, Edge, On-Premise
Cloud Architecture - Multi Cloud, Edge, On-PremiseCloud Architecture - Multi Cloud, Edge, On-Premise
Cloud Architecture - Multi Cloud, Edge, On-Premise
 
Java Web Programming Using Cloud Platform: Module 10
Java Web Programming Using Cloud Platform: Module 10Java Web Programming Using Cloud Platform: Module 10
Java Web Programming Using Cloud Platform: Module 10
 
WSO2Con EU 2016: Planning Your Cloud Strategy
WSO2Con EU 2016: Planning Your Cloud StrategyWSO2Con EU 2016: Planning Your Cloud Strategy
WSO2Con EU 2016: Planning Your Cloud Strategy
 
PHP Buildpacks in the Cloud on Bluemix
PHP Buildpacks in the Cloud on BluemixPHP Buildpacks in the Cloud on Bluemix
PHP Buildpacks in the Cloud on Bluemix
 
Cloud Foundry for PHP developers
Cloud Foundry for PHP developersCloud Foundry for PHP developers
Cloud Foundry for PHP developers
 
PaaS Anywhere - Deploying an OpenShift PaaS into your Cloud Provider of Choice
PaaS Anywhere - Deploying an OpenShift PaaS into your Cloud Provider of ChoicePaaS Anywhere - Deploying an OpenShift PaaS into your Cloud Provider of Choice
PaaS Anywhere - Deploying an OpenShift PaaS into your Cloud Provider of Choice
 
Deploying Apps to the Cloud
Deploying Apps to the CloudDeploying Apps to the Cloud
Deploying Apps to the Cloud
 
AWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for Government
 
OpenShift, Docker, Kubernetes: The next generation of PaaS
OpenShift, Docker, Kubernetes: The next generation of PaaSOpenShift, Docker, Kubernetes: The next generation of PaaS
OpenShift, Docker, Kubernetes: The next generation of PaaS
 
Pivoting Spring XD to Spring Cloud Data Flow with Sabby Anandan
Pivoting Spring XD to Spring Cloud Data Flow with Sabby AnandanPivoting Spring XD to Spring Cloud Data Flow with Sabby Anandan
Pivoting Spring XD to Spring Cloud Data Flow with Sabby Anandan
 
Is Serverless The New Swiss Cheese? - AWS Seattle User Group
Is Serverless The New Swiss Cheese? - AWS Seattle User GroupIs Serverless The New Swiss Cheese? - AWS Seattle User Group
Is Serverless The New Swiss Cheese? - AWS Seattle User Group
 
Extending DevOps to Big Data Applications with Kubernetes
Extending DevOps to Big Data Applications with KubernetesExtending DevOps to Big Data Applications with Kubernetes
Extending DevOps to Big Data Applications with Kubernetes
 
AWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for Government
 

More from Guy Korland

Quasi-Linearizability: relaxed consistency for improved concurrency.
Quasi-Linearizability: relaxed consistency for improved concurrency.Quasi-Linearizability: relaxed consistency for improved concurrency.
Quasi-Linearizability: relaxed consistency for improved concurrency.
Guy Korland
 
The Next Generation Application Server – How Event Based Processing yields s...
The Next Generation  Application Server – How Event Based Processing yields s...The Next Generation  Application Server – How Event Based Processing yields s...
The Next Generation Application Server – How Event Based Processing yields s...
Guy Korland
 

More from Guy Korland (14)

FalkorDB - Fastest way to your Knowledge
FalkorDB - Fastest way to your KnowledgeFalkorDB - Fastest way to your Knowledge
FalkorDB - Fastest way to your Knowledge
 
Redis Developer Day TLV - Redis Stack & RedisInsight
Redis Developer Day TLV - Redis Stack & RedisInsightRedis Developer Day TLV - Redis Stack & RedisInsight
Redis Developer Day TLV - Redis Stack & RedisInsight
 
Using Redis As Your Online Feature Store: 2021 Highlights. 2022 Directions
Using Redis As Your  Online Feature Store:  2021 Highlights. 2022 DirectionsUsing Redis As Your  Online Feature Store:  2021 Highlights. 2022 Directions
Using Redis As Your Online Feature Store: 2021 Highlights. 2022 Directions
 
Vector database
Vector databaseVector database
Vector database
 
The evolution of DBaaS - israelcloudsummit
The evolution of DBaaS - israelcloudsummitThe evolution of DBaaS - israelcloudsummit
The evolution of DBaaS - israelcloudsummit
 
From kv to multi model RedisDay NYC19
From kv to multi model   RedisDay NYC19From kv to multi model   RedisDay NYC19
From kv to multi model RedisDay NYC19
 
From Key-Value to Multi-Model - RedisConf19
From Key-Value to Multi-Model - RedisConf19From Key-Value to Multi-Model - RedisConf19
From Key-Value to Multi-Model - RedisConf19
 
Paractical Solutions for Multicore Programming
Paractical Solutions for Multicore ProgrammingParactical Solutions for Multicore Programming
Paractical Solutions for Multicore Programming
 
Crafting a Ready-to-Go STM
Crafting  a Ready-to-Go STMCrafting  a Ready-to-Go STM
Crafting a Ready-to-Go STM
 
Building Scalable Producer-Consumer Pools based on Elimination-Diraction Trees
Building Scalable Producer-Consumer  Pools based on Elimination-Diraction TreesBuilding Scalable Producer-Consumer  Pools based on Elimination-Diraction Trees
Building Scalable Producer-Consumer Pools based on Elimination-Diraction Trees
 
Lowering STM Overhead with Static Analysis
Lowering STM Overhead with Static AnalysisLowering STM Overhead with Static Analysis
Lowering STM Overhead with Static Analysis
 
Quasi-Linearizability: relaxed consistency for improved concurrency.
Quasi-Linearizability: relaxed consistency for improved concurrency.Quasi-Linearizability: relaxed consistency for improved concurrency.
Quasi-Linearizability: relaxed consistency for improved concurrency.
 
The Next Generation Application Server – How Event Based Processing yields s...
The Next Generation  Application Server – How Event Based Processing yields s...The Next Generation  Application Server – How Event Based Processing yields s...
The Next Generation Application Server – How Event Based Processing yields s...
 
Deuce STM - CMP'09
Deuce STM - CMP'09Deuce STM - CMP'09
Deuce STM - CMP'09
 

Recently uploaded

Recently uploaded (20)

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu SubbuApidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 

Open stack bigdata NY cloudcamp

Editor's Notes

  1. Primary and Backup dependency Moving data around for scaling Availability - moving data without losing it. Moving processes to the data rather than the other way around Managing the app as a whole