SlideShare uma empresa Scribd logo
1 de 47
Baixar para ler offline
Streamlining HPC Workloads
with Containers
@DustinKirkland
what does
high-performance computing
look like?
Wikipedia
says...
Or perhaps in China...
Google image search shows...
The
university
student
learns...
HackerNews suggests...
Your DevOps engineer launches...
x1.32xlarge
But
then
there
is
your
real,
actual
data
center...
what do all of these
have in common?
a lot,
actually
they’re all running
Linux
directly on the
bare metal itself
performance is
maximized
overhead is
minimized
big problems are
distributed across a cluster
everyone prefers a
clean environment
virtual machines
always involve overhead
VM MonitorVMXON VMXOFF
Guest
VM EntryVM Exit
oh, and let’s reboot
a datacenter
BIOS is checking memory for problems…
Scanning 1,199,511,627,776 bytes…
This may take several minutes…
Running test 1 of 8: 1.0% complete
Overall test status: 0.1% complete
Time elapsed: 17m23s
Status:
No problems have been found yet.
so let’s have a look at
containers
first,
process containers
awesome for HPC
functions
LXD
second,
machine containers
➢ Ultra fast “vm-lite” guests (bare metal speed)
➢ Any distribution of Linux - e.g. Ubuntu, CentOS
➢ Starts in less than 1 second
➢ 15x density of KVM or ESX for idle workloads
host A
nova-lxd lxc cli
lxdkernel
other restful apps
lxc
machine
LXD REST API
host B
lxc
machine
lxdkernel
host C host D host ...
lxc
machine
lxc
machine
lxc
machine
lxdkernel lxdkernel lxdkernel
➢ Ultra fast “vm-lite” guests (bare metal speed)
➢ Any distribution of Linux - e.g. Ubuntu, CentOS
➢ Starts in less than 1 second
➢ 15x density of KVM or ESX for idle workloads
host A
nova-lxd lxc cli
lxdkernel
other restful apps
LXD REST API
host B
lxdkernel
host C host D host ...
lxdkernel lxdkernel lxdkernel
lxc
machine
lxc
machine
lxc
machine
lxc
machine
lxc
machine
CPU Cores
CPU Cycles
Memory
Disk Space
Disk IO
Network IO
One LXD container,
with 100% of the system:
“alloy” mode
exclusive access
to system resources
but secured from the
underlying hardware and OS
cgroups, user namespaces,
apparmor, seccomp
instant
startup
looks like a machine,
Linux on Linux
zero
latency
zero
overhead
identical
performance
snapshot
restore
live
migration
demo
ubuntu.com/lxd
github.com/lxc
linuxcontainers.org

Mais conteúdo relacionado

Mais procurados

XPDS14: MirageOS 2.0: branch consistency for Xen Stub Domains - Anil Madhavap...
XPDS14: MirageOS 2.0: branch consistency for Xen Stub Domains - Anil Madhavap...XPDS14: MirageOS 2.0: branch consistency for Xen Stub Domains - Anil Madhavap...
XPDS14: MirageOS 2.0: branch consistency for Xen Stub Domains - Anil Madhavap...
The Linux Foundation
 

Mais procurados (20)

Kubernetes 架構與虛擬化之差異
Kubernetes 架構與虛擬化之差異Kubernetes 架構與虛擬化之差異
Kubernetes 架構與虛擬化之差異
 
Containerd: Building a Container Supervisor by Michael Crosby
Containerd: Building a Container Supervisor by Michael CrosbyContainerd: Building a Container Supervisor by Michael Crosby
Containerd: Building a Container Supervisor by Michael Crosby
 
XPDS14: MirageOS 2.0: branch consistency for Xen Stub Domains - Anil Madhavap...
XPDS14: MirageOS 2.0: branch consistency for Xen Stub Domains - Anil Madhavap...XPDS14: MirageOS 2.0: branch consistency for Xen Stub Domains - Anil Madhavap...
XPDS14: MirageOS 2.0: branch consistency for Xen Stub Domains - Anil Madhavap...
 
Open v switch20150410b
Open v switch20150410bOpen v switch20150410b
Open v switch20150410b
 
Ryan Koop's Docker Chicago Meetup Demo March 12 2014
Ryan Koop's Docker Chicago Meetup Demo March 12 2014Ryan Koop's Docker Chicago Meetup Demo March 12 2014
Ryan Koop's Docker Chicago Meetup Demo March 12 2014
 
Docker Athens: Docker Engine Evolution & Containerd Use Cases
Docker Athens: Docker Engine Evolution & Containerd Use CasesDocker Athens: Docker Engine Evolution & Containerd Use Cases
Docker Athens: Docker Engine Evolution & Containerd Use Cases
 
Project Atomic-Nulecule
Project Atomic-NuleculeProject Atomic-Nulecule
Project Atomic-Nulecule
 
All Things Open 2015: DOCKER: EVERYTHING YOU SHOULD KNOW
All Things Open 2015: DOCKER: EVERYTHING YOU SHOULD KNOWAll Things Open 2015: DOCKER: EVERYTHING YOU SHOULD KNOW
All Things Open 2015: DOCKER: EVERYTHING YOU SHOULD KNOW
 
The State of containerd
The State of containerdThe State of containerd
The State of containerd
 
Continuous integration and deployment with docker
Continuous integration and deployment with dockerContinuous integration and deployment with docker
Continuous integration and deployment with docker
 
Live migrating a container: pros, cons and gotchas
Live migrating a container: pros, cons and gotchasLive migrating a container: pros, cons and gotchas
Live migrating a container: pros, cons and gotchas
 
Docker Oxford launch - Introduction to Docker
Docker Oxford launch - Introduction to DockerDocker Oxford launch - Introduction to Docker
Docker Oxford launch - Introduction to Docker
 
Containerd Internals: Building a Core Container Runtime
Containerd Internals: Building a Core Container RuntimeContainerd Internals: Building a Core Container Runtime
Containerd Internals: Building a Core Container Runtime
 
Virtualization inside kubernetes
Virtualization inside kubernetesVirtualization inside kubernetes
Virtualization inside kubernetes
 
Practical CNI
Practical CNIPractical CNI
Practical CNI
 
Containerd Project Update: FOSDEM 2018
Containerd Project Update: FOSDEM 2018Containerd Project Update: FOSDEM 2018
Containerd Project Update: FOSDEM 2018
 
Distributed Version Control Systems
Distributed Version Control SystemsDistributed Version Control Systems
Distributed Version Control Systems
 
Dev opsmeetup sept2013-leaseweb
Dev opsmeetup sept2013-leasewebDev opsmeetup sept2013-leaseweb
Dev opsmeetup sept2013-leaseweb
 
Containerize! Between Docker and Jube.
Containerize! Between Docker and Jube.Containerize! Between Docker and Jube.
Containerize! Between Docker and Jube.
 
AppSec USA 2014 talk by Chris Swan "Implications & Opportunities at the Bleed...
AppSec USA 2014 talk by Chris Swan "Implications & Opportunities at the Bleed...AppSec USA 2014 talk by Chris Swan "Implications & Opportunities at the Bleed...
AppSec USA 2014 talk by Chris Swan "Implications & Opportunities at the Bleed...
 

Destaque

Destaque (20)

[Container world 2017] The Questions You're Afraid to Ask about Containers
[Container world 2017] The Questions You're Afraid to Ask about Containers[Container world 2017] The Questions You're Afraid to Ask about Containers
[Container world 2017] The Questions You're Afraid to Ask about Containers
 
Ubuntu 16.04 LTS Security Features
Ubuntu 16.04 LTS Security FeaturesUbuntu 16.04 LTS Security Features
Ubuntu 16.04 LTS Security Features
 
Open ZFS Keynote (public)
Open ZFS Keynote (public)Open ZFS Keynote (public)
Open ZFS Keynote (public)
 
What HPC can learn from DevOps?
What HPC can learn from DevOps?What HPC can learn from DevOps?
What HPC can learn from DevOps?
 
Openstack Summit Container Day Keynote
Openstack Summit Container Day KeynoteOpenstack Summit Container Day Keynote
Openstack Summit Container Day Keynote
 
Managing Container Clusters in OpenStack Native Way
Managing Container Clusters in OpenStack Native WayManaging Container Clusters in OpenStack Native Way
Managing Container Clusters in OpenStack Native Way
 
Webinar container management in OpenStack
Webinar container management in OpenStackWebinar container management in OpenStack
Webinar container management in OpenStack
 
Cloud init and cloud provisioning [openstack summit vancouver]
Cloud init and cloud provisioning [openstack summit vancouver]Cloud init and cloud provisioning [openstack summit vancouver]
Cloud init and cloud provisioning [openstack summit vancouver]
 
Open Container Technologies and OpenStack - Sorting Through Kubernetes, the O...
Open Container Technologies and OpenStack - Sorting Through Kubernetes, the O...Open Container Technologies and OpenStack - Sorting Through Kubernetes, the O...
Open Container Technologies and OpenStack - Sorting Through Kubernetes, the O...
 
Architecting Ceph Solutions
Architecting Ceph SolutionsArchitecting Ceph Solutions
Architecting Ceph Solutions
 
My SQL and Ceph: Head-to-Head Performance Lab
My SQL and Ceph: Head-to-Head Performance LabMy SQL and Ceph: Head-to-Head Performance Lab
My SQL and Ceph: Head-to-Head Performance Lab
 
TUT18972: Unleash the power of Ceph across the Data Center
TUT18972: Unleash the power of Ceph across the Data CenterTUT18972: Unleash the power of Ceph across the Data Center
TUT18972: Unleash the power of Ceph across the Data Center
 
OpenStack Magnum
OpenStack MagnumOpenStack Magnum
OpenStack Magnum
 
Container World 2017!
Container World 2017!Container World 2017!
Container World 2017!
 
Container World 2017 - Characterizing and Contrasting Container Orchestrators
Container World 2017 - Characterizing and Contrasting Container OrchestratorsContainer World 2017 - Characterizing and Contrasting Container Orchestrators
Container World 2017 - Characterizing and Contrasting Container Orchestrators
 
SoCal DevOps Meetup 1/26/2017 - Habitat by Chef
SoCal DevOps Meetup 1/26/2017 - Habitat by ChefSoCal DevOps Meetup 1/26/2017 - Habitat by Chef
SoCal DevOps Meetup 1/26/2017 - Habitat by Chef
 
Container Camp London (2016-09-09)
Container Camp London (2016-09-09)Container Camp London (2016-09-09)
Container Camp London (2016-09-09)
 
Shifter: Containers in HPC Environments
Shifter: Containers in HPC EnvironmentsShifter: Containers in HPC Environments
Shifter: Containers in HPC Environments
 
LAS16-211: Using LAVA V2 for advanced KVM testing
LAS16-211: Using LAVA V2 for advanced KVM testingLAS16-211: Using LAVA V2 for advanced KVM testing
LAS16-211: Using LAVA V2 for advanced KVM testing
 
HPC Storage Appliances for the Enterpris
HPC Storage Appliances for the EnterprisHPC Storage Appliances for the Enterpris
HPC Storage Appliances for the Enterpris
 

Semelhante a Streamlining HPC Workloads with Containers

LXC Docker and the Future of Software Delivery
LXC Docker and the Future of Software DeliveryLXC Docker and the Future of Software Delivery
LXC Docker and the Future of Software Delivery
Docker, Inc.
 
node.js, javascript and the future
node.js, javascript and the futurenode.js, javascript and the future
node.js, javascript and the future
Jeff Miccolis
 
The Ultimate Deobfuscator - ToorCON San Diego 2008
The Ultimate Deobfuscator - ToorCON San Diego 2008The Ultimate Deobfuscator - ToorCON San Diego 2008
The Ultimate Deobfuscator - ToorCON San Diego 2008
Stephan Chenette
 
ClickOS_EE80777777777777777777777777777.pptx
ClickOS_EE80777777777777777777777777777.pptxClickOS_EE80777777777777777777777777777.pptx
ClickOS_EE80777777777777777777777777777.pptx
BiHongPhc
 

Semelhante a Streamlining HPC Workloads with Containers (20)

Kubernetes: My BFF
Kubernetes: My BFFKubernetes: My BFF
Kubernetes: My BFF
 
Lessons learnt on a 2000-core cluster
Lessons learnt on a 2000-core clusterLessons learnt on a 2000-core cluster
Lessons learnt on a 2000-core cluster
 
LXC, Docker, and the future of software delivery | LinuxCon 2013
LXC, Docker, and the future of software delivery | LinuxCon 2013LXC, Docker, and the future of software delivery | LinuxCon 2013
LXC, Docker, and the future of software delivery | LinuxCon 2013
 
LXC Docker and the Future of Software Delivery
LXC Docker and the Future of Software DeliveryLXC Docker and the Future of Software Delivery
LXC Docker and the Future of Software Delivery
 
Deep Dive on Amazon EC2 instances
Deep Dive on Amazon EC2 instancesDeep Dive on Amazon EC2 instances
Deep Dive on Amazon EC2 instances
 
Erik Skytthe - Monitoring Mesos, Docker, Containers with Zabbix | ZabConf2016
Erik Skytthe - Monitoring Mesos, Docker, Containers with Zabbix | ZabConf2016Erik Skytthe - Monitoring Mesos, Docker, Containers with Zabbix | ZabConf2016
Erik Skytthe - Monitoring Mesos, Docker, Containers with Zabbix | ZabConf2016
 
Containers > VMs
Containers > VMsContainers > VMs
Containers > VMs
 
Microkernels and Beyond
Microkernels and BeyondMicrokernels and Beyond
Microkernels and Beyond
 
LinuxCon 2011: OpenVZ and Linux Kernel Testing
LinuxCon 2011: OpenVZ and Linux Kernel TestingLinuxCon 2011: OpenVZ and Linux Kernel Testing
LinuxCon 2011: OpenVZ and Linux Kernel Testing
 
LinuxCon 2011: OpenVZ and Linux Kernel Testing
LinuxCon 2011: OpenVZ and Linux Kernel TestingLinuxCon 2011: OpenVZ and Linux Kernel Testing
LinuxCon 2011: OpenVZ and Linux Kernel Testing
 
Virtual Machines Security Internals: Detection and Exploitation
 Virtual Machines Security Internals: Detection and Exploitation Virtual Machines Security Internals: Detection and Exploitation
Virtual Machines Security Internals: Detection and Exploitation
 
MunichJS - 2011-04-06
MunichJS - 2011-04-06MunichJS - 2011-04-06
MunichJS - 2011-04-06
 
node.js, javascript and the future
node.js, javascript and the futurenode.js, javascript and the future
node.js, javascript and the future
 
The Ultimate Deobfuscator - ToorCON San Diego 2008
The Ultimate Deobfuscator - ToorCON San Diego 2008The Ultimate Deobfuscator - ToorCON San Diego 2008
The Ultimate Deobfuscator - ToorCON San Diego 2008
 
Operating System Multiple Choice Questions
Operating System Multiple Choice QuestionsOperating System Multiple Choice Questions
Operating System Multiple Choice Questions
 
ClickOS_EE80777777777777777777777777777.pptx
ClickOS_EE80777777777777777777777777777.pptxClickOS_EE80777777777777777777777777777.pptx
ClickOS_EE80777777777777777777777777777.pptx
 
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
 
Docker & Daily DevOps
Docker & Daily DevOpsDocker & Daily DevOps
Docker & Daily DevOps
 
Docker and-daily-devops
Docker and-daily-devopsDocker and-daily-devops
Docker and-daily-devops
 
Docker's Jérôme Petazzoni: Best Practices in Dev to Production Parity for Con...
Docker's Jérôme Petazzoni: Best Practices in Dev to Production Parity for Con...Docker's Jérôme Petazzoni: Best Practices in Dev to Production Parity for Con...
Docker's Jérôme Petazzoni: Best Practices in Dev to Production Parity for Con...
 

Último

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Último (20)

Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 

Streamlining HPC Workloads with Containers