OpenStack is an open source cloud computing platform that provides infrastructure as a service. It supports various virtualization technologies and operating systems. Key components include Nova (compute), Swift (object storage), Glance (image registry), Neutron (networking), and Horizon (dashboard/GUI). OpenStack is deployed across many large companies and used both privately and publicly by organizations like Rackspace, HP, IBM, Cisco, and more. It provides an alternative to proprietary cloud platforms like AWS and works with various hypervisors, storage backends, and networking technologies.
Build a Cloud Day presentation about Fuse Fabric technology in the cloud and how integration projects / architectures can be designed top of cloudstack, openstack, amazon, ...
Summary of past Cassandra benchmarks performed by Netflix and description of how Netflix uses Cassandra interspersed with a live demo automated using Jenkins and Jmeter that created two 12 node Cassandra clusters from scratch on AWS, one with regular disks and one with SSDs. Both clusters were scaled up to 24 nodes each during the demo.
OpenStack is open source software for building private and public clouds. It provides capabilities for provisioning VMs on demand, managing volumes and networks, and enabling multi-tenancy and quotas. It consists of several projects including Nova (compute), Glance (images), Swift (object storage), Keystone (identity), Horizon (dashboard), Quantum/Neutron (networking), and Cinder (block storage). When a user requests a new VM via the dashboard, several OpenStack components work together to authenticate the request, schedule the VM, and provision it on a compute node using the hypervisor.
Supporting and Using EC2/CIMI on top of Cloud Environments via DeltacloudOved Ourfali
Deltacloud is an open source project that abstracts differences between cloud providers and supports EC2 and CIMI APIs. It can be used on top of oVirt to provide EC2 and CIMI functionality. The Deltacloud server is installed and configured to use the oVirt driver. Then basic EC2 and CIMI operations like launching VMs, listing images, starting/stopping VMs can be performed by sending HTTP requests to the Deltacloud server endpoints. This allows existing tools that use EC2 or CIMI to manage VMs running on oVirt. Future work includes better support for additional oVirt and cloud API features.
Storage 101: Rook and Ceph - Open Infrastructure Denver 2019Sean Cohen
Starting from the basics, we explore the advantages of using Rook as a Storage operator to serve Ceph storage, the leading Software-Defined Storage platform in the Open Source world. Ceph automates the internal storage management, while Rook automates the user-facing operations and effectively turns a storage technology into a service transparent to the user. The combination delivers an impressive improvement in UX and provides the ideal storage platform for Kubernetes.
A comprehensive examination of use cases and open problems will complement our review of the Rook architecture. We will deep-dive into what Rook does well, what it does not do (yet), and what trade-offs using a storage operator involves operationally. With live access to a running cluster, we will showcase Rook in action as we discuss its capabilities.
https://www.openstack.org/summit/denver-2019/summit-schedule/events/23515/storage-101-rook-and-ceph
This deep dive will address the questions on how to install, deploy and operate OpenStack by providing informative slides which will help users get ahead start with this awesome project
Apache CloudStack is an open source cloud computing platform that provides infrastructure as a service. It was originally developed by Citrix and is now an Apache incubator project. It is hypervisor-agnostic and supports platforms like Xen, KVM, and VMware. It provides services for compute, storage, networking and a web-based user interface.
OpenStack is an open source cloud computing platform that provides infrastructure as a service. It supports various virtualization technologies and operating systems. Key components include Nova (compute), Swift (object storage), Glance (image registry), Neutron (networking), and Horizon (dashboard/GUI). OpenStack is deployed across many large companies and used both privately and publicly by organizations like Rackspace, HP, IBM, Cisco, and more. It provides an alternative to proprietary cloud platforms like AWS and works with various hypervisors, storage backends, and networking technologies.
Build a Cloud Day presentation about Fuse Fabric technology in the cloud and how integration projects / architectures can be designed top of cloudstack, openstack, amazon, ...
Summary of past Cassandra benchmarks performed by Netflix and description of how Netflix uses Cassandra interspersed with a live demo automated using Jenkins and Jmeter that created two 12 node Cassandra clusters from scratch on AWS, one with regular disks and one with SSDs. Both clusters were scaled up to 24 nodes each during the demo.
OpenStack is open source software for building private and public clouds. It provides capabilities for provisioning VMs on demand, managing volumes and networks, and enabling multi-tenancy and quotas. It consists of several projects including Nova (compute), Glance (images), Swift (object storage), Keystone (identity), Horizon (dashboard), Quantum/Neutron (networking), and Cinder (block storage). When a user requests a new VM via the dashboard, several OpenStack components work together to authenticate the request, schedule the VM, and provision it on a compute node using the hypervisor.
Supporting and Using EC2/CIMI on top of Cloud Environments via DeltacloudOved Ourfali
Deltacloud is an open source project that abstracts differences between cloud providers and supports EC2 and CIMI APIs. It can be used on top of oVirt to provide EC2 and CIMI functionality. The Deltacloud server is installed and configured to use the oVirt driver. Then basic EC2 and CIMI operations like launching VMs, listing images, starting/stopping VMs can be performed by sending HTTP requests to the Deltacloud server endpoints. This allows existing tools that use EC2 or CIMI to manage VMs running on oVirt. Future work includes better support for additional oVirt and cloud API features.
Storage 101: Rook and Ceph - Open Infrastructure Denver 2019Sean Cohen
Starting from the basics, we explore the advantages of using Rook as a Storage operator to serve Ceph storage, the leading Software-Defined Storage platform in the Open Source world. Ceph automates the internal storage management, while Rook automates the user-facing operations and effectively turns a storage technology into a service transparent to the user. The combination delivers an impressive improvement in UX and provides the ideal storage platform for Kubernetes.
A comprehensive examination of use cases and open problems will complement our review of the Rook architecture. We will deep-dive into what Rook does well, what it does not do (yet), and what trade-offs using a storage operator involves operationally. With live access to a running cluster, we will showcase Rook in action as we discuss its capabilities.
https://www.openstack.org/summit/denver-2019/summit-schedule/events/23515/storage-101-rook-and-ceph
This deep dive will address the questions on how to install, deploy and operate OpenStack by providing informative slides which will help users get ahead start with this awesome project
Apache CloudStack is an open source cloud computing platform that provides infrastructure as a service. It was originally developed by Citrix and is now an Apache incubator project. It is hypervisor-agnostic and supports platforms like Xen, KVM, and VMware. It provides services for compute, storage, networking and a web-based user interface.
Enhanced introduction to CloudStack for the Geneva Java User Group. Includes Apache processes, DevCloud use cases, opportunities for Java developers and 4.0 release testing procedure.
Ceph storage for ocp deploying and managing ceph on top of open shift conta...OrFriedmann
Ceph is an open-source software-defined storage solution that provides unified block, file, and object storage. It uses a distributed cluster of storage nodes and microservice daemons to store and retrieve data with no single point of failure. Rook is an open-source project that provides storage orchestration for Kubernetes and allows easy deployment and management of Ceph clusters on Kubernetes through custom resources like CephCluster, CephBlockPool, and CephObjectStore.
Ritesh Nanda and Syed Armani are cloud architects who discuss OpenStack, an open source cloud computing platform. OpenStack provides infrastructure as a service and allows users to manage compute, storage, and networking resources. Key OpenStack components include Nova (compute), Swift (object storage), Glance (images), Keystone (identity), Horizon (dashboard), Quantum/Neutron (networking), Cinder (block storage), and Ceilometer (telemetry). The architects describe the purpose and architecture of these components. They conclude that OpenStack is well-suited for private, public, and hybrid clouds and is being adopted by enterprises.
The container revolution, and what it means to operators.pptxRobert Starmer
The document discusses the rise of containers as a DevOps technology that accelerates the development process. It provides a brief history of containers, explaining how Docker simplified their use. Containers allow for faster development cycles than VMs by providing process-level segregation. While containers abstract operations, container management platforms are still needed to provide scaling, scheduling, security and other operational functions. The document also discusses how OpenStack can manage containers running on VMs, bare metal or directly, and how containers are increasingly being used to deploy OpenStack services themselves.
A quick intro to DevCloud the CloudStack sandbox, and how to use CloudMonkey to manage your cloud.
DevCloud is a virtualbox image that contains the CloudStack source code and that is setup to run the storage infrastructure needed by CloudStack plus the networking setup to build the guest network of the VMs. Tiny Linux instances can be started within the Devcloud VM making use of nested virtualization.
This is a perfect setup to discover cloudstack, give demos and test new codes. It is used to test new releases and verify basic functionality. You can run DevCloud on your laptop and then use the command line interface CloudMonkey to make API calls to your DevCloud instance.
This is the perfect complement to the talk on CloudMonkey and shows the basic functionality of a cloud. Instance creation, snapshots, networking, network offering and AWS EC2 compatibility.
My talk from BACD http://buildacloud.org workshop in Ghent, Belgium
All videos can be viewed at: http://www.youtube.com/playlist?list=PLb899uhkHRoZZefRW5XmCb8QBcRO7o74E
This is an introductory talk for the workshop, it introduces CloudStack and the community at the Apache Software Foundation, it presents the basic layers of the Cloud IaaS, PaaS, and SaaS and shows how the CloudStack ecosystem addresses all layers. It presents the basic features of cloudstack, networking with a focus on SDN (Software Defined Networking) , storage with a focus on large scale object store (Ceph), a use case with Spotify, a PaaS with Karafe and fuse Fabric, the API using deltacloud which provides the CIMI standard interface and an application integration using the CloudStack API with Activeeon.
This is the perfect complement to the videos on youtube and serves as a introduction to CloudStack.
New Ceph capabilities and Reference ArchitecturesKamesh Pemmaraju
Have you heard about Inktank Ceph and are interested to learn some tips and tricks for getting started quickly and efficiently with Ceph? Then this is the session for you!
In this two part session you learn details of:
• the very latest enhancements and capabilities delivered in Inktank Ceph Enterprise such as a new erasure coded storage back-end, support for tiering, and the introduction of user quotas.
• best practices, lessons learned and architecture considerations founded in real customer deployments of Dell and Inktank Ceph solutions that will help accelerate your Ceph deployment.
Ceph is an open source project, which provides software-defined, unified storage solutions. Ceph is a distributed storage system which is massively scalable and high-performing without any single point of failure. From the roots, it has been designed to be highly scalable, up to exabyte level and beyond while running on general-purpose commodity hardware.
Rook turns distributed storage systems into self-managing, self-scaling, self-healing storage services. It automates the tasks of a storage administrator: deployment, bootstrapping, configuration, provisioning, scaling, upgrading, migration, disaster recovery, monitoring, and resource management.
Rook uses the power of the Kubernetes platform to deliver its services via a Kubernetes Operator for each storage provider.
Oleg Chunikhin, Co-Founder and CTO @ Kublr.com, will present an introduction to storage management on k8s using Rook and Ceph.
Introduction to Ceph, an open-source, massively scalable distributed file system.
This document explains the architecture of Ceph and integration with OpenStack.
Kubernetes Day 2017 - Build, Ship and Run Your APP, Production !!smalltown
This document summarizes a talk about building, shipping, and running applications in production using containers on AWS. It discusses migrating an existing service from an on-premise data center to AWS, refactoring the application into microservices and containerizing it using Docker. It then covers setting up a Kubernetes cluster on CoreOS to orchestrate the containers across AWS, addressing challenges like application state, updates and monitoring. Terraform is presented as a way to define infrastructure as code and provision AWS resources. Logging, metrics collection and monitoring the Kubernetes cluster are also discussed.
QCT Ceph Solution - Design Consideration and Reference ArchitecturePatrick McGarry
This document discusses QCT's Ceph storage solutions, including an overview of Ceph architecture, QCT hardware platforms, Red Hat Ceph software, workload considerations, reference architectures, test results and a QCT/Red Hat whitepaper. It provides technical details on QCT's throughput-optimized and capacity-optimized solutions and shows how they address different storage needs through workload-driven design. Hands-on testing and a test drive lab are offered to explore Ceph features and configurations.
Ceph is a open source , software defined storage excellent and the only ( i would say ) storage backend as a cloud storage. Ceph is the Future of Storage. In this presentation i am explaining ceph and openstack briefly , you would definitely enjoy it.
Storage tiering and erasure coding in Ceph (SCaLE13x)Sage Weil
Ceph is designed around the assumption that all components of the system (disks, hosts, networks) can fail, and has traditionally leveraged replication to provide data durability and reliability. The CRUSH placement algorithm is used to allow failure domains to be defined across hosts, racks, rows, or datacenters, depending on the deployment scale and requirements.
Recent releases have added support for erasure coding, which can provide much higher data durability and lower storage overheads. However, in practice erasure codes have different performance characteristics than traditional replication and, under some workloads, come at some expense. At the same time, we have introduced a storage tiering infrastructure and cache pools that allow alternate hardware backends (like high-end flash) to be leveraged for active data sets while cold data are transparently migrated to slower backends. The combination of these two features enables a surprisingly broad range of new applications and deployment configurations.
This talk will cover a few Ceph fundamentals, discuss the new tiering and erasure coding features, and then discuss a variety of ways that the new capabilities can be leveraged.
This document provides an overview and summary of Red Hat Storage and Inktank Ceph. It discusses Red Hat acquiring Inktank Ceph in April 2014 and the future of Red Hat Storage having two flavors - Gluster edition and Ceph edition. Key features of Red Hat Storage 3.0 include enhanced data protection with snapshots, cluster monitoring, and deep Hadoop integration. The document also introduces Inktank Ceph Enterprise v1.2 and discusses Ceph components like RADOS, LIBRADOS, RBD, RGW and how Ceph can be used with OpenStack.
Constructing Web APIs with Rack, Sinatra and MongoDBOisin Hurley
Slides for my talk at Ruby Ireland on 10 May 11. Showing some of the capabilities of mongoDB, using it from a Sinatra applications and deploying it to Heroku and Cloud Foundry
Terraform modules provide reusable, composable infrastructure components. The document discusses restructuring infrastructure code into modules to make it more reusable, testable, and maintainable. Key points include:
- Modules should be structured in a three-tier hierarchy from primitive resources to generic services to specific environments.
- Testing modules individually increases confidence in changes.
- Storing module code and versions in Git provides versioning and collaboration.
- Remote state allows infrastructure to be shared between modules and deployments.
1. The document discusses how OpenStack can be used to build private and hybrid clouds for enterprises using open source technology free from vendor lock-in.
2. It provides examples of how OpenStack can enable continuous software delivery, cloud-enable applications, and provide IT as a service while reducing reliance on proprietary virtualization.
3. Asdtech offers turnkey OpenStack services including consultancy, cloud setup, custom development, migration, support and training to help enterprises orchestrate their existing infrastructure or build new clouds.
This document provides an overview of Kubernetes and its components. It discusses the Go programming language features used in Kubernetes. It also describes how Kubernetes is architected, including the kube-apiserver, kube-scheduler, Kubelet, reconciliation process, and networking with Flannel. The presenter is Anseungkyu who worked on OpenStack private clouds and is now the deputy representative for OpenStack Korea.
The document discusses the principles, architecture, and practice of Swift object storage at SinaAppEngine. It describes how Swift provides high reliability through consistent hashing, replication across multiple zones, and an eventual consistency model. It outlines SinaAppEngine's implementation of Swift for storage, including authentication, quotas, and domain remapping. Problems addressed include improving replication efficiency and SQLite performance as well as controlling rsync bandwidth. The document provides an overview of how Swift storage works at scale.
Swift Architecture and Practice, by Alex YangHui Cheng
The document discusses the principles, architecture, and practice of Swift object storage at SinaAppEngine. It describes how Swift provides high reliability through consistent hashing, replication across multiple zones, and an eventual consistency model. It outlines SinaAppEngine's implementation of Swift for storage, including authentication, quotas, and domain remapping. Problems addressed include improving replication efficiency and SQLite performance as well as controlling rsync bandwidth. The document provides an overview of how Swift storage works at scale.
Enhanced introduction to CloudStack for the Geneva Java User Group. Includes Apache processes, DevCloud use cases, opportunities for Java developers and 4.0 release testing procedure.
Ceph storage for ocp deploying and managing ceph on top of open shift conta...OrFriedmann
Ceph is an open-source software-defined storage solution that provides unified block, file, and object storage. It uses a distributed cluster of storage nodes and microservice daemons to store and retrieve data with no single point of failure. Rook is an open-source project that provides storage orchestration for Kubernetes and allows easy deployment and management of Ceph clusters on Kubernetes through custom resources like CephCluster, CephBlockPool, and CephObjectStore.
Ritesh Nanda and Syed Armani are cloud architects who discuss OpenStack, an open source cloud computing platform. OpenStack provides infrastructure as a service and allows users to manage compute, storage, and networking resources. Key OpenStack components include Nova (compute), Swift (object storage), Glance (images), Keystone (identity), Horizon (dashboard), Quantum/Neutron (networking), Cinder (block storage), and Ceilometer (telemetry). The architects describe the purpose and architecture of these components. They conclude that OpenStack is well-suited for private, public, and hybrid clouds and is being adopted by enterprises.
The container revolution, and what it means to operators.pptxRobert Starmer
The document discusses the rise of containers as a DevOps technology that accelerates the development process. It provides a brief history of containers, explaining how Docker simplified their use. Containers allow for faster development cycles than VMs by providing process-level segregation. While containers abstract operations, container management platforms are still needed to provide scaling, scheduling, security and other operational functions. The document also discusses how OpenStack can manage containers running on VMs, bare metal or directly, and how containers are increasingly being used to deploy OpenStack services themselves.
A quick intro to DevCloud the CloudStack sandbox, and how to use CloudMonkey to manage your cloud.
DevCloud is a virtualbox image that contains the CloudStack source code and that is setup to run the storage infrastructure needed by CloudStack plus the networking setup to build the guest network of the VMs. Tiny Linux instances can be started within the Devcloud VM making use of nested virtualization.
This is a perfect setup to discover cloudstack, give demos and test new codes. It is used to test new releases and verify basic functionality. You can run DevCloud on your laptop and then use the command line interface CloudMonkey to make API calls to your DevCloud instance.
This is the perfect complement to the talk on CloudMonkey and shows the basic functionality of a cloud. Instance creation, snapshots, networking, network offering and AWS EC2 compatibility.
My talk from BACD http://buildacloud.org workshop in Ghent, Belgium
All videos can be viewed at: http://www.youtube.com/playlist?list=PLb899uhkHRoZZefRW5XmCb8QBcRO7o74E
This is an introductory talk for the workshop, it introduces CloudStack and the community at the Apache Software Foundation, it presents the basic layers of the Cloud IaaS, PaaS, and SaaS and shows how the CloudStack ecosystem addresses all layers. It presents the basic features of cloudstack, networking with a focus on SDN (Software Defined Networking) , storage with a focus on large scale object store (Ceph), a use case with Spotify, a PaaS with Karafe and fuse Fabric, the API using deltacloud which provides the CIMI standard interface and an application integration using the CloudStack API with Activeeon.
This is the perfect complement to the videos on youtube and serves as a introduction to CloudStack.
New Ceph capabilities and Reference ArchitecturesKamesh Pemmaraju
Have you heard about Inktank Ceph and are interested to learn some tips and tricks for getting started quickly and efficiently with Ceph? Then this is the session for you!
In this two part session you learn details of:
• the very latest enhancements and capabilities delivered in Inktank Ceph Enterprise such as a new erasure coded storage back-end, support for tiering, and the introduction of user quotas.
• best practices, lessons learned and architecture considerations founded in real customer deployments of Dell and Inktank Ceph solutions that will help accelerate your Ceph deployment.
Ceph is an open source project, which provides software-defined, unified storage solutions. Ceph is a distributed storage system which is massively scalable and high-performing without any single point of failure. From the roots, it has been designed to be highly scalable, up to exabyte level and beyond while running on general-purpose commodity hardware.
Rook turns distributed storage systems into self-managing, self-scaling, self-healing storage services. It automates the tasks of a storage administrator: deployment, bootstrapping, configuration, provisioning, scaling, upgrading, migration, disaster recovery, monitoring, and resource management.
Rook uses the power of the Kubernetes platform to deliver its services via a Kubernetes Operator for each storage provider.
Oleg Chunikhin, Co-Founder and CTO @ Kublr.com, will present an introduction to storage management on k8s using Rook and Ceph.
Introduction to Ceph, an open-source, massively scalable distributed file system.
This document explains the architecture of Ceph and integration with OpenStack.
Kubernetes Day 2017 - Build, Ship and Run Your APP, Production !!smalltown
This document summarizes a talk about building, shipping, and running applications in production using containers on AWS. It discusses migrating an existing service from an on-premise data center to AWS, refactoring the application into microservices and containerizing it using Docker. It then covers setting up a Kubernetes cluster on CoreOS to orchestrate the containers across AWS, addressing challenges like application state, updates and monitoring. Terraform is presented as a way to define infrastructure as code and provision AWS resources. Logging, metrics collection and monitoring the Kubernetes cluster are also discussed.
QCT Ceph Solution - Design Consideration and Reference ArchitecturePatrick McGarry
This document discusses QCT's Ceph storage solutions, including an overview of Ceph architecture, QCT hardware platforms, Red Hat Ceph software, workload considerations, reference architectures, test results and a QCT/Red Hat whitepaper. It provides technical details on QCT's throughput-optimized and capacity-optimized solutions and shows how they address different storage needs through workload-driven design. Hands-on testing and a test drive lab are offered to explore Ceph features and configurations.
Ceph is a open source , software defined storage excellent and the only ( i would say ) storage backend as a cloud storage. Ceph is the Future of Storage. In this presentation i am explaining ceph and openstack briefly , you would definitely enjoy it.
Storage tiering and erasure coding in Ceph (SCaLE13x)Sage Weil
Ceph is designed around the assumption that all components of the system (disks, hosts, networks) can fail, and has traditionally leveraged replication to provide data durability and reliability. The CRUSH placement algorithm is used to allow failure domains to be defined across hosts, racks, rows, or datacenters, depending on the deployment scale and requirements.
Recent releases have added support for erasure coding, which can provide much higher data durability and lower storage overheads. However, in practice erasure codes have different performance characteristics than traditional replication and, under some workloads, come at some expense. At the same time, we have introduced a storage tiering infrastructure and cache pools that allow alternate hardware backends (like high-end flash) to be leveraged for active data sets while cold data are transparently migrated to slower backends. The combination of these two features enables a surprisingly broad range of new applications and deployment configurations.
This talk will cover a few Ceph fundamentals, discuss the new tiering and erasure coding features, and then discuss a variety of ways that the new capabilities can be leveraged.
This document provides an overview and summary of Red Hat Storage and Inktank Ceph. It discusses Red Hat acquiring Inktank Ceph in April 2014 and the future of Red Hat Storage having two flavors - Gluster edition and Ceph edition. Key features of Red Hat Storage 3.0 include enhanced data protection with snapshots, cluster monitoring, and deep Hadoop integration. The document also introduces Inktank Ceph Enterprise v1.2 and discusses Ceph components like RADOS, LIBRADOS, RBD, RGW and how Ceph can be used with OpenStack.
Constructing Web APIs with Rack, Sinatra and MongoDBOisin Hurley
Slides for my talk at Ruby Ireland on 10 May 11. Showing some of the capabilities of mongoDB, using it from a Sinatra applications and deploying it to Heroku and Cloud Foundry
Terraform modules provide reusable, composable infrastructure components. The document discusses restructuring infrastructure code into modules to make it more reusable, testable, and maintainable. Key points include:
- Modules should be structured in a three-tier hierarchy from primitive resources to generic services to specific environments.
- Testing modules individually increases confidence in changes.
- Storing module code and versions in Git provides versioning and collaboration.
- Remote state allows infrastructure to be shared between modules and deployments.
1. The document discusses how OpenStack can be used to build private and hybrid clouds for enterprises using open source technology free from vendor lock-in.
2. It provides examples of how OpenStack can enable continuous software delivery, cloud-enable applications, and provide IT as a service while reducing reliance on proprietary virtualization.
3. Asdtech offers turnkey OpenStack services including consultancy, cloud setup, custom development, migration, support and training to help enterprises orchestrate their existing infrastructure or build new clouds.
This document provides an overview of Kubernetes and its components. It discusses the Go programming language features used in Kubernetes. It also describes how Kubernetes is architected, including the kube-apiserver, kube-scheduler, Kubelet, reconciliation process, and networking with Flannel. The presenter is Anseungkyu who worked on OpenStack private clouds and is now the deputy representative for OpenStack Korea.
The document discusses the principles, architecture, and practice of Swift object storage at SinaAppEngine. It describes how Swift provides high reliability through consistent hashing, replication across multiple zones, and an eventual consistency model. It outlines SinaAppEngine's implementation of Swift for storage, including authentication, quotas, and domain remapping. Problems addressed include improving replication efficiency and SQLite performance as well as controlling rsync bandwidth. The document provides an overview of how Swift storage works at scale.
Swift Architecture and Practice, by Alex YangHui Cheng
The document discusses the principles, architecture, and practice of Swift object storage at SinaAppEngine. It describes how Swift provides high reliability through consistent hashing, replication across multiple zones, and an eventual consistency model. It outlines SinaAppEngine's implementation of Swift for storage, including authentication, quotas, and domain remapping. Problems addressed include improving replication efficiency and SQLite performance as well as controlling rsync bandwidth. The document provides an overview of how Swift storage works at scale.
Swift is an object storage software used at SinaAppEngine for storage. It uses consistent hashing to distribute objects uniformly across multiple servers for high reliability and availability. Data is replicated across multiple servers and zones to protect against hardware failures. Consistency is ensured through a quorum-based protocol where writes require acknowledgment from a majority of replicas and reads require a majority to respond. Problems include inefficient replication that impacts performance and availability, and improvements involve optimizing replication to run during idle times and adding nodes for scaling out.
OpenStack is an open source cloud operating system that controls pools of compute, storage, and networking resources throughout a datacenter, managed through APIs, dashboards, and command-line tools. It includes several core projects like Nova for compute resources, Swift for object storage, and Glance for images. OpenStack is developed as open source by an international community of over 1,600 individuals from over 100 companies and is used in production private and public clouds by companies in industries like financial services, e-commerce, and more.
- CloudStack is an open source cloud computing platform that was donated to the Apache Software Foundation in 2012. It provides infrastructure as a service and supports various hypervisors and physical hardware.
- CloudStack has a scalable architecture designed to support thousands of hosts and VMs across multiple availability zones. It provides rich networking and storage capabilities.
- CloudStack can support both traditional server virtualization workloads as well as "Amazon-style" workloads with software defined networks and object storage.
- The CloudStack community is growing rapidly and encourages participation through mailing lists, IRC, forums and meetup groups.
What is OpenStack and the added value of IBM solutionsSasha Lazarevic
OpenStack has become de-facto standard for private cloud implementations. This is presentation of OpenStack basics, with a conclusion that can be valuable to professional services. I recommend the clients to pay attention to IBM's value-added solutions like Cloud Manager and Cloud Orchestrator.
OpenStack is an open source cloud project and community with broad commercial and developer support. OpenStack is currently developing two interrelated technologies: OpenStack Compute and OpenStack Object Storage. OpenStack Compute is the internal fabric of the cloud creating and managing large groups of virtual private servers and OpenStack Object Storage is software for creating redundant, scalable object storage using clusters of commodity servers to store terabytes or even petabytes of data. In this tutorial, Bret Piatt will explain how to deploy OpenStack Compute and Object Storage, including an overview of the architecture and technology requirements.
The document outlines an agenda for a CloudStack developer day, including presentations on what CloudStack is, its deployment architecture, networking features, software architecture, integration capabilities, and how to contribute to the Apache CloudStack community. The key topics will be an introduction to CloudStack, an overview of its basics and deployment architecture including networking, a discussion of its current and future software architecture, and sessions on UI customization, the API, and how to get involved in the Apache CloudStack project.
This talk covered the OpenStack basics that VMware Administrators need to be aware of to be successful in their deployments. We also had the Tesora team join us on stage to discuss the importance of Database-as-a-Service with the Trove project!
The document summarizes CloudStack architecture plans for the future. It discusses moving to management server clusters per availability zone rather than per region. It also discusses using an object storage system for templates and snapshots rather than a separate NFS server. Finally, it discusses a possible future model where CloudStack manages existing virtualization clusters rather than deploying and managing its own system VMs.
Web Scale Applications using NeflixOSS Cloud PlatformSudhir Tonse
Web Scale Applications using NeflixOSS Cloud Platform. Infographics on IaaS, PaaS, SaaS. Commandments of developing a cloud based distributed application.
In this session Arash will show you how to use Open Cloud service delivery models such as Open IaaS and Open PaaS to deploy OpenCms as a service for your organization or your customers. You will learn how Open Source cloud operating systems and platforms such as OpenStack and Cloud Foundry can help jumping and scaling between OpenCms content clouds. Arash will also compare other PaaS solutions like AppScale, CloudBees, OpenShift and Jelastic and show if and how OpenCms can work with them. He will introduce you to the Cloud Federation concept, which helps to avoid vendor lock-in with private, public and hybrid cloud environments. Last but not least, he will explain how to achieve a high level of data security in Open Clouds, so that even system administrators won’t be able to access your OpenCms data. This session is targeted at all types of OpenCms users, such as business users, service providers and developers.
WSO2 Carbon and WSO2 Stratos Summer Release Roundup WSO2
- The webinar covered the upcoming releases of WSO2 Carbon and Stratos in summer 2012, including new products, features, and capabilities.
- Carbon 4.0 includes improvements to deployment synchronization, performance, and multi-tenancy, as well as new products like API Manager and Storage Server.
- Stratos 2.0 features a new cartridge model for multiple languages/frameworks, support for additional IaaS providers, and an enhanced management console.
- Both releases focus on improved scalability, manageability, and a modular approach to building and deploying middleware components.
AWS reinvent 2019 recap - Riyadh - Containers and Serverless - Paul MaddoxAWS Riyadh User Group
This document provides an overview and agenda for an AWS storage, compute, containers, serverless, and management tools presentation. It includes summaries of several upcoming AWS services and features related to EBS, S3, EC2, EKS, Fargate, Lambda, and AWS Cost Optimizer. The speaker is introduced as Paul Maddox, Principal Architect at AWS, with a background in development, SRE, and systems architecture.
KT ucloud storage provides an OpenStack Swift-based object storage service. Some key points:
- OpenStack Swift allows for redundant, scalable storage across standardized servers capable of storing petabytes of data. It ensures data replication and integrity.
- KT customized Swift for their use, adding middleware for SSL performance, integrating their single sign-on system, and enabling real-time usage reporting.
- To commercialize Swift, KT deployed it based on commodity servers and disks across 5 zones with 3 replications for redundancy, using automated deployment with Chef.
- While Swift is stable, deploying it on hardware can introduce problems. KT also found it difficult to increase revenue as object storage is a new concept for
KT ucloud storage provides an OpenStack Swift-based object storage service. Some key points:
- OpenStack Swift allows for redundant, scalable storage across standardized servers and can store petabytes of data. It provides reliability through data replication and a distributed architecture with no single point of failure.
- KT customized Swift for their commercial deployment, adding features like SSL performance tuning, APIs for management, and real-time usage reporting. They deployed it based on a reference architecture with multiple zones and replications.
- While OpenStack Swift is stable, deploying it on hardware can introduce problems. KT had to address issues in gathering customer usage data and analyzing patterns to focus improvements. They also found it challenging to increase revenue as
What is OpenStack? This presentation is an overview about the most fascinating projects out there today.
In this presentation, I cover the following topics:
- Quick introduction to OpenStack project
- Explain the OpenStack architecture and how its built
- Get you familiar with the different terminology and concepts
- Get you familiar with OpenStack services (components)
- Go over installation methods and tools
- Review risks
Session presented at the 2nd IndicThreads.com Conference on Cloud Computing held in Pune, India on 3-4 June 2011.
http://CloudComputing.IndicThreads.com
Abstract:“With increasing demand, ever-growing datasets, unpredictable traffic patterns and need for faster response times, “scalable architecture” has become a necessity. Here, we will see how the traditional concepts and best practices for scalability have to be adopted for the cloud. Further, we will go through the unique advantages that Amazon AWS cloud offers for architecting scalable applications. As an architect, you need to identify the components and bottlenecks in your architecture and modify your application to leverage the underlying scalability.
We will cover the following topics:
Scalability principles for the cloud
Leveraging AWS services for application components
Shared nothing architecture
Asynchronous work queues for loosely coupled applications
Database scalability
Tools, connectors and enablers to help build, deploy and monitor your cloud environment
Scalability using Platform-as-a-Service offerings on top of AWS
An example of a horizontally scalable architecture for an enterprise application on Amazon AWS
This talk will act as a primer for a cloud architect to achieve an auto-scalable, highly available, fully-monitored edge-cached application.”
Speaker:
Kalpak Shah is the Founder & CEO of Clogeny Technologies Pvt. Ltd. and guides the overall strategic direction of the company. Clogeny is focused on niche software and product development in cloud computing and scalable applications domains. He is passionate about the ground-breaking economics and technology afforded by the cloud computing platforms. He has been leading and architecting cutting-edge product development across the cloud stack including IaaS, PaaS and SaaS vendors.
He has previously worked at organizations like Sun Microsystems and Symantec in the storage domain primarily distributed and disk filesystems. Kalpak has a Bachelors’ of Engineering degree in computer engineering from PICT, University of Pune.
Visão Técnica - RHOS (Red Hat Enterprise Linux OpenStack)Raul Leite
OpenStack is an open source cloud computing platform that provides infrastructure as a service. It allows users to provision compute, storage, and networking resources on demand in a self-service manner similar to public cloud offerings. OpenStack is modular and scalable, with components that can be customized or replaced as needed. Key components include compute (Nova), storage (Cinder, Swift, Glance), networking (Neutron), identity (Keystone), dashboard (Horizon), telemetry (Ceilometer), and orchestration (Heat). Red Hat provides commercial support for OpenStack through its Red Hat Enterprise Linux OpenStack Platform.
Cisco: Cassandra adoption on Cisco UCS & OpenStackDataStax Academy
n this talk we will address how we developed our Cassandra environments utilizing Cisco UCS Open Stack Platform with the DataStax Enterprise Edition software. In addition we are utilizing OpenSource CEPH storage in our Infrastructure to optimize the Performance and reduce the costs.
This document discusses OpenStack activities and adoption in China. It outlines several OpenStack conferences and meetups held in major Chinese cities that helped spread awareness of OpenStack. It also profiles some of the key Chinese OpenStack players and community groups, including COSUG (China OpenStack User Group) and COSCL (China Open Source Cloud League). Additionally, it introduces StackLab.org, a community-run public OpenStack cloud launched to help increase adoption and benefit users, developers and companies.
DevOps in a Public OpenStack Cloud - Hui ChengHui Cheng
The document summarizes Hui Cheng's presentation on DevOps in OpenStack Public Cloud at the OpenStack Summit in Fall 2012. The presentation discusses challenges in building an OpenStack public cloud at Sina, including network topology, security, storage solutions, identity integration, billing, and dashboard improvements. It also covers operating an OpenStack production platform, including the platform stack, automated deployment, continuous integration, and project management. Finally, it introduces StackLab, a community OpenStack public cloud launched by Sina to benefit users, contributors, and sponsors.
The document summarizes information about the OpenStack Foundation, including its mission to protect, empower and promote OpenStack software and community. It describes the transition from OpenStack LLC to the independent OpenStack Foundation, with elected Project Technical Leads making project decisions overseen by the Policy Board. It outlines the Foundation's services and three types of membership - Individual, Platinum and Gold. The Board of Directors has responsibilities including overseeing operations, setting goals and advocating for the Foundation. OpenStack is the second largest open source foundation by funding.
The document discusses the China OpenStack User Group (COSUG), which was started in 2011 to promote OpenStack adoption in China. COSUG now has over 960 mailing list subscribers and nearly 2,000 registered users on its website. It coordinates OpenStack meetups and training programs across multiple Chinese cities. COSUG also works with the China OpenSource Cloud League (COSCL), a collaboration between Chinese companies and universities contributing to OpenStack projects. The document promotes StackLab, a free OpenStack sandbox environment, and outlines COSUG's plans to expand training programs, community quality, and cooperation with other OpenStack communities.
Lessons from Building OpenStack Public CloudHui Cheng
The document summarizes the schedule and agenda for the OpenStack China Tour event in Shenzhen. It includes presentations on building public clouds with OpenStack, OpenStack in Hong Kong, Cinder block storage, Juju tool for OpenStack, choosing infrastructure for private clouds, and Swift architecture. The organizer is the China OpenStack User Group (COSUG) and CSDN.
The document discusses OpenStack Quantum and OpenFlow/SDN. It provides an overview of Quantum, which allows network connectivity as a service in OpenStack. It describes how Quantum works by creating networks and ports and plugging interface devices. It also lists several Quantum plugins that can be used, such as plugins for Cisco, Linux bridge, NVP, and Open vSwitch. Finally, it introduces OpenFlow/SDN and provides basics on the OpenFlow protocol and how OpenFlow switching works.
The document discusses OpenStack Swift, an open source cloud storage system. It provides an overview of Swift and how it enables applications for the web and mobile through standards-based APIs and scalability. SwiftStack's CEO Joe Arnold also discusses field observations of how Swift supports infrastructure as a service through capabilities like large storage capacities, high concurrency, and multi-tenancy.
Cloud Based VDI with OpenStack, by Shifen YangHui Cheng
This document summarizes Cloud Based VDI with OpenStack. It discusses modifications made to Nova, Keystone, Glance, Quantum, and Swift to support a virtual desktop infrastructure using OpenStack. Key points include extending APIs, adding Active Directory support, implementing differential snapshots in Nova, enforcing storage quotas in Swift, and designing for disaster recovery. The overall goal is to launch and manage virtual machines, enable user login and authentication, check access policies, update storage rings, and implement content delivery, all through an OpenStack-based cloud computing platform.
OpenStorage with OpenStack allows storage software to be open source without vendor lock-in. It uses open data formats and some or all of the components no longer need to come from the same vendor. NexentaStor is a leading OpenStorage solution that can be used with OpenStack Nova to provision volumes from snapshots for virtual machines efficiently using cloning. NexentaStor provides features like deduplication, compression, thin provisioning and triple parity RAID that are beneficial for cloud storage and virtual environments.
1. The document discusses how there is money to be made in OpenStack through various business models like hardware, software, support and services given OpenStack's growing popularity and leadership in the open source IaaS market.
2. It estimates the total addressable OpenStack market to be around $15 billion based on the overall cloud infrastructure software and hardware market size.
3. According to the document, OpenStack is easier to invest in than proprietary solutions because it was designed for and is developed by an open source community rather than a single vendor.
4. Different sections outline potential revenue streams for players in the OpenStack ecosystem from various roles like system integration, support, software and hardware.
Network Service in OpenStack Cloud, by Yaohui JinHui Cheng
The document discusses network service in OpenStack cloud. It introduces SDN and network virtualization technologies like OpenFlow, VXLAN, and NVGRE. It describes how OpenStack's Quantum project provides network as a service and supports various plug-ins for virtual and physical network provisioning, including options from Open vSwitch, NVP, Cisco, and OpenFlow controllers. It also outlines the speaker's work on a virtual network solution called vCube that enables coexistence of VLAN and GRE networks in OpenStack and provides QoS and visibility capabilities.
COSUG was started in August 2011 and holds meetups in Shanghai, Beijing, and other Chinese cities to discuss OpenStack. It currently has over 858 subscribers to its Google Group, over 1,500 registered users on its website, and over 1,000 users in its QQ chat group. COSUG plans to expand its OpenStack training program and community involvement. It focuses on users for deployment and operations, while COSCL jointly develops OpenStack projects with support from companies like Intel and Sina.
OpenStack Chances and Chanlleges_OSAC_by_ChengHuiHui Cheng
Billing and metering are important parts of any IaaS cloud. Kanyun and Dough are community projects started by Sina to contribute these capabilities to OpenStack. They help with monitoring resource usage, collecting metrics, and handling billing and payments. This helps make OpenStack more production-ready for real-world cloud deployments.
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Tatiana Kojar
Skybuffer AI, built on the robust SAP Business Technology Platform (SAP BTP), is the latest and most advanced version of our AI development, reaffirming our commitment to delivering top-tier AI solutions. Skybuffer AI harnesses all the innovative capabilities of the SAP BTP in the AI domain, from Conversational AI to cutting-edge Generative AI and Retrieval-Augmented Generation (RAG). It also helps SAP customers safeguard their investments into SAP Conversational AI and ensure a seamless, one-click transition to SAP Business AI.
With Skybuffer AI, various AI models can be integrated into a single communication channel such as Microsoft Teams. This integration empowers business users with insights drawn from SAP backend systems, enterprise documents, and the expansive knowledge of Generative AI. And the best part of it is that it is all managed through our intuitive no-code Action Server interface, requiring no extensive coding knowledge and making the advanced AI accessible to more users.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdfflufftailshop
When it comes to unit testing in the .NET ecosystem, developers have a wide range of options available. Among the most popular choices are NUnit, XUnit, and MSTest. These unit testing frameworks provide essential tools and features to help ensure the quality and reliability of code. However, understanding the differences between these frameworks is crucial for selecting the most suitable one for your projects.
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
This presentation provides valuable insights into effective cost-saving techniques on AWS. Learn how to optimize your AWS resources by rightsizing, increasing elasticity, picking the right storage class, and choosing the best pricing model. Additionally, discover essential governance mechanisms to ensure continuous cost efficiency. Whether you are new to AWS or an experienced user, this presentation provides clear and practical tips to help you reduce your cloud costs and get the most out of your budget.
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
11. Open Source
Apache 2.0 license, NO ‘enterprise’ version
Open Design
Open Design Summit
Open Development
Anyone can involve development process Open development management via
Launchpad & Github
Open Community
OpenStack Foundation in 2012
12. OpenStack Mission
"To produce the ubiquitous Open Source
cloud computing platform that will meet the
needs of public and private cloud providers
regardless of size, by being simple to
implement and massively scalable."
16. Where to Get Started?
Ubuntu 12.04 server 集成OpenStack
Trystack.org申请测试账号
devstack.sh一键安装
17. OpenStack Development
Authorization Hosting code &
(group membership) Bug tracking formal docs
Feature planning Mailing lists
(Blueprints)
User support
(Answers)
Wiki
Informal docs
Continuous
integration
18. Nova Key Features
ReST-base API
Asynchronous communication
Horizontally scalable
Shared nothing architecture*
Distribute everything
Test everything
100% Python Based
* http://en.wikipedia.org/wiki/Shared_nothing_architecture
* http://wiki.openstack.org/BasicDesignTenets
19. OpenStack Compute: Nova
nova-api
Compute API Server
OpenStack API, EC2 compatibility API
nova-compute
Compute worker
Manage compute host and VMs
Libvirt(QEMU,KVM,LXR), XenServer and XCP, ESX(i)*
nova-network
Network controller
Manage network resources: IPAM, VLAN, NAT
*http://wiki.openstack.org/HypervisorSupportMatrix
20. OpenStack Compute: Nova(cont.)
nova-scheduler
Determines the placement of new resources
nova-volume
Block storage, remote attach a LVM volume using iSCISI protocol
Like Amazon EBS, but far way from mature
RabbitMQ
Message Queue
Cast and RPC Call for services
23. Nova Network
L2
FLAT, FLATDHCP, VLAN
L3
IPAM(IP Address Management)
Fixed IP, Floating IP
Gateway, NAT, VPN
24. Quantum
Quantum Basics
Nova: virtual server
Quantum: virtual network
Quantum :
Expose a API for creating virtual networks and attaching
instances(e.g.,novaservers) to those networks
Manage switches(virtual or physical) in the data center to
implement connectivity described via API
Provide a“plugin” architecture to leverage support using different
back-end technologies
25.
26. Quantum: available plugins
Open vSwitch
- Builds isolated networks with OVS and L2-in-L3 tunnel
Cisco UCS
- Isolation based on VLAN and net-profiles applied to Cisco
UCS converged network adapters
Linux Bridge
- Build isolated networks with VLAN interfaces and linux
bridges
- Works with every Linux Distro
NTT-Data Ryu
- Acts as a proxy for the NTT Ryu platform
Nicira NVP
- Acts as a proxy for the Nicira NVP platform
27. Swift: Storage Types
Types Protocol Application
Block Storage SATA, SCISI, iSCISI SAN, NAS, EBS
File Storage Ext3/4, XFS, NTFS PC, Servers, NFS
Object Storage HTTP, REST Amazon S3,
Google Cloud Storage,
Rackspace Cloud Files
Specific Storage Specific protocol MySQL, MongoDB,
based on tcp HDFS
We want a Object Storage like Amazon S3.
28. Features Swift Amazon S3
object/bucket CRUD √ √
account/bucket/object ACL √ √
object metadate √ √
large object √ √
rate limit √ √
Swift expiring object √ √
vs static web √ √
REST API √ √
Amazon S3 Account support √ X
Account metadata √ X
Bucket metadata √ X
Bucket sync across cluster √ X
Object versioning X √
Log to bucket X √
Notification X √
Reduced Redundancy Storage X √
SOAP API X √
Server Side Encryption X √
BitTorrent protocol X √
29. Swift Evaluation
Extremely Durable and Highly Available
Superior Scalability
Linear Growth of Performance
Symmetric Architecture
No Single-failure
Simple & Reliable
30. Swift Components
The Ring: Mapping of names to entities (accounts,containers,
objects) on disk.
Stores data based on zones, devices, partitions, and replicas
Weights can be used to balance the distribution of partitions
Used by the Proxy Server for many background processes
Proxy Server: Request routing, exposes the public API
Replication: Keep the system consistent, handle failures
Updaters: Process failed or queued updates
Auditors: Verify integrity of objects, containers, and account
31. Swift Architecture
• 1 Zone = 1 Physical Server with 12x2T disk
GET abc.png • Write/Read applies quorum protocol
PUT abc.png
Load Balancer
Zone1 Zone2 Zone3 Zone4 Zone5
Proxy Server Proxy Server Proxy Server Proxy Server Proxy Server
Object Server Object Server Object Server Object Server Object Server
Container Container Container Container Container
Server Server Server Server Server
Account Server Account Server Account Server Account Server Account Server
31
32. Swift Installation
Swift packages
Proxy Server
Account Server
Container Server
Object Server Physical Deployment
Storage Nodes
OS installation
sda sdb sdc sdd sdk
raid 1 ……
disk1 disk2 disk3 disk4 disk5 disk12
35. Infrastructure & Platform
Physical Servers
Traditional Operation
Virtualization Platform(IaaS)
●VM Management System(VMMS) → Sina Web
Service(SWS)
●VMMS is private solution developed in-house
●SWS is based on OpenStack
新浪云计算
Application Platform(PaaS)
●Virtual Host → Sina App Engine(SAE)
●SAE provides both Public and Private Service.
●Proved to be Efficient and Robust
36. Nova Network
Networking is the biggest challenges for IaaS
Network Topology:
• VLAN
• FlatDHCP
• FlatDHCP & Multihost
37. Network Topology (VLAN)
Capability:
• Accessibility of VMs within one tenant
• Isolation of VMs from different tenants
• VM is able to access public network
• VM can be accessible from public network
• Isolation between virtual network and
internal network
Drawback:
• Pre-allocate network for future projects
• Hard-limit of vlan 4096
• Traffic bottleneck in the gateway/NAT
38. Network Topology(Flat)
Capability:
• Accessibility of all VMs in the fixed IP
range
• VM is able to access public network
• VM can be accessible from public network
• Full isolation between virtual network and
internal network
Bonus:
• Do not need pre-allocate for new projects
• Eliminating bottleneck between tenants
Drawback:
• Tenant isolation has gone
• Traffic bottleneck still exists in NAT
39. Network Topology(Flat & Multihost)
Capability:
• Accessibility of all VMs in the fixed IP range
• VM is able to access public network
• VM can be accessible from public network
Bonus:
• Totally distributed architecture avoid single-
point failure.
• Multiple gateway eliminates NAT bottleneck
• High speed between OS regions
Drawback:
• Tenant isolation lessens
• Need security facility(SWS-filter) to protect
intranet
If security problems were solved, this would be our best choice!
40. Security in OpenStack
Security Group --- L3 Filter Static filters --- L2 Filter
Role-based firewall MAC, IP, and ARP spoofing protection
One security group is a Role Not configurable
Ingress filtering Defined in /etc/libvirt/nwfilter/*.xml
Target is the instance Implemented by ebtables
Source can be CIDR or another group ebtables -t nat --list
Implemented by iptables
See details: iptables -t filter -n -L
Whitelist mechanism(ACCEPT rules)
41. Security Enhancement
SWS Filter
Prevent Intranet Penetration
• Intranet is the internal network outside of
OpenStack
Egress filtering
• Target is internal network
• Source is instances in OpenStack
Implementation
• Whitelist mechanism(ACCEPT rules)
• On the top of nova-filter-top Forward
Chain
Rational
• SWS filter is managed by cloud manager
• Only explicit authorized packets can reach Internal network C
• Packet should be controlled within Compute Node
43. Load Balancer
Goals
Load Balance
• Dispatch request DNS Acceleration Design
• Support multiple routing algorithm
• Health check
Smart DNS
Acceleration
• Reality: narrow bandwidth between ISPs
• Building fiber channels from ISPs to pivot Public Network
• Given the same endpoint within user’s ISP
Telecom Unicom Mobile Others ISP
IPv4 Shortage
• Reality: dozens of public IPs support
hundreds of VMs High speed fiber channel
• IPv4 has been exhausted
• IPv6 is not realistic yet in China
Pivot
49. Sina Contributions
Sina creating open source project “Dough” to contribute
metering & billing capability
Present in OpenStack Design Summit & Conference
49