4. Cloud
Built for traditional enterprise
apps & client-server compute
• Enterprise arch for 100s of hosts
• Scale-up (pool-based resourcing)
• IT management-centric
• 1 administrator for Dozens of servers
• Apps assume reliability
• Proprietary vendor stack
Designed around big data,
massive scale & next-gen apps
• Cloud arch for 1000s of hosts
• Scale-out (horizontal resourcing)
• Autonomic management
• 1 administrator for 1,000’s of servers
• Apps assume failure
• Open, value-added stack
Server Virtualization++
Cloud Computing
Virtualization is not Cloud computing
5. • Tenets of Cloud
o Shared infrastructure and Multi-tenancy
o Self Service
o Elasticity
o Built for massive Scale
o Service agility
o Pay-as-you-go
o APIs and Extreme Automation
• IAAS/PAAS/SAAS
• Public/Private/Hybrid clouds
Cloud Computing (contd..)
6. • Turnkey orchestration platform for delivering IAAS clouds
o Secure, multi-tenant
o Self-service
o Service agility and elasticity
o Built for large scale
o Pay-as-you-go
• Deploys on premise (private) or as a hosted (public) cloud
• Can be used for hybrid clouds
• built in java, provides native REST API’s and EC2 API
• Has python, Ruby clients and CLI as well
What is Apache CloudStack
7. A
bit
of
History
• Original
company
Cloud.com
(2008)
• Open
source
(GPLv3)
as
CloudStack
(2010)
• Acquired
by
Citrix
(July
2011)
• Relicensed
under
ASL
v2
April
3,
2012
• Accepted
as
Apache
IncubaKng
Project
April
16,
2012
• Many
non-‐Citrix
contributors,
commiPers,
PPMC
members
• Top
Level
Poject
(April
2013)
8. Who is contributing
• Sungard: Unit test cases
• Carnigo: Object store plug-in
• Ceph/Rbd support by Wido
• CLVM/KVM by Marcus
• Nicira NVP: Schuberg Philis
• Basho: Object Store
• Brocade ADX ADC support
• Midokura midonet SDN controller integration
9. How to contribute
• Its not just about code! As community member
you can engage in
o Discussions: Design, Use Case, deployment
issues
o Bug reporting, feature requests
o Code reviews
o Build, tools, infrastructure
o Helping out on the IRC
o Documentation
o Submit bug fixes, features
10. How to contribute (contd..)
• Git repo, bug tracker, wiki are on ASF infra
• Project website
o http://cloudstack.apache.org/
o http://www.cloudstack.org
• Mailing lists (cloudstack.org/discuss/mailing-
lists.html)
o dev-subscribe@cloudstack.apache.org
o users-subscribe@cloudstack.apache.org
• Cloudstack -101
11. CloudStack managed cloud
Compute StorageNetwork
Admin
Users
Org B
End User
Cloud
Admin
On-demand infrastructure as a service
CloudStack
Management Server
REST API
UI Cli EC2
Admin
Users
Org A
Consume
resources
Provision
resources
manage
resources
12. • Hosts
• Servers onto which services will be provisioned
• Primary Storage
• VM storage
• Cluster
• A grouping of hosts and their associated storage
• Pod
• Collection of clusters
• Network
• Logical network associated with service offerings
• Secondary Storage
• Template, snapshot and ISO storage
• Zone
• Collection of pods, network offerings and secondary
storage
• Management Server Farm
• Responsible for all management and provisioning
tasks
Core CloudStack Components
Zone
CloudStack Pod
Cluster
Host
Host
Network
Primary
Storage
VM
VM
CloudStack Pod
Cluster
Secondary
Storage
13. Pod 1
….
Cluster N
Access Layer
Host 2
Cluster 1
CloudStack Deployment Architecture
Host 1
Ø Hypervisor is the basic unit
of scale.
Ø Cluster consists of one ore
more hosts of same
hypervisor
Ø All hosts in cluster have
access to shared (primary)
storage
Ø Pod is one or more clusters,
usually with L2 switches.
Ø Availability Zone has one or
more pods, has access to
secondary storage.
Ø One or more zones
represent cloudPrimary
Storage
Zone 1
….
L3 core
Secondary
Storage
Pod N
CloudStack
Management
Server
Internet
14. Zone1
Data Center 1
Data Center 2
Zone
3
Zone
2
Data Center 2
Zone
3
Zone
2
Data Center 2
Zone
3
Zone
2
Data Center 2
Zone
3
Zone
2
Data Center 2
Zone
3
Zone
2
Data Center 3
Zone
4
Management
Server
Ø Single Management Server can
manage multiple zones
Ø Zones can be geographically
distributed but low latency links
are expected for better
performance
Ø Single MS node can manage up
to 5K hosts.
Ø Multiple MS nodes can be
deployed as cluster for scale or
redundancy
CloudStack Managing Multiple Zones
19. Volume & Snapshot Management
Volume
VM 1
Add / Delete
Volumes
Schedule
Snapshots
Hourly
Daily
Weekly
Monthly
Now
Create Templates
from Volumes
Volume Template
View Snapshot
History
….
20. A
Very
Flexible
IaaS
Pla5orm
Compute
XenServer VMware KVMOracle VM Bare metal
Hypervisor
Storage
Local Disk iSCSI NFS
Fiber
Channel
Swift
Block & Object
Network
Network Type Isolation
Load
balancer
Firewall VPN
Network & Network Services
Primary
Storage
Secondary
Storage
Ceph Riak
21. Pod 1
Host 2
Cluster 1
Host 1
Primary
Storage
L3 switch
Secondary
Storage
L2 switch
CloudStack Storage
• Configured at Cluster-level. Close to hosts for
better performance
• Stores all disk volumes for VMs in a cluster
• Cluster can have one or more primary storages
• Local disk, iSCSI, FC or NFS
Primary Storage
• Configured at Zone-level
• Stores all Templates, ISOs and Snapshots
• Zone can have one or more secondary storages
• NFS, OpenStack Swift
Secondary Storage
• Storage available on hypervisor hist
Local Storage
Local
storage
Availability zone
22. • Primary Storage
• Cluster level storage for VMs
• Connected directly to hosts
• NFS, iSCSI, FC and Local
• Secondary Storage
• Zone level storage for template, ISOs and
snapshots
• NFS or OpenStack Swift via CloudStack
System VM
• Templates and ISOs
• Imported into CloudStack
• Can be private or public
Role of Storage and Templates
Zone
Secondary Storage
Pod
Cluster
Host
Host
Primary Storage
Template
23. 1. User Requests Instance
2. Provision Optional Network
Services
3. Copy instance template from
secondary storage to primary
storage on appropriate cluster
4. Create any requested data
volumes on primary storage for the
cluster
5. Create instance
6. Start instance
Provisioning Process
Zone
Secondary Storage
Pod
Cluster
Host
Host
Primary Storage
VM
Template
24. Domain is a unit of isolation
that represents a customer
org, business unit or a
reseller
Domain can have arbitrary
levels of sub-domains
A Domain can have one or
more accounts
An Account represents one
or more users and is the
basic unit of isolation
Admin can limit resources at
the Account or Domain
levels
Admin
Org A
Admin
Reseller A
Domain
Domain
Admin
Org C
Sub-Domain
User 1
User 2
Group B
Account
Group A
Account
VMs, IPs, Snapshots…
VMs, IPs, Snapshots…
Resources
Resources
Multi-tenancy & Account Management
25. User Dashboard: Consumed Resources
• Running, Stopped &
Total VMs
• Public IPs
• Private networks
• Latest Events
27. Edge services with System VMs
• System VMs optimize and scale the datapath on behalf of CloudStack
o Stateless, can be destroyed and recreated from database state
o Highly Available
o Communicates with Management Server over management network
o Usually have 3 interfaces: control, guest and public
• Console Proxy VM
o Provides AJAX-style HTTP-only console viewer
o Grabs VNC output from hypervisor
o Scales out (more spawned) as load increases
o Java-based server Communicates with MS over message bus
• Secondary Storage VM
o Provides image (template) management services
o Download from HTTP file share or Swift
o Copy between zones
o Scale out to handle multiple NFS mounts
o Java-based server communicates with MS over message bus
28. • Virtual Router VM
o Provides multiple network services
o IPAM (DHCP), DNS, NAT, Source NAT, Firewall, PF, VPN
o User-data, Meta-data, SSH keys and password change server
o Redundancy via VRRP
o MS configures VR over SSH
§ Proxied via the hypervisor on XS and KVM
Edge services with System VMs (contd.)
29. Network & Network Services
• Create Networks and attach VMs
• Acquire public IP address for
NAT & load balancing
• Control traffic to VM using
ingress and egress firewall rules
• Set up rules to load balance
traffic between VMs
30. Networking feature overview
• Orchestration of L2 – L7 network services
o IPAM, DNS, Gateway, Firewall, NAT, LB, VPN, etc
• Mix-and-match services and providers
• Out-of-the-box integration with automated deployment of virtual
routers
o Highly available network services using CloudStack HA and VRRP
• Orchestrate external providers such as hardware firewalls and
load balancers
o Devices can provide multiple services
o Admin API to configure external devices
o Plugin-based extensions for network behavior and admin API extensions
• Multiple multi-tenancy [network isolation] options
• Integrated traffic accounting
• Access control
• Software Defined Networking (Nicira NVP)
31. L2 Features
• Choice of network isolation
o Physical, VLAN, L3 (anti-spoof), Overlay[GRE]
o Physical isolation through network labels [limited to # of
nics or bonds]
• Multi-nic
o Deploy instance in multiple networks
o Control default route
• Access control
o Shared networks, project networks
• QoS [max rate]
• Traffic monitoring
• Hot-plug / detach of nics
32. L3 Features
• IPAM [DHCP], Public IP address management
o VR acts as DHCP server
o Can request multiple public IPs per tenant
• Gateway (default gateway)
o Redundant VR (using VRRP)
o Inter-subnet routing
o Static routing control
• Remote Access VPN
o L2TP over IPSec using PSK
o Virtual Router only
• Firewall based on source cidr
• Static NAT [1:1]
o Including “Elastic IP” in Basic Zone
• Source NAT
o Per-network, or interface NAT
• Public Traffic usage
o Monitoring on the Virtual Router / External network device
o Integration with sFlow collectors
• Site-to-Site VPN
o IPSec VPN based on VR
• L3 ACLs
33. L4 Features
• Security groups for L3-isolation
o “Basic Zone” in docs
o Default AWS-style networking
o Scales much better than VLANs
• Stateful firewall for TCP, UDP and ICMP
• Port forwarding [“Advanced Zone”]
o Conserve public Ips
34. L7 features
• Loadbalancer
o VR has HAProxy built in
o External Loadbalancer support
§ Netscaler (MPX/SDX/VPX)
§ F5 BigIP
§ Can dedicate an LB appliance to an account or share it
among tenants
o Loadbalancer supported with L3-isolation as well
o Stickiness support
o SSL support [future]
o Health Checks [future]
• User-data & meta-data
o Fetched from virtual router
• Password change server
35. CloudStack Terminology
• Guest network
o The tenant network to which instances are attached
• Storage network
o The physical network which connects the hypervisor to primary storage
• Management network
o Control Plane traffic between CloudStack management server and hypervisor clusters
• Public network
o “Outside” the cloud [usually Internet]
o Shared public VLANs trunked down to all hypervisors
• All traffic can be multiplexed on to the same underlying physical network
using VLANs
o Usually Management network is untagged
o Storage network usually on separate nic (or bond)
• Admin informs CloudStack how to map these network types to the
underlying physical network
o Configure traffic labels on the hypervisor
o Configure traffic labels on Admin UI
36. CloudStack Network Service Providers
• A Network Service Provider is hardware or virtual
appliance that makes a network service possible
in CloudStack ; for example, a Citrix NetScaler
appliance can be installed in the cloud to provide
Load-Balancing services.
• Administrators can have multiple instances of the
same service provider in a network; for example,
more than one Citrix NetScaler or Juniper SRX
device can be added to CloudStack
• CloudStack supports the following Network
Providers:
o CloudStack Virtual Router (default)
o Citrix NetScaler SDX, VPX and MPX models
o Juniper SRX
o F5 BigIP
37. Network Service Providers Matrix
Feature Virtual
Router
Citrix
NetScaler
Juniper
SRX
F5 BigIP
Remote Access VPN YES N/A N/A N/A
Firewall YES N/A YES N/A
Source NAT YES N/A YES N/A
Static NAT YES YES YES N/A
Load Balancing YES YES N/A YES
Port Forwarding YES N/A YES N/A
Elastic IP N/A YES N/A N/A
Elastic LB N/A YES N/A N/A
DHCP/DNS/User Data YES N/A N/A N/A
• Network offerings is basically a definition of what Network Services are
available when this offering is used. The available Network Services
are: VPN, DHCP, DNS, Firewall, Load Balancer, User Data, Source
NAT, Static NAT, Port Forwarding and Security Groups*
38. • Cloud provider defines the
feature set for guest networks
• Toggle features or service
levels
o Security groups on/off
o Load balancer on/off
o Load balancer software/hardware
o VPN, firewall, port forwarding
• User chooses network offering
when creating network
• Enables upgrade between
network offerings
• Default offerings built-in
o For classic CloudStack
networking
Network Offerings
39. Add Guest Networks
• Choice to choose L3
subnet, default gateway
• Choice of network
offerings
40. Editing Guest Networks
When editing a guest network users can
change the network offering. They can
either upgrade to a “premium” network
offering (for example offering that uses
hardware Load-balancer) or downgrade to a
“cheaper” network.
41. • Restarting the network will simply
resend all the LB, Firewall and
Port-Forwarding rules to the
network provider
• Restarting the Network with
“Clean up”:
• restarKng
network
elements
-‐
virtual
routers,
DHCP
servers
• If
virtual
router
is
used,
it
will
be
destroyed
and
recreated
• Reapplying
all
public
IPs
to
the
network
provider
• Reapplying
load-‐Balancing/Port-‐Forwarding/Firewall
rules
Restarting/Cleaning Up a Guest Network
42. • An Isolated Guest Network can only be deleted if no VMs
are using these network (e.g. Completely destroyed and
expunged)
• Deleting a Network will Destroy the Virtual Router (if used)
and will release the Public IPs back to the IP Pool
Deleting a Guest Network
43. Basic vs Advanced Networking
• Segmentation based on feature set and ease-of-
deployment
• Both are feature-rich
• Basic implements true AWS-style L3-isolation
o Tenants do not get contiguous IP addresses or subnets
o Network segmentation based on Security Groups
o Tremendous scale (tens of thousands)
• Advanced Zone offers full L3 subnets and L2
isolation
o VLANs are default implementation (4K limit)
o More features (source NAT, PF, LB, VPN)
44. Storage 1
Hypervisor
1
Hypervisor
N
Hypervisor
8
Access
Switch(es)Cloudstack
Server
VM Traffic
Control Plane Traffic
Storage Traffic
Cloudstack
Servers
Storage k
…
Pod 1
CLUSTER 1
…
CLUSTER 4
Core (L3) Network
…
Pod 2 Pod N
Physical Network in Zone
Storage 2
Hypervisor
N+1
Public Traffic
46. Guest Networks with L3 isolation
Guest
1
VM
1
Guest
2
VM
1
Guest
1
VM
2
Guest
2
VM
2
Public
Internet
10.1.0.1
Public
IP
address
65.37.141.11
65.37.141.24
65.37.141.36
65.37.141.80
Guest
address
10.1.0.2
Guest
address
10.1.0.3
Guest
address
10.1.0.4
Guest
address
10.1.16.12
Load
Balancer
Guest
2
VM
3
Guest
1
VM
3
Guest
1
VM
4
Guest
address
10.1.16.21
Guest
address
10.1.16.47
Guest
address
10.1.16.85
L3
Core
Switch
Pod
1
L2
Switch
Pod
3
L2
Switch
10.1.16.
1
…
…10.1.8.1Pod
2
L2
Switch
47. Hypervisor
1
Hypervisor
N
Hypervisor
8
Access
Switch(es)
VM Traffic
…
Pod K
CLUSTER 1
…
CLUSTER 4
Core (L3) Network
…
Pod M Pod N
Guest Networks with L2 isolation
Hypervisor
N+1
Public Traffic
Hypervisor
R
R V
VV
V
Hypervisor
V V
V
R
Tenant VM
Tenant Virtual Router
48. …
L2 isolation: VLAN networking
… …
User
2
User
2
User
1
User
1
User
1
User
1
User
1
User
2
User
1
49. SDN at Work
Host 1
Host 2
Host 3
Host 4
GRE Tunnel
GRE Tunnel GRE Tunnel
VM
1
VM
2
VM
3
V
R
OVS
OVS OVS
CloudStack Mgmt Server
SDN
Controlle
r
VM
1
VM
2
VM
3
V
R
OVS
GRE Tunnel
50. Guest virtual layer-2 network
Guest
1
VM
1
Guest
1
VM
2
Guest
1
VM
3
Guest
1
VM
4
Public
Internet
Public
Network
Guest
Virtual
Network
10.1.1.0/24
Gateway
address
10.1.1.1
NAT
DHCP
Load
Balancing
VPN
Public
IP
address
65.37.141.11
65.37.141.36
Guest
address
10.1.1.2
Guest
address
10.1.1.3
Guest
address
10.1.1.4
Guest
address
10.1.1.5
Guest
1
Virtual
Router
Guest
2
VM
1
Guest
2
VM
2
Guest
2
VM
3
Guest
Virtual
Network
10.1.1.0/24
Gateway
address
10.1.1.1
NAT
DHCP
Load
Balancing
VPN
Guest
address
10.1.1.2
Guest
address
10.1.1.3
Guest
address
10.1.1.4
Guest
2
Virtual
Router
Public
IP
address
65.37.141.24
65.37.141.80
51. Layer-2 Guest Virtual Network
Public
Network/
Internet
Guest
Virtual
Network
10.1.1.1/8
VLAN
100
Gateway
address
10.1.1.1
DHCP,
DNS
NAT
Load
Balancing
VPN
Public
IP
65.37.141.11
10.1.1.1
Guest
VM 1
10.1.1.3
Guest
VM 2
10.1.1.4
Guest
VM 3
10.1.1.5
Guest
VM 4
CS
Virtual
Router
Public
Network/
Internet
Guest
Virtual
Network
10.1.1.1/8
VLAN
100
Private
IP
10.1.1.112
DHCP,
DNS
Public
IP
65.37.141.
112
10.1.1.1
Guest
VM 1
10.1.1.3
Guest
VM 2
10.1.1.4
Guest
VM 3
10.1.1.5
Guest
VM 4
NetScaler
Load
Blancer
Private
IP
10.1.1.111
Public
IP
65.37.141.111 Juniper
SRX
Firewall
CS Virtual Router provides Network Services External Devices provide Network Services
CS
Virtual
Router
52. Layer-3 Guest Network
Public
Network
65.11.0.0/16
65.11.1.2
Guest
VM 1
Guest
VM 2
Guest
VM 3
Guest
VM 4
Public
Network/
Internet
NetScaler
Load
Blancer
Network Services Managed Externally Network Services Managed by CS
65.11.1.3
65.11.1.4
65.11.1.5
DHCP,
DNS
CS
Virtual
Route
r
Security
Group
1
Security
Group
2
10.1.2.3
Guest
VM 1
Guest
VM 2
Guest
VM 3
Guest
VM 4
10.2.12.4
10.5.2.99
10.1.2.18
DHCP,
DNS
CS
Virtual
Router
Security
Group
1
Security
Group
2
EIP,
ELB
65.11.1.2
65.11.1.3
65.11.1.4
L3
switch
53. Multi-tier network
10.1.1.1
Web VM
1
10.1.1.3
Web VM
2
10.1.1.4
Web VM
3
10.1.1.5
Web VM
4
Virtual
Network
10.1.1.0/24
VLAN
100
Virtual
Network
10.1.2.0/24
VLAN
1001
10.1.2.31
App VM
1
Virtual
Network
10.1.3.0/24
VLAN
141
10.1.2.24
App VM
2
10.1.3.24
DB VM
1
CS
Virtual Router
Customer
Premises
IPSec or SSL site-to-site VPN
Internet
Monitoring VLAN
Virtual Router Services
• IPAM
• DNS
• LB [intra]
• S-2-S VPN
• Static Routes
• ACLs
• NAT, PF
• FW [ingress & egress]
• BGP
Loadbalancer
57. DevCloud
• Several use cases
o Try CloudStack in an isolated sandbox. Runs within
the appliance
o Develop CloudStack on own machine, build locally
and deploy new version in DevCloud (Build and test)
o Develop and Run locally, use DevCloud as Xen hosts
