3 Networking CloudStack Developer Day By Alex Huang Architect, Cloud Platforms Group, Citrix Systems Inc.

1. CloudStack Networking
Alex Huang
November 5 2012

2. Outline
• Design Goals
• CloudStack Terminology
• Architectural Model
• Networking APIs

3. Design Goals
• Enable networking partners to innovate and
differentiate.
• Give control and choice to the cloud operator.
• Simplify presentation to the end user.
• Enable developers to concentrate on
innovation.

4. Explosion of Network Features
• L2
– Physical, VLAN, L3 (anti-spoof), Overlay[GRE], SDN.
– QoS, traffic monitoring, broadcast & multicast.
• L3
– IPAM [DHCP], Public IP address management, Gateway, VPN,
Firewall, Static NAT, Source NAT, Site-to-Site VPN, L3 ACLs
• L4
– Security groups for L3-isolation, Stateful firewall for TCP, UDP
and ICMP, Port forwarding
• L7
– Loadbalancing, User-data, Password Change
• More will come
– Key is CloudStack must not control innovation.

5. Enabling Innovation
• CloudStack must not define the innovation.
– Partners define their own APIs.
– Partners and CloudStack can work together on unified APIs
through design process on Apache.
• Differentiate between orchestration and provisioning.
– CloudStack only orchestrates.
– Provisioning is always pushed to the partner.
• Clearly defined data center abstraction layer.
– Changes in this layer are broadcasted to partners.
• Utilize CloudStack’s orchestration to deploy and auto-
scale partners’ technologies.

6. CloudStack Terminology (End User)
• Network
– A single concept to encapsulate multiple network technologies to simplify
representation to the end user.
– One Network to rule them all, One Network to define them, One Network to
bring them all and in the cloud bind them.
– Each Network always carries its Network Traffic Type.
– CloudStack DOESN’T understand how to provision this conceptual network on
to the physical network.
• NetworkService
– L2-L7 network services that partners have written to operate within a
Network.
– Currently defined: Load Balancing, Port Forwarding, Firewall, Gateway, DNS,
DHCP, Static NAT, VPN, Source NAT, User Data.
• NetworkOffering
– A packaging of the NetworkServices provided to the end user on a particular
Network.
– NetworkOfferings are put together by cloud operator.

7. CloudStack Terminology (Operator)
• Network Traffic Type
– Traffic types are mapped to the underlying physical
network by the cloud operator.
– Traffic type is not the same as network (Guest traffic type
can actually be carried on multiple networks)
– Currently defined: Public, Guest, Storage (Backup really),
Management
• NetworkServiceProvider
– Plugin that understands how to provide one or more
NetworkServices by using VPX or physical resource.
• PhysicalNetwork
– Actual wiring of the data center.

8. CloudStack Terminology (Partner)
• NetworkGuru
– Plugin that understands the network isolation technology, mac addressing scheme, and IP
addressing scheme deployed and how to map Network Traffic Types to the underlying physical
network.
– CloudStack passes Network to NetworkGuru to “implement” before the network is needed by
a virtual machine.
– CloudStack asks the NetworkGuru to issue ip, mac, and isolation to a virtual machine before it
starts.
– CloudStack informs the NetworkGuru when a virtual machine stops so it can collect resources.
– When all virtual machines in a Network are stopped, CloudStack garbage-collects the Network
by asking the NetworkGuru to shutdown the network.
– CloudStack provides a default implementation for VLAN based isolation technology.
• NetworkElement
– Interface that specifies the events CloudStack signals to the NetworkServiceProviders when a
Network needs to be “implemented” and shutdown and when a virtual machine joins and
leaves a Network.

9. “Architect” Model
• The builder offers multiple blueprints for the owner to build the house.
• Owner chooses on a blueprint and then adds on with additional enhancements
such as hardwood floors, granite counter tops, etc.
• General contractor builds to the blueprint by orchestrating between different sub-
contractors to build different parts of the blueprint.
• There are two general category of contractors.
– Rough-in sub-contractors who take care of plumbing, electricity, framing, foundation.
– Finish sub-contractors who put in flooring, kitchen cabinets etc.
• Each sub-contractor is responsible for only their work but looks over the entire
blueprint to make sure their work can actually be done.
– E.g. A lighting plan may conflict or needs to change depending on the framing plan.
• General contractor is responsible for sequencing the sub-contractors to make sure
everything the sub-contractor is dependent on is ready when the sub-contractor
arrives to do his work.
• Every change requires a the blueprint to be republished so every sub-contractor
can make their appropriate changes.

10. Comparison
Building a house Building a network
• Owner • End user
• Builder • Cloud Operator
• General Contractor • CloudStack Orchestration
• Rough-in Sub-Contractors • NetworkGurus
• Finish Sub-Contractors • NetworkServiceProviders
• Blueprint • Network
• Cabinets, Flooring, Counter • NetworkServices
Tops, etc

11. Architectural Principles
• CloudStack clearly defines the difference between orchestration
and provisioning.
– Orchestration the ordering of what needs to happen in CloudStack’s
abstraction layer.
– Provisioning is the actual work performed at the resource.
• CloudStack clearly defines the difference between network
definition and network services.
– Network definition is handled by NetworkGuru.
– Network services is handled by NetworkServiceProvider.
• CloudStack broadcasts changes in the network every time
NetworkServices and virtual machines changes in the Network.
• CloudStack allows the Cloud Operator to setup the appropriate
mappings between virtual concepts such as Network and Network
Traffic Type to the underlying physical network.

12. Sequence Flow for VM Creation Kernel
End User Security User VM VirtualMac Network Storage Network Job
Rest API Checkers Mgr hine Mgr Mgr Mgr Guru Scheduling
Deploy VM
ACL Checks
Allocate Entity in CS
Allocate VM
Allocate NIC
Allocate IP
Allocate Volume
Schedules Deploy Job
Returns with job id, VM id
Query Job Result
Returns with job status

13. Sequence Flow for VM Creation
Deploymen Server
User VM VirtualMac Network Storage Network Network Template t
Job Threads Services API Resources
Mgr hine Mgr Mgr Mgr Guru Element Mgr Planner
Start VM
Start User VM
Start VM
Get a Deployment Plan (Host and StoragePool)
Prepare Nics
Reserve resources for Nic
Notify that Nic is about to be started in network
Agent Calls
Prepare Volumes
Prepare template on Primary Storage
Agent Calls
Agent Start VM Call
Stores job result

14. CloudStack User APIs [sample]
• Networks (L2)
– createNetwork [requires network offering id],
– deleteNetwork (A), listNetworks,
– restartNetwork (A): restarts all devices (if allowed)
supporting the network and re-applies
configuration
– updateNetwork: update network offering and
restart network

15. Restarting and Cleaning Up a Guest Network
• Restarting the network will
simply resend all the LB,
Firewall and Port-Forwarding
rules to the network provider
• Restarting the Network with
“Clean up”:
• restarting network elements - virtual
routers, DHCP servers
• If virtual router is used, it will be destroyed
and recreated
• Reapplying all public IPs to the network
provider
• Reapplying load-Balancing/Port-
Forwarding/Firewall rules

16. Deleting a Guest Network
• An Isolated Guest Network can only be deleted if no VMs are
using these network (e.g. Completely destroyed and expunged)
• Deleting a Network will Destroy the Virtual Router (if used) and
will release the Public IPs back to the IP Pool

17. Extending CloudStack Networking
2. prepare (Network, Nic, DeployDestination, VmInfo)
1. prepare (part of start vm)
Network Network Element PluggableService
Manager
Needs to be added as of 5/2/2012 Device Configuration
MyDnsDeviceSer Admin API (CRUD)
DnsService
vice
3. addDnsRecord(ip, fqdn)
Demonstrates one way to MyDnsDeviceMa MySQL
MyDnsElement
inform an external DNS nager
server when an instance
starts. AgentManag
4.Enqueue AddDnsRecord er Queue
Classes shaded blue form a
plugin / service bundle to
integrate an external DNS MyDnsDeviceRes
server. Clients of the ource
instance can then use DNS
names to access the 5.API call to Dns Device
instance.