Cloud computing is more than a buzz-phrase it’s a transformative IT paradigm shift. The emphasis in the cloud is on elasticity, scalability, agility and open. Not just open standards but open APIs and open source. The delivery of software is also going through a paradigm shift. Open source software was often a commoditization of a market leader; Unix to Linux or Oracle to MySQL what’s changing is that the iterative nature, user context and the motto of releasing early and often are driving real innovation in open source. This session will cover those essential open source technologies for delivering cloud computing in the enterprise. Speaker Bio: Mark Hinkle is the Senior Director, Open Source Solutions at Citrix Systems Inc. He joined Citrix as a result of their July 2011 acquisition of Cloud.com where he was their Vice President of Community. He is currently responsible for Citrix open source efforts around the open source cloud computing platform, Apache CloudStack and the Xen Hypervisor. Previously he was the VP of Community at Zenoss Inc., a producer of the open source application, server, and network management software, where he grew the Zenoss Core project to over 100,000 users and 20,000 organizations on all seven continents. He also is a longtime open source expert and author having served as Editor-in-Chief for both LinuxWorld Magazine and Enterprise Open Source Magazine. His blog on open source, technology, and new media can be found at http://www.socializedsoftware.com.

1. Essential Open Source Software
for
Building the Open Cloud
Mark R. Hinkle
Sr. Director , OPEN SOURCE SOLUTIONS
Citrix Systems INC.
@mrhinkle
mrhinkle@gmail.com
Cloud Computing Expo East 2013

2. Mark Hinkle, Sr. Director, Open Source Solutions
• Dedicated
to
the
success
of
the
Apache
CloudStack,
Open
Daylight
&
Xen
Project
Communi3es
on
Citrix
behalf
• Run
BuildACloud.org
learning
ac3vi3es
all
over
the
world
• Joined
Citrix
via
Cloud.com
acquisi3on
July
2011
• Zenoss
Core
Open
Source
project
to
100,000
users,
1.5
million
downloads
• Former
LinuxWorld
Magazine
Editor-­‐in-­‐Chief
• Open
Management
ConsorGum
organizer
• Author
-­‐
“Windows
to
Linux
Business
Desktop
MigraGon”
–
Thomson
• NetDirector
Project
-­‐
Open
Source
Configura3on
Management
• Some3mes
Author
and
Blogger
at
SocializedSoJware.com
• NetworkWorld
Open
Source
Subnet
Essential OSS for Building the Open Cloud by @mrhinkle
2

3. Why Open Source and the Cloud Computing?
• User-­‐Driven
Context
from
Solving
Real
Problems
• Lower
Barrier
to
Par3cipa3on
• Larger
user
base,
users
helping
users
• Aggressive
release
cycles
stay
current
with
the
state-­‐of-­‐the-­‐art
• Open
Source
innova3ng
faster
than
commercial
• Open
data,
Open
standards,
Open
APIs
Essential OSS for Building the Open Cloud by @mrhinkle
3

4. Quick Cloud Computing Overview or
the Obligatory “What is the Cloud
Explanation”

5. Five Characteristics of Cloud
1. On-­‐Demand
Self-­‐Service
2. Broad
Network
Access
3. Resource
Pooling
4. Rapid
Elas3city
5. Measured
Service
Essential OSS for Building the Open Cloud by @mrhinkle
5

6. Cloud Computing Service Models
USER CLOUD a.k.a. SOFTWARE AS A SERVICE
Single application, multi-tenancy, network-based, one-to-many delivery of
applications, all users have same access to features.
Examples: Salesforce.com, Google Docs, Red Hat Network/RHEL
DEVELOPMENT CLOUD a.k.a. PLATFORM-AS-A-SERVICE
Application developer model, Application deployed to an elastic service that
autoscales, low administrative overhead. No concept of virtual machines or
operating system. Code it and deploy it.
Examples: VMware CloudFoundry, Google AppEngine, Windows Azure,
Rackspace Sites, Red Hat OpenShift, Active State Stackato, Appfog
SYSTEMS CLOUD a.k.a INFRASTRUCTURE-AS-A-SERVICE
Servers and storage are made available in a scalable way over a network.
Examples: EC2,Rackspace CloudFiles, OpenStack, CloudStack,
Eucalyptus, OpenNebula
Essential OSS for Building the Open Cloud by @mrhinkle
6

7. Deployment Models: Public, Private & Hybrid
Essential OSS for Building the Open Cloud by @mrhinkle
7

8. Building Open Source Clouds

9. First Question…What do you want to do?
ü Self-­‐Service
enterprise
workloads
ü Customer
facing
Applica3ons
ü Dev/Test
environments
ü Provide
Something-­‐as-­‐a-­‐Service
ü Deploy
similar
workloads
at
scale
ü All
of
the
Above
9
Essential OSS for Building the Open Cloud by @mrhinkle

10. Generic Cloud Architecture
Essential OSS for Building the Open Cloud by @mrhinkle
10

11. Building Open Source Clouds
Virtualized
Compute
Virtualized
Storage
Virtualized
Networking
Infrastructure
Orchestra3on
Pla^orm-­‐as-­‐a-­‐Service
Infrastructure-­‐as-­‐a-­‐Service
Ops
Management
Tools
Enterprise 3rd Party Developers
Essential OSS for Building the Open Cloud by @mrhinkle
Config
Mgmt
Monitoring
Provisioning
Orchestra3on

12. Choose
what
you
want,
customize
for
needs.
Essential OSS for Building the Open Cloud by @mrhinkle

13. The Three Virtual Pillars of Cloud
Computing
13
Compute
Network
Storage

14. Hypervisors
Open
Source
• Xen,
Project
Xen
Cloud
PlaMorm
(XCP)
• KVM
–
Kernel-­‐based
VirtualizaGon
• VirtualBox*
-­‐
Oracle
supported
Virtualiza3on
Solu3ons
• OpenVZ*
-­‐
Container-­‐based,
Similar
to
Solaris
Containers
or
BSD
Zones
• LXC
–
User
Space
chrooted
installs
Proprietary
• VMware
• Citrix
Xenserver
(based
• Microsoc
Hyper-­‐V
• OracleVM
(Based
on
OS
Xen)
Essential OSS for Building the Open Cloud by @mrhinkle
14

15. Scale-Up or Scale-Out
VerGcal
Scaling
(Scale-­‐Up)
Allocate
addi3onal
resources
to
VMs,
requires
a
reboot,
no
need
for
distributed
app
logic,
single-­‐point
of
OS
failure
Horizontal
Scaling
(Scale-­‐Out)
Applica3on
needs
logic
to
work
in
distributed
fashion
(e.g.
HA-­‐Proxy
and
Apache,
Hadoop)
Essential OSS for Building the Open Cloud by @mrhinkle
15

16. Sourcing Cloud Appliances
Tool/Project
What
you
can
do
with
them
Bitnami
BitNami
provides
free,
ready
to
run
environments
for
your
favorite
open
source
web
applica3ons
and
frameworks,
including
Drupal,
Joomla!,
Wordpress,
PHP,
Rails,
Django
and
many
more.
Boxgrinder
BoxGrinder
is
a
set
of
projects
that
help
you
grind
out
appliances
for
mul3ple
virtualiza3on
and
Cloud
providers
Oz
Command-­‐line
tool
that
has
the
ability
to
create
images
for
common
Linux
distribu3ons
to
run
on
KVM
SUSE
Studio
SUSE
Studio
supports
building
and
deploying
directly
to
cloud
services
such
as
Amazon
EC2.
Essential OSS for Building the Open Cloud by @mrhinkle
16

17. Cloud Computing Storage
Project
DescripGon
Ceph
Distributed
file
storage
system
developed
by
DreamHost
GlusterFS
Scale
Out
NAS
system
aggrega3ng
storage
over
Ethernet
or
Infiniband
OpenStack
Storage
Long-­‐term
object
storage
system
Riak
CS
Riak
CS
is
open
source
socware
designed
to
provide
simple,
available,
distributed
cloud
storage
at
any
scale.
Riak
CS
is
S3-­‐
API
compa3ble
and
supports
per-­‐tenant
repor3ng
for
billing
and
metering
use
cases.
Sheepdog
Distributed
storage
for
KVM
hypervisors
Essential OSS for Building the Open Cloud by @mrhinkle
17

18. Software Defined Networking (SDN)

19. Cloud Promise, Reality and Networks
Cloud
Promise
Cloud
Reality
Centralized
ConfiguraGon
and
AutomaGon
Without
true
virtualiza3on,
network
devices
must
s3ll
be
manually
configured.
Instant
Self-­‐Service
Provisioning
In
a
physical
network,
it
could
take
a
long
3me
for
network
engineer
to
provision
new
services.
ElasGcity
and
Scalability
By
horizontally
scaling
up
the
physical
network,
elas3city
is
lost.
Designed
for
Failure
Failover
can
be
automated
and
physical
network
limita3ons
can
be
alleviated.
Source: Midokura
Essential OSS for Building the Open Cloud by @mrhinkle
19

20. Open Flow
OpenFlow
enables
networks
to
evolve,
by
giving
a
remote
controller
the
power
to
modify
the
behavior
of
network
devices,
through
a
well-­‐defined
"forwarding
instruc3on
set".
The
growing
OpenFlow
ecosystem
now
includes
routers,
switches,
virtual
switches,
and
access
points
from
a
range
of
vendors.
Image from http://www.openflow.org/documents/openflow-wp-latest.pdf
Essential OSS for Building the Open Cloud by @mrhinkle
20

21. Overview of Software Defined Networking
Business
Applica3ons
Network
Services
SDN
Control
Software
API API
Network
Devices
Network
Devices
Network
Devices
Network
Devices
Network
Devices
Network
Devices
Application
Layer
Control
Layer
Infrastructure
Layer
Control Data Plane Interface (e.g. OpenFlow)
Essential OSS for Building the Open Cloud by @mrhinkle
21

22. Software Defined Networking (SDN)
Project Description
Floodlight
The
Floodlight
controller
is
an
enterprise-­‐class,
Apache-­‐licensed,
Java-­‐based
OpenFlow
Controller.
Indigo
Indigo
is
an
open
source
project
to
support
OpenFlow
on
a
range
of
physical
switches.
By
leveraging
hardware
features
of
Ethernet
switch
ASICs,
Indigo
supports
high
rates
for
high
port
counts,
up
to
48
10-­‐gigabit
ports.
Mul3ple
gigabit
pla^orms
with
10-­‐gigabit
uplinks
are
also
supported.
Open
Daylight
Linux
Founda3on
Collabora3ve
Project
based
on
Cisco
One
Controller
and
plugins
from
numerous
vendors
in
development.
E.g
IBM
DOVE
OpenStack
“Quantum”
Networking
Pluggable,
scalable,
API-­‐driven
network
and
IP
management
Open
vSwitch
Open
vSwitch
is
a
open
source
(ASL
2.0),
mul3layer
virtual
switch
designed
to
enable
massive
network
automa3on
through
programma3c
extension,
while
s3ll
suppor3ng
standard
management
interfaces
and
protocols
(e.g.
NetFlow,
sFlow,
SPAN,
RSPAN,
CLI,
LACP,
802.1ag).
Essential OSS for Building the Open Cloud by @mrhinkle
22

23. Cloud Orchestration

24. Compute Clouds (IaaS)
Year
Started
License
VirtualizaGon
Technologies
Apache
CloudStack
2008
Apache
Xenserver,
Xen
Cloud
Pla^orm,
KVM,
VMware
(Hyper-­‐V
developing)
Eucalyptus
2006
GPL
Xen,
KVM,
VMware
(commercial
version)
OpenNebula
2005
Apache
Xen,
KVM,
VMware
OpenStack
2010
(Developed
by
NASA
by
Anso
Labs
previously)
Apache
VMware
ESX
and
ESXi,
,
Xen,
Xen
Cloud
Pla^orm
KVM,
LXC,
QEMU
and
Virtual
Box
Numerous companies are building cloud software on OpenStack including Nebula, Piston Inc., CloudScaling
Essential OSS for Building the Open Cloud by @mrhinkle
24

25. OpenStack – Ecosystem of Projects
Enterprise
Message
Queue
based
on
Rabbit
MQ
(ESB)
Object
Storage
“Swic”
Image
Service
“Glance
”
Compute
“Nova”
Dashboard
“Horizon”
KVM,
VMware,
Xen
Cloud
Pla^orm
Ceph,
Gluster
Advanced
Cloud
and
Networking
services
accessing
the
Quantum
API
Firewall
Service
Gateway
Service
Quantum
Networking
Fabric
REST
API
Plugins
OpenvSwitch
Quantum
Plugin-­‐ins
Iden3ty
Services
“Keystone”
API
20+ Collective projects hosted at: https://launchpad.net/openstack
Essential OSS for Building the Open Cloud by @mrhinkle
25

26. Platform-as-a-Service (PaaS)
Project
Year
Started
Sponsors
Languages/Frameworks
CloudFoundry
2011
VMware
Spring
for
Java,
Ruby
for
Rails
and
Sinatra,
node.js,
Grails,
Scala
on
Lic
and
more
via
partners
(e.g.
Python,
PHP)
Cloudify
2012
Gigaspaces
[Groovy
for
deployment
recipes]
OpenShic
2011
Red
Hat
Java,
Ruby,
PHP,
Perl
and
Python
Stackato
2012
Ac3veState
Java,
Python,
PHP,
Ruby,
Perl,
Node.js,
others
WSO2
Stratus
2010
WSO2
Jboss,
Java
EE6
Essential OSS for Building the Open Cloud by @mrhinkle
26

27. Cloud APIs
• jclouds
• libcloud
• deltacloud
• fog
Essential OSS for Building the Open Cloud by @mrhinkle
27

28. Cloud Management Tools

29. Automation in the Cloud
Meat Cloud Cloud Operations
Essential OSS for Building the Open Cloud by @mrhinkle
29

30. 4 Types of Management Tools
Provisioning
Installation of operating systems and other software
Configuration Management
Sets the parameters for servers, can specify installation parameters
Orchestration/Automation
Automate tasks across systems
Monitoring
Records errors and health of IT infrastructure
Essential OSS for Building the Open Cloud by @mrhinkle
30

31. Management Toolchains
Configura3on
Patching
and
Provisioning
Monitoring
Toolchain (n):
A set of tools where
the output of one
tool becomes the
input of another tool
Essential OSS for Building the Open Cloud by @mrhinkle
31

32. Provisioning
Project Installation Targets
Apache
Provisionr(incuba3ng)
Can
provision
10s
to
1000s
of
machines
on
various
clouds.
Cobbler
Distributed
virtual
infrastructure
using
koan
(kickstart
of
a
network
to
PXE
boot
VMs)
for
Red
Hat,
OpenSUSE
Fedora,
Debian,
Ubuntu
VMs
Crowbar
(Bare
metal
provisioning)
JuJu
Public
Clouds
-­‐
Amazon
Web
Services
HP
Cloud,
Private
OpenStack
clouds,
Bare
Metal
via
MAAS.
Salt
Cloud
Tool
to
provision
“salted”
VMs
that
can
then
be
updated
by
a
central
server
via
ZeroMQ
Essential OSS for Building the Open Cloud by @mrhinkle
32

33. Configuration Management Tools
Project
Year
Started
Language
License
Client/Server
Cfengine
1993
C
Apache
Yes
Chef
2009
Ruby
Apache
Chef
Solo
–
No
Chef
Server
-­‐
Yes
Puppet
2004
Ruby
GPL
Yes
&
standalone
Salt
2011
Python
Apache
yes
Essential OSS for Building the Open Cloud by @mrhinkle
33

34. Automation/Orchestration Tools
Project
DescripGon
Ansible
Ansible's
SSH-­‐key
based
access
allows
contributors
to
the
Fedora
Project
to
assist
in
automa3ng
infrastructure
while
having
access
limited
appropriately.
Capistrano
U3lity
and
framework
for
execu3ng
commands
in
parallel
on
mul3ple
remote
machines,
via
SSH.
It
uses
a
simple
DSL
that
allows
you
to
define
tasks,
which
may
be
applied
to
machines
in
certain
roles
RunDeck
Rundeck
is
an
open-­‐source
process
automa3on
and
command
orchestra3on
tool
with
a
web
console.
Func
Func
provides
a
two-­‐way
authen3cated
system
for
generically
execu3ng
tasks,
integra3ons
with
puppet
and
cobbler.
MCollec3ve
The
Marionese
Collec3ve
AKA
MCollec3ve
is
a
framework
to
build
server
orchestra3on
or
parallel
job
execu3on
systems.
Salt
Execute
arbitrary
shell
commands
or
choose
from
dozens
of
pre-­‐built
modules
of
common
(or
complex)
commands.
Scalr
Provide
scaling
across
mul3ple
cloud
compu3ng
pla^orms,
integrates
with
Chef.
Essential OSS for Building the Open Cloud by @mrhinkle
34

35. Configuration Management Tools
Project
Year
Started
Language
License
Client/Server
Cfengine
1993
C
Apache
Yes
Chef
2009
Ruby
Apache
Chef
Solo
–
No
Chef
Server
-­‐
Yes
Puppet
2004
Ruby
GPL
Yes
&
standalone
Salt
2011
Python
Apache
yes
Essential OSS for Building the Open Cloud by @mrhinkle
35

36. Automation/Orchestration Tools
Project
DescripGon
Ansible
Ansible's
SSH-­‐key
based
access
allows
contributors
to
the
Fedora
Project
to
assist
in
automa3ng
infrastructure
while
having
access
limited
appropriately.
Capistrano
U3lity
and
framework
for
execu3ng
commands
in
parallel
on
mul3ple
remote
machines,
via
SSH.
It
uses
a
simple
DSL
that
allows
you
to
define
tasks,
which
may
be
applied
to
machines
in
certain
roles
RunDeck
Rundeck
is
an
open-­‐source
process
automa3on
and
command
orchestra3on
tool
with
a
web
console.
Func
Func
provides
a
two-­‐way
authen3cated
system
for
generically
execu3ng
tasks,
integra3ons
with
puppet
and
cobbler.
MCollec3ve
The
Marionese
Collec3ve
AKA
MCollec3ve
is
a
framework
to
build
server
orchestra3on
or
parallel
job
execu3on
systems.
Salt
Execute
arbitrary
shell
commands
or
choose
from
dozens
of
pre-­‐built
modules
of
common
(or
complex)
commands.
Scalr
Provide
scaling
across
mul3ple
cloud
compu3ng
pla^orms,
integrates
with
Chef.
Essential OSS for Building the Open Cloud by @mrhinkle
36

37. Monitoring Tools
License
Type
of
Monitoring
CollecGon
Methods
Cac3
/
RRDTool
GPL
Performance
SNMP,
syslog
Graphite
Apache
2.0
Performance
Agent
Nagios
GPL
Availability
SNMP,TCP,
ICMP,
IPMI,
syslog
Zabbix
GPL
Availability/
Performance
and
more
SNMP,
TCP/ICMP,
IPMI,
Synthe3c
Transac3ons
Zenoss
GPL
Availability,
Performance,
Event
Management
SNMP,
ICMP,
SSH,
syslog,
WMI
Essential OSS for Building the Open Cloud by @mrhinkle
37

38. Conceptual Automated Toolchain
BootStrapped
Image
CloudStack
OpenStack
ConfiguraGon
Puppet
Chef
Start/Stop
Services
RunDeck
Capistrano
MCollec3ve
Provision
Cobbler
SUSE
Stuido
Monitoring
Nagios
Zenoss
Cac3
Generate
Images
SUSE
Studio
BoxGrinder
Essential OSS for Building the Open Cloud by @mrhinkle
38

39. Questions?
Essential OSS for Building the Open Cloud by @mrhinkle
39
Slides
Can
be
Viewed
and
Downloaded
at:
hsp://www.slideshare.net/socializedsocware/
Copyright Mark R. Hinkle, available under the CCbySA license some rights reserved. 2012 -2013
Professional: mark.hinkle@citrix.com
Personal: mrhinkle@gmail.com
Phone: 919.228.8049
Personal: http://www.socializedsoftware.com
Twitter: @mrhinkle
Mark R. Hinkle
Senior Director,
Open Source Solutions
Citrix Systems Inc.
Open Source Enthusiast