An introduction to Apache libcloud, a python module to abstract the differences in cloud providers API.

1. Sebastien Goasguen,
@sebgoa
Apache libcloud

2. Background

5. The Ten areas covered by the 60 core WS-* Specifications
WS-* Specification Area Typical Grid/Web Service Examples
1: Core Service Model XML, WSDL, SOAP
2: Service Internet WS-Addressing, WS-MessageDelivery; Reliable
Messaging WSRM; Efficient Messaging MOTM
3: Notification WS-Notification, WS-Eventing (Publish-
Subscribe)
4: Workflow and Transactions BPEL, WS-Choreography, WS-Coordination
5: Security WS-Security, WS-Trust, WS-Federation, SAML,
WS-SecureConversation
6: Service Discovery UDDI, WS-Discovery
7: System Metadata and State WSRF, WS-MetadataExchange, WS-Context
8: Management WSDM, WS-Management, WS-Transfer
9: Policy and Agreements WS-Policy, WS-Agreement
10: Portals and User Interfaces WSRP (Remote Portlets)
Web 2.0 Grids and Parallel Computing, Geoffrey Fox

6. WS-* Areas and Web 2.0
WS-* Specification Area Web 2.0 Approach
1: Core Service Model XML becomes optional but still useful
SOAP becomes JSON RSS ATOM
WSDL becomes REST with API as GET PUT etc.
Axis becomes XmlHttpRequest
2: Service Internet No special QoS. Use JMS or equivalent?
3: Notification Hard with HTTP without polling– JMS perhaps?
4: Workflow and Transactions
(no Transactions in Web 2.0)
Mashups, Google MapReduce
Scripting with PHP JavaScript ….
5: Security SSL, HTTPAuthentication/Authorization,
OpenID is Web 2.0 Single Sign on
6: Service Discovery http://www.programmableweb.com
7: System Metadata and State Processed by application – no system state –
Microformats are a universal metadata approach
8: Management==Interaction WS-Transfer style Protocols GET PUT etc.
9: Policy and Agreements Service dependent. Processed by application
10: Portals and User Interfaces Start Pages, AJAX and Widgets(Netvibes) Gadgets

7. No more EC2 SOAP

8. GCE is RESTfull

9. Standards and de-facto standards

10. Adapters
Client side

11. Interfaces
e.g EC2 interface to CloudStack: ec2stack
e.g GCE interface to CloudStack: gstack
Server side

12. Libcloud API

13. libcloud
• Python module that
provides a Cloud
Provider API
abstraction
• Ability to write apps
using multiple
providers
• Now with an
improved CloudStack
driver

14. Functionality
Manages:
• Compute nodes
• Data volumes
• DNS
• Load balancers

15. DNS providers
6 providers:
• Route53
• Rackspace
• Linode
• Zerigo
• Gandi
• hostvirtual

16. LB providers
7 providers:
• EC2 ELB
• CloudStack
• Rackspace
• Ninefold
• gce
• Gogrid
• brightbox

17. Compute Providers
40 providers including:
• CloudStack
• OpenStack
• Opennebula
• Rackspace
• EC2
• GCE

18. Installation
pip install apache-libcloud
Or from source:
git clone https://git-wip-
us.apache.org/repos/asf/libcloud.git
python ./setup.py install

19. Test installation
$ python
Python 2.7.5 (default, May 19 2013,
13:26:46)
[GCC 4.2.1 Compatible Apple Clang 4.1
((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or
"license" for more information.
>>> import libcloud

20. Base API
Base Classes
• Node()
• NodeImage()
• NodeSize()
• NodeLocation()
• StorageVolume()
• NodeDriver()

21. Base API
Base Methods
• list_nodes()
• list_images()
• list_sizes()
• list_locations()
• create_node()
• destroy_node()
• create_volume()
• …..

22. New API
• list_key_pairs()
• create_key_pair()
• get_key_pair()
• delete_key_pair()
• Possible to promote new API as soon as
there is support in several providers.

23. Deploying nodes
Create a node and execute a script to
configure it
deploy_node()
script=ScriptDeployment(wordpress)
msd = MultiStepDeployment([script])
deploy_node(name='apachecon',image=image,siz
e=size,ex_keyname=mykey,pub_key_identity=</p
ath/to/key>,deploy=msd)

24. Contributing to libcloud
http://docs.libcloud.apache.org

25. Github integration
http://github.com/apache/libcloud

26. CloudStack Driver
Apache Libcloud

27. Ecosystem
Contributing to Apache CloudStack

28. API

29. Extension methods
For specific drivers, e.g:
def ex_list_networks()
def ex_create_security_group(self,
name, description)

30. CloudStack + libcloud
from libcloud.compute.types import Provider
from libcloud.compute.providers import
get_driver
Driver = get_driver(Provider.CLOUDSTACK)
conn=Driver(key=apikey,secret=secretkey,secure=T
rue,host=host,path=path)

31. CloudStack/libcloud basics
conn.list_locations()
images=conn.list_images()
offerings=conn.list_sizes()
conn.list_nodes()
conn.create_node(name='yoyo’,
image=images[0],
size=offerings[0],
extra_args={'keypair':’testkey'})

32. CloudStack basic zones
Instances isolated at Layer 3 through
security groups
Usual access to instances through ssh key
pairs.
Similar to AWS EC2

33. CloudStack Basic Zone
conn.ex_list_security_groups()
conn.ex_create_security_group(name=’libcloud')
conn.ex_authorize_security_group_ingress(security
groupname=’libcloud',protocol='TCP',startport=22,
cidrlist='0.0.0.0/0')
conn.ex_delete_security_group(’libcloud’)

34. CloudStack Advanced Zone
Creates isolated guest networks (L2
isolation).
Need to manage IP and port forwarding
rules, possibly NAT and firewalls.

35. Advanced zone
ex_list_networks()
ex_list_public_ips()
ex_create_port_forwarding_rule()
ex_create_ip_forwarding_rule()
ex_allocate_public_ip()
ex_release_public_ip()

36. CloudStack “inherited” Drivers
Apache Libcloud

37. KTUCloud

38. iKoula
http://www.ikoula.com

39. Exoscale
http://exoscale.ch

40. from libcloud.compute.types import Provider
from libcloud.compute.providers import
get_driver
Driver = get_driver(Provider.IKOULA)
conn = Driver(key=apikey, secret=secretkey)

41. from libcloud.compute.types import Provider
from libcloud.compute.providers import
get_driver
apikey = os.getenv('EXOSCALE_API_KEY')
secretkey = os.getenv('EXOSCALE_SECRET_KEY')
Driver = get_driver(Provider.EXOSCALE)
conn = Driver(key=apikey, secret=secretkey)

42. Time for Demos ?
Apache Libcloud

43. Conclusions
• Use libcloud to abstract API differences.
• Useful if you are using multiple cloud
providers
• Actively developed and easy to contribute
to
• @libcloud
• dev@libcloud.apache.org