The Green Evolution of EMOTIVE Cloud EMOTIVE Cloud: The BSC’s IaaS open-source solution for Cloud Computing

The Green Evolution of EMOTIVE Cloud
EMOTIVE Cloud: The BSC’s IaaS open-source solution for Cloud Computing

Alexandre Vaqué Brull

Master in Computer Architecture, Network and Systems
Department of Computer Architecture
Universitat Politècnica de Catalunya

Advisors: Jordi Torres and Jordi Guitart

September 2011

Acknowledgements
Voldria aprofitar aquestes línies per donar els meus sincers agraïments a totes les persones que m'han
ajudat a realitzar aquest projecte:

Voldria mostrar la meva plena gratitud als Drs. Jordi Torres i Jordi Guitart per la confiança dipositada
en mi, per brindar-me l'oportunitat de formar part d'aquest magnífic equip i per la seva constant i gran
ajuda. D‟ells he aprés molt tant en l‟àmbit professional, com també personal.

Volia agrair també al Dr. Iñigo Goiri la seva constant ajuda i la seva paciència. I pels seus coneixe-
ments compartits.

He après molt de tots tres i han sigut un pilars bàsics per l‟elaboració d‟aquesta màster tesis. Sense
l‟ajuda d‟aquestes persones no ho hagués pogut aconseguir. Sempre els tindré molt present per tot lo
que han fet per mi.

També volia agrair el suport de la meva parella Sara Serra en aquest projecte i a la meva família la
constant ajuda que m‟han ofert sempre. Moltes gràcies a tots els companys de feina, professors (en
destacat el Dr. David Carrera), amics i familiars que m'han recolzat i s'han preocupat en tot moment,
que encara que no els mencioni de forma explícita, no els hi puc negar el meu sincer agraïment.

3

Table of Contents
1 Abstract ............................................................................................................................................. 1
2 Introduction ....................................................................................................................................... 2
2.1 Motivation .................................................................................................................................. 2
2.2 Goals ........................................................................................................................................... 3
2.3 Document Structure .................................................................................................................... 4
3 Background and Related Work ......................................................................................................... 5
3.1 Virtualization .............................................................................................................................. 5
3.1.1 Virtualization technologies ................................................................................................. 7
3.1.2 Libvirt: The virtualization API ............................................................................................ 8
3.2 Cloud Computing ....................................................................................................................... 9
3.2.1 Virtualization Unlocks Cloud Computing........................................................................... 9
3.3 Cloud Middleware .................................................................................................................... 10
3.3.1 OpenNebula ...................................................................................................................... 10
3.3.2 Eucalyptus ..........................................................................................................................11
3.3.3 Open Stack ........................................................................................................................ 12
3.4 Interoperability in the Cloud .................................................................................................... 13
3.4.1 API OCCI.......................................................................................................................... 14
3.4.2 TCloud .............................................................................................................................. 17
3.5 Green Computing ..................................................................................................................... 18
3.5.1 The greening of the Cloud ................................................................................................ 19
4 Contribution .................................................................................................................................... 21
4.1 EMOTIVE ORIGINAL ............................................................................................................ 21
4.2 The Evolution of EMOTIVE .................................................................................................... 23
4.2.1 Introduction ....................................................................................................................... 23
4.2.2 New Modular Architecture................................................................................................ 24
4.2.3 Green IT evolution ............................................................................................................ 25
4.3 Extended Virtualization support in EMOTIVE ........................................................................ 25
4.4 EMOTIVE Networks ............................................................................................................... 28
4.4.1 VLAN................................................................................................................................ 28
4.4.2 VPN ................................................................................................................................... 31
4.4.3 Networks by Software are Green ...................................................................................... 32
4.5 EMOTIVE Interoperability ...................................................................................................... 34
4.5.1 API OCCI and Web Services ............................................................................................ 34
4.5.2 REST vs. SOAP ................................................................................................................ 35
4

4.5.3 API OCCI in EMOTIVE................................................................................................... 35
4.6 EMOTIVE for Green Computing............................................................................................. 39
4.6.1 Green Hypervisor Comparison ......................................................................................... 39
4.6.2 Architecture comparison (Atom-Xeon-Hybrid)................................................................ 42
4.6.3 Middleware scheduling comparison (OpenNebula and EMOTIVE) ................................ 46
4.6.4 Middlewares qualitative comparison ................................................................................ 50
5 Conclusions ..................................................................................................................................... 52
5.1 Summary .................................................................................................................................. 52
5.2 Publications .............................................................................................................................. 52
5.3 Suggestions for future work ..................................................................................................... 52
6 References ....................................................................................................................................... 55

5

1 Abstract

In recent years, projects and researches about Cloud and Green Computing are growing. A new
Computational generation is emerging where ecological concerns have strengthened. Technological
growth leads to increased energy consumption, and thus this new Computational Generation is emitting
much CO2 in the atmosphere.

Cloud Computing has achieved to move centralized physical resources to shared virtual resources,
reducing costs and maintenance while increasing efficiency. Virtualization plays an important role in
Cloud Computing (especially IaaS) because it allows to create environments "on demand" within the
extensive Cloud platforms. This way, we can accommodate more than one virtual machine in the same
host, avoiding the expense traditional physical machine for only one service.

In this sense, we can say that virtualization environments together with Cloud Computing platforms
provide the IT market of extraordinary flexibility, a wide range of possible configurations and virtually
unlimited resources.

In the BarcelonaTech (UPC) and BSC (Barcelona Supercomputing Center), we have a Cloud Platform
based on virtualizated environments that is called EMOTIVE Cloud (Elastic Management of Tasks in
Virtualized Environments). So EMOTIVE middleware provides virtualized environments to the users
and allows executing tasks.

This project main aim is to expand and evolve the capabilities of EMOTIVE platform to improve
certain limitations. This includes enabling easy interoperability with other Cloud providers thanks to a
new architecture and adding new functionalities such as new hypervisors and networks management. In
addition, we perform new studies to evaluate these new features and extend the research and
investigations related to EMOTIVE and Cloud Computing.

In addition, we will also provide a qualitative comparison of EMOTIVE running over different
hypervisors (e.g. XEN, KVM and Virtual Box), with respect to other IaaS open-source solutions (e.g.
OpenNebula) and different kinds of computer architectures (XEON, ATOM and Hybrids solutions).
This will help users to find the best solution according to their needs: performance, green, agility,
usability, etc.

Keywords: Virtualization, Service provider, Cloud, Resource management, Green-computing,
Scheduling, consolidation, EMOTIVE Cloud, IaaS, Virtualization, Open source.

1

2 Introduction
2.1 Motivation
The use of Cloud Computing and Virtualization is increasing. It is evident with the large current offer
of Cloud Computing services. If we focus on Cloud infrastructure providers we can find a wide variety
of services and products. Cloud will be attractive to companies, whether small or large, because Cloud
computing will replace many internal IT traditional services, for example many internal IT resources
will be externalized in the Cloud. But keep in mind that the IT and Cloud Computing sector is
constantly growing, thus increasing energy consumption. It is an important problem that affects CO2
world emissions. According to Greenpeace information, the datacenters that store Cloud Computing
services will triplicate their emissions to the atmosphere in 2020. There is currently some research to
try to improve the consumption/performance ratio, especially for datacenters and supercomputers.

BSC and UPC are researching about Cloud and Green Computing. The middleware EMOTIVE Cloud
(Elastic Management Of Tasks In Virtualized Environments) is being used by BSC to do research in
Cloud Computing (thanks to its Infrastructure as a Service (IaaS) capabilities), as well as in some re-
search projects such as BREIN (1), OPTIMIS (2), VENUS-C (3), NUBA (4) and others. EMOTIVE
enables the smart management of virtual environments using different scheduling policies. Additional-
ly, it is very easy to extend thanks to its modular Web Service architecture evolved in this project. In
this master thesis, we research to improve this middleware and its green features.

Similarly to EMOTIVE, there are many other Cloud providers like Amazon EC2 Cloud (5) and Cloud
middleware such as OpenNebula (6). These new technologies allow the creation of virtual machines of
demand, and they even allow outsourcing virtual machines from external Clouds and migrating these
machines between Clouds. These are powerful features that can help to improve service availability and
enhance resource management and power consumption. However, currently the problem is that there is
no preset standards to use these new features homogenously between providers. In general, most
providers only offer their own proprietary interface or support only a given virtualization technology
(which conditionates what kind of virtual machine images they can use).

For this reason, new research is emerging to define possible standards, which allow providers to offer a
common interface and thus to interoperate among them and above all to create big Cloud communities.
Additionally, Cloud middleware are evolving to support different virtualization technologies, or at least
different image formats, to enable real interoperability.

As commented, providers must be also conscious of their energy impact, and try to reduce their energy
usage. For this reason, they are currently incorporating more complex management policies that allow
them to use the energy in an efficient way, or to consider the ecological impact when taking decisions.
In this sense, building providers based on heterogeneous architectures, with different energy
consumption profiles, is a powerful tool for these policies to really achieve their energy-related goals.

This is one of the reasons that justify why an efficient management of Cloud providers is mandatory
today. There is an important tradeoff to be solved between the performance of the applications running
in the provider and its power consumption. The goal is to fulfill the performance requirements of the
applications while minimizing power consumption. Apart from this, there are other aspects that require

2

complex management in Cloud providers. For instance, offering resources in the Cloud is no longer
about offering raw virtual machines. Clients require virtual machines to come prepared to be used to
deploy distributed services without painful configuration steps. In this sense, support for the creation
and management of virtual networks among virtual machines is a must.

2.2 Goals
In this section, we describe this thesis‟ goals, which aim to resolve some limitations presented in the
motivation section.

The main goal of the project is to extend the capabilities of the EMOTIVE Cloud platform and partly to
expand our research focused to be greener. Wherefore the new features added in EMOTIVE will try to
find a green feature.

The main project goals are:
1. To add new features to expand EMOTIVE with new functionalities, easy management, and a green
approach. We will add new hypervisors and virtual networks management.
2. To redesign the architecture and interfaces of EMOTIVE Cloud. We will go from Web Services
SOAP to RESTful.
3. Add a new OCCI interface to have interoperability with other Cloud providers and middlewares.
4. Be able to use hybrid computer architectures to exploit Green factor in EMOTIVE.
5. To study, compare and evaluate EMOTIVE:
- Running with different computer architectures (XEON, ATOM and Hybrid
solutions).
- Running with different hypervisors (KVM, XEN and Virtual Box).
- With other Middlewares such as OpenNebula, Eucalyptus and also OpenStack.

Next we will proceed to detail the goals commented:

(Goal #1) The most important new feature added in EMOTIVE is the API Xen substitution to add
Libvirt API. Initially, EMOTIVE could only use Xen hypervisor with XEN API. Libvirt API allows to
expand the number of hypervisor inside EMOTIVE. So now we can use Xen, KVM, and VirtualBox
hypervisors. In addition, Libvirt can be used to manage virtual networks within virtual machines. In
this way, we add also network management in EMOTIVE Cloud. And also we developed VPNs
creation. To sum up we will be able to create, destroy, list and edit VLANs and VPN (point to point and
multipoint to point).
(Goal #2) Another important new feature is the EMOTIVE architecture restructuration by adapting
Web Services SOAP to Web Services RESTful.

(Goal #3) Having the new REST architecture, we adapt the RESTful methods to have compatibility
with API OCCI, which makes possible to have Interoperability between Clouds. With this new feature
also we have to keep in mind the EMOTIVE compatibility with Amazon EC2. So now EMOTIVE
allows the interoperability thanks to API OCCI and Amazon EC2 interfaces.

(Goal #4) After adding these features, we have developed the EMOTIVE middleware in order to have
more green aspects. So we tried to improve and adapt EMOTIVE in this sense. An interesting green
3

aspect is that we can use hybrid computer architectures between Xeon and Atom servers together.

(Goal #5). Having achieved these goals, to test new features we made some comparisons between
EMOTIVE and different computer architectures, hypervisors and middlewares. These comparisons
allow to see the new EMOTIVE power, explore the new possibilities, and study the green approach.

We expect the research in this project to be useful for future research and to be used in other research
projects. For example, with these new functionalities we can adapt EMOTIVE to the requirements of
NUBA national research project, or the VENUS-C European project.

2.3 Document Structure
This document is organized as follows. Chapter 2 explains the goals and main motivations of this
master thesis. Chapter 3 presents some background and related work about basic concepts, including
also the state of the art.

Chapter 4 is the most important because describes the actual work completed as part of this Master
Thesis. On the one hand, introduces the technical aspects of EMOTIVE, describes and analyzes the
implementation, describe its new architecture, how it is implemented and the new features added in the
middleware. The subchapter 4.6 explains the evaluation results, describing how EMOTIVE has been
tested and comparing and summarizing the results.

Finally, Chapter 5 presents the conclusions and proposes possible future work related to this project.

4

3 Background and Related Work

First of all it is necessary to understand the basic ideas of Virtualization and Cloud Computing to
understand this master thesis. In this chapter, we present Virtualization concepts and technologies,
Cloud Computing Infrastructure ideas, Middleware products similar to EMOTIVE, Interoperability to
do federation and hybrid Clouds and Green Computing state of the art.
These topics are very important and useful to know better the puzzle of the Cloud Computing. It is
important to know that Cloud Computing without Virtualization cannot offer any of the new kind of
services that offers nowadays.
We will talk about the middleware layer and the infrastructure-as-a-service (IaaS) system. Cloud
Providers use Middlewares to offer their Services. Service Providers in the Cloud oﬀer complex
services ready to be used. And the customers would pay depending on the volume of consumed
services, as we do with electricity or water.
This middleware is a set of resources. These resources can form Private Clouds, Public Clouds and
Hybrid Clouds. These days, one of the challenges we face is to manage better the wide range of
Clouds. In this sense, nowadays the number of companies that need more than one cloud is increasing.
Moreover, they can migrate VM from one Cloud to another and exploit new advantages. This
represents a challenge and we will show a new approach for future research
New technologies go mainly in this direction. The most well-known and useful to help to organize the
infrastructure of Cloud Computing and Virtualization are: VMWare, Xen, KVM, OpenNebula,
Eucalyptus, Libvirt and others. In the next chapters we will talk about them. They are all good
technologies to virtualize environments, manage, create some abstraction layer, etc.
With this new technology and its new possibilities, we can both contribute to improve the green
computing and we can achieve other new challenges as Manageability and Self -*, Federation &
Interoperability, Virtualization, Elasticity and Adaptability.

3.1 Virtualization
A virtual machine is an implementation of a machine (i.e. a computer) that executes programs like a
physical machine. The main idea of Virtualization is simulate machines into machines.

There are a lot of advantages to virtualize. A person who has not experienced the benefits of
virtualization often asks what the big deal is. The often repeated argument about consolidation is
flawed: after all, instead of running 10 VMs, each serving a single application, you can have one multi-
purpose server. To have to maintain a single system means less work. True? Not quite. There are
reasons for separating applications in either physical or virtual computers. The first option is clearly too
expensive in so many ways (not only initial costs, but also power, cooling, space, maintenance) and
VMs even have some advantages over real computers.

Listed below there are a number of advantages:
- Isolation: Virtual machines are independent
- Security: Each machine has privileged independent access and is very easy to backup and restore
5

virtual-machines.
- Hardware and Software flexibility (CPU, memory, disk, net, OS, etc)
- Agility: Instant server
- Portability: (file) easy to clone or to be transported to another server
- Services Consolidation, CPD cost savings, high availability and other similar features.
- Lab environments. For example, in order to test new software.
- Cloud Computing environments. They help to deal with increased load.

Hypervisor is a supervising master program that manages Virtual Machines. A computing layer which
allows multiple operating systems to run on a host computer at the same time. The role of the
Hypervisor is to support Guest Operating Systems on a single machine. It was originally developed in
the 1970s as part of the IBM S/360.Many modern variants have been done by different developers.

Figure 1- Hypervisor architecture

Virtualization types:
Computer systems introduce a division into levels of abstraction separated by well defined interfaces.
There are several ways to achieve Virtualization with different levels of abstraction obtaining different
advantages and disadvantages.

Virtualization introduces an abstraction layer to show to higher layers a different overlayed system.
Virtualization can be classified by abstracts system layer interface. These types are Hardware
Emulation, Full virtualization, and Paravirtualization.

 Hardware Emulation simulates a complete hardware
allowing an unmodified OS to be run. Every instruction is
simulated on the underlying hardware. It is a problem because
this means that high performance is lost. It is emulator; this kind
of VM can even run multiple virtual machines. It is slower than
full virtualization and paravirtualization. Figure 2 - Hardware Emulation

Many techniques are used to implement emulation. Some examples of emulation are Virtual Box,
Bochs and QEMU.

 Full virtualization uses a virtual machine that mediates
between guest operating system and the native hardware. It is
slower than native hardware because there is another layer, the
hypervisor. The hypervisor do the mediations between hardware
and OS. Figure 3 - Full virtualization

One of the biggest advantages of full virtualization is that guest OS can run unmodified. Certain
machine instructions must be trapped and handled within the hypervisor because the underlying
hardware is not owned by an operating system but instead, it is shared by it through the hypervisor.

6

There are alternatives how VMWare ESX, Parallels and KVM.

 Paravirtualization is similar it to full virtualization. It
uses a hypervisor for shared access to the underlying hardware
but integrates some virtualization parts into the operating
system. The guest system needs to be modified for the
hypervisor.
Figure 4 - Paravirtualization
To implement this method, hypervisor offers an API to be used by the
guest OS. This call is called hypercall. This issue increases the performance with regard to
fullvirtualization.

A disadvantage is that guest OS needs to be modified. But the advantages is: It can run multiple
different operating systems concurrently. Xen is an example of paravirtualization.

3.1.1 Virtualization technologies
XEN: Xen is a virtual-machine monitor for x86, x86-64, Itanium and PowerPC. It allows several guest
operating systems to execute on the same computer hardware concurrently. Xen is GNU, provides
paravirtualization and supports x86 and x64 processors. It allows several guest operating systems to
execute on one physical machine simultaneously. The first guest OS is known as Domain-0 in Xen
terminology. Domain-0 automatically boots whenever Xen software boots. Users need to login on
Domain-0 to execute other guest OS.
KVM: Kernel-based Virtual Machine is a Linux kernel virtualization infrastructure. KVM currently
supports native virtualization using Intel VT or AMD-V. Limited support for paravirtualization is also
available for Linux guests and Windows in the form of a paravirtual network driver, a balloon driver to
affect operation of the guest virtual memory manager, and CPU optimization for Linux guests.
VMWare: is commercial virtualization software that provides full virtualization. VMWare has many
flavors such as VMWare Workstation, VMWare Server and VMWare ESX that provide different levels
of flexibility and functionality. VMWare is highly portable as it is independent of the underlying
physical hardware, making it possible to create one instance of a guest OS using VMWare and copy it
to many physical systems.
VirtualBox: a newcomer to the ranks of virtualization market, Sun Microsystems VirtualBox is a
software package that provides paravirtualization. It was initially developed by a German company,
Innotek, but now it is under the control of Sun Microsystems as part of the Sun xVM virtualization
platforms. It supports Linux, Mac, Solaris, and Windows platforms as the host OS.
Amazon EC2: Amazon Elastic Compute Cloud (Amazon EC2) (5) is a web service that provides
resizable compute capacity in the Cloud. It is designed to make web-scale computing easier for
developers. This technology allows having virtual-machines in the Cloud. So EC2 is more IaaS
solution than Virtualization technology.

7

3.1.2 Libvirt: The virtualization API
Introduction
Libvirt (7) is a generic virtualization API to interact with the virtualization capabilities that supports
many different hypervisors Xen, QEMU, KVM, ESX, VirtualBox, etc.) and can be used in User Mode
Linux... This has a common layer of abstraction and control for virtual machines. Also it can manage
virtual networks and virtual storage. The main components of Libvirt are a control daemon a stable C
language API and a shell environment. Libvirt has a long term stable C API and a JAVA API used in
EMOTIVE Cloud.

There are a number of open-source projects that use Libvirt how: virt-manager, virt-install and virtual
machine control mechanism. It is important to know that Libvirt stores information configuration in an
XML (independent to hypervisor).‫‏‬

In conclusion Libvirt is a toolkit to interact with the virtualization capabilities of recent versions of
Linux and OSes. It is a free software available under the GNU Lesser General Public License. And now
has a set of bindings for common languages.

Libvirt is a virtualization technology that is used in a lot of Cloud Computing Infrastructure products.
This technology helps to unlock virtualization in the Cloud, similar to Amazon EC2 and other Cloud
technologies.

Libvirt API:
C and Java API have these main methods:
· initialize · destroy · getNodeCPUNum
· getDomain · pause / unpause · getNodeCPUSpeed
· getDomainID · save · getCPUNum
· getDomainNameList · restore · setCPU
· create · migrate · updateCache

Libvirt (Virsh)‫‏‬
Virsh is a user Mode Linux to easy management
about Virtual Machines.
 List domains: virsh list --all
 Boot a domain: virsh create
/etc/xen/machine1.xml
 Connect to domain console: virsh console
machine1
 Reboot domain: virsh reboot machine1
 Shutdown domain: virsh undefine machine1
 Kill domain: virsh shutdown machine1
 All virsh options: virsh help

Figure 5 - Libvirt

8

3.2 Cloud Computing
Cloud Computing is composed by three layers: IaaS (Infrastructure as a Service), PaaS (Platform as a
Service) and SaaS (Software as a Service). In this master thesis we focus in the lower layer of Cloud
Computing so we talk about Infrastructure as a Service (8) (9) (10).

Within the Cloud Computing IaaS there are three types: Public, Private, and Hybrid Cloud. Public
Cloud is a totally outsourced payment service and it does not depend on the company. A good example
is Google's services products, Salesforce.com and LotusLive iNotes, which are open to any user
through a subscription. Other example is pay-as-you-go compute capacity like Amazon EC2 and
goGrid. Private Cloud is the kind of Cloud that is created specifically for a company, maximizing
existing resources and acquiring new ones. This is achieved through virtualization. These clouds are
only for internal use of the company. This is a new model, but with a wide scope and range. For
example we can find eucalyptus, OpenNebula, IBM Smart Business Storage Cloud. And the Hybrid
Cloud is a fusion of the two previous types. There may occasionally need more resources than those
offered by the Private Cloud. Thus, when needed they can be extended from private to public, giving
the company an optimal environment for their production processes and projects.

Hybrid Clouds will be very used in the next future, because then can improve and profit private existent
resources. And if needed, they can extend themselves using Public Cloud resources. To sum up, hybrid
clouds take advantage of both public cloud and private features and also contribute with new ones.

There are a growing number of providers offering IaaS solutions for elastic capacity, whereby server
instances are executed in their own infrastructure and billed on a utility computing basis (typically
virtual machines on a per instance per hour basis). There are also a number of commercial and open-
source products which seek to replicate this functionality in-house while exposing compatible
interfaces so as hybrid cloud operating environments can be created.

3.2.1 Virtualization Unlocks Cloud Computing
The use of Cloud Computing and Virtualization is increasing (11) (12), as is evident with the large
current offer of Cloud Computing services. If we focus on Cloud infrastructure providers we find
various services and content. Furthermore these technologies have the support of big companies.

As the possibilities of Cloud Computing and services offered through it are improving and expanded,
the companies will be increasingly attracted to this technology for its intrinsic value and simplicity.
Cloud computing will eliminate many aspects of IT that traditionally required to have internal IT
resources, which will be attractive to companies, whether small or large. To open a business in the
Cloud it is not necessary to spend initial capital (Capex) and only pay for its use (Opex).
In traditional IT researchers found that about 80% of local computing resources of a company are
underutilized (eg, it is possible that at some point we are only running two or three applications in our
computer software, and so we lose much of its computational potential), certainly would make sense to
incorporate many of these applications to the Cloud. These advantages are encouraging, not only
because of the cost reduction in software licensing, but also by reduced IT spending and the obvious
advantages that provides standardization. Moreover, the outsourcing of these functions allows
companies to focus on their core competencies, leaving the management of their servers, applications
and manage their data in the hands of companies specializing in this environment can offer a
continuous service.
9

Service providers feel the urgent need to make the transition from their network architectures,
computer, software, etc and their service delivery models to better adaptation to this new environment.
This is only possible if resources can be dynamically reconfigured to meet new service requests with
minimal human intervention.
Service providers should also be able to allocate dynamically increase or decrease the capacity of
resources with the least effort possible thanks to its programmability and to respond to unpredictable
demand curves, and nobody can predict which device application or service will be the next big driver
of bandwidth in the cloud. Moreover, there are other critical issues, such as latency, security and speed
of service, which entails the need for a network that will prioritize and manage traffic reliably.
It is becoming increasingly clear that the traditional approach to service delivery cannot long resist the
emerging innovations and their related applications. In this new era, service providers need to adopt a
perspective and a totally different approach to replace the current access to intelligent services,
automated service-oriented. Those who assume that their services should evolve to become a well
capable of adapting quickly to innovation initiated by virtualization and cloud computing to exploit and
transform new benefits services will succeed. Those who do not will face a number of difficulties.
"Virtualization Unlocks Cloud Computing" (13) is a phrase that defines, what is virtualization for
Cloud Computing. So virtualization unlocks Cloud Computing and the power of virtualization is the
key to enable the technology to cloud computing environments. In summary virtualization is a step
toward internal Cloud Computing. It must be clear that Virtualization is not part of Cloud Computing.
Virtualization is a technology that enables Cloud Computing.
Cloud means "outsourcing of IT technology". It is a model based on elastic infrastructure that can scale
up or down according to demand. Here the virtualization plays an important role.
There are at least five things that virtualization opens the door to cloud computing, and push faster
organizations that address:
- Enables economies of scale.
- Decouples users from implementation.
- Speed, flexibility, agility.
- Breaks software pricing and licensing.
- Enables, motivates chargeback.
So the evolution is first virtualization, later Private Cloud and now it is beginning the Cloud Computing
era.

3.3 Cloud Middleware
This section describes some Cloud Middleware whose functionalities are comparable to EMOTIVE.

3.3.1 OpenNebula
OpenNebula is a middleware to easily build any type of Cloud and has been designed to be integrated
with any networking and storage solution. OpenNebula has a big open-source community led by UCM,
fully open-source cloud software, and not open core.
OpenNebula can transform any data center into a flexible and agile virtual infrastructure which
dynamically adapts to the changing demands of the service workload.

10

OpenNebula manages storage, network and
virtualization technologies to enable the dynamic
placement on distributed infrastructures, combining
both data center resources and remote cloud
resources, according to allocation policies.
It is a flexible, extensible and with excellent
performance and scalability to manage a lot of Virtual
Machines. Figure 6 shows the architecture of
OpenNebula.
Figure 6 – ONE Architecture

It can manage private, public or hybrid Cloud solutions. Private Cloud with Xen, KVM and VMware,
Public Cloud supporting EC2 Query, OGF OCCI, Sunstone and vCloud APIs,… and Hybrid Cloud
with Amazon EC2, and other providers through Deltacloud.

3.3.2 Eucalyptus
Eucalyptus (Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems) is
an open-source solution that originates from an NSF funded research project at University of
California, Santa Barbara, primarily as a tool for cloud-computing research. Eucalyptus Cloud (14) (15)
is an infrastructure software that enables enterprises and government agencies to establish their own
private cloud environments on clusters.
Eucalyptus can make a more efficient use of their computing capacity, thus increasing productivity and
innovation.
The main feature about Eucalyptus is the compatibility with Amazon‟s EC2 interface, but the
infrastructure is designed to support multiple client-side interfaces. Eucalyptus is implemented using
commonly available Linux tools and basic Web-service technologies making it easy to install and
maintain.
The feature highlights is:
 Interface compatibility with EC2 (both Web service and Query interfaces)
 Manage environments with multiple hypervisors (Xen, KVM, vSphere, ESX, ESXi) under one
management console
 Stand-alone RPMs for non-Rocks RPM based systems
 Secure internal communication using SOAP with WS-security
 Overlay functionality requiring no modification to the target Linux environment
 Basic “Cloud Administrator” tools for system management and user accounting
 Advanced storage integration (iSCSI, SAN, NAS) enables you to easily connect and manage your
existing storage systems from within the Eucalyptus cloud
 The ability to configure multiple clusters, each with private internal network addresses, into a single
Cloud.
 Sophisticated user, group, and role management allows precise control of resources within a private
Cloud

11

Eucalyptus Architecture: Web Service Cloud

Eucalyptus Components:
 Cloud Controller (CLC) - The CLC is
responsible for exposing and managing the
underlying virtualized resources (servers
machines, network, and storage) via user-
facing APIs. The CLC uses an Amazon EC2
API and a Web-based user interface.
 Cluster Controller (CC) - The CC controls
the execution of virtual machines (VMs)
running on the nodes and manages the
virtual networking between VMs and
between VMs and external users.
Figure 7 – Eucalyptus Architecture
 Node Controller (NC) - The NC (through the
functionality of a hypervisor) controls VM
activities, including the execution, inspection, and
termination of VM instances.

3.3.3 Open Stack
At present time, people are talking about a new middleware developed by a large number of important
companies. To create an innovative and open-source Cloud Computing software for building reliable
cloud infrastructure.

This is called OpenStack, is a collaborative software project among several big important players in the
Cloud Computing space, designed to create freely available code, badly needed standards, and common
ground for the benefit of both Cloud providers and Cloud customers. The open source software model
has been proven to promote the standards and interoperability critical to the success of our industry.
The explosive growth of the internet can be attributed to open, universal standards like HTTP and
HTML.

OpenStack is an initiative for the definition of an open architecture for IaaS Cloud Computing and an
open-source project that is currently three:
 Openstack compute: for large-scale deployments of automatically provisioned virtual compute
instances.
 Openstack object storage: for large-scale, redundant storage of static objects.
 Openstack Image Service: provides discovery, registration, and delivery services for virtual disk
images.

Openstack has alpha version and is immature at the moment in comparation to OpenNebula and
Eucalypthus. For example, OpenNebula is maturing, proven, and works pretty well. Although
OpenStack has a big community with a big number of developers and companies so I have no doubt it
will be a very nice piece of software in the future. But if you want to put it into production today, it is
too hard because it is alpha version and not even close to feature-complete.

12

3.4 Interoperability in the Cloud
This chapter talks about the state of the art to create an open standard for Cloud Computing.
Many people in the industry believe it is critically important for the Cloud to be open and share
concerns about the private nature of the leading Cloud platforms. In fact, there are already a few
projects focused on the goal of a truly open source Cloud with mass adoption.

However the present Cloud offers have followed this trend and are largely private. No one benefits
from a fractured landscape of closed and incompatible Clouds where migration is difficult to do and
true Cloud transparency is impossible.

Nowadays the solution to interoperate between Clouds is using Web Services. For example, Amazon
EC2 has a web service interface to manage his own virtual-machines, VMware has a vCloud interface
and other products have their own web services interface. But the problem is that these interfaces are
private and based on their own Clouds Computing. Eucalyptus Systems considers API AWS (Amazon
Web Services) the default standard for the industry because of its popularity. Eucalyptus Cloud is an
open-source virtualization middleware but it uses an Amazon EC2 interface.
On the other hand, OpenNebula propose an open source interface called OCCI very easy to use and to
extend. OCCI was originally initiated by UCM (Complutense University of Madrid) and now the Open
Cloud Computing interface comprises a set of open community-lead specifications delivered through
the Open Grid Forum.
TID (Telefonica I+D), as well, proposed tCloud. It is based on vCloud API specification 0.8.
δ-cloud RedHat‟s project defines a web service API for interacting with Cloud service providers and
resources in those Clouds in a unified manner. Libcloud is a pure python client library for interacting
with many of the popular Cloud server providers.
Usually the APIs wars have been a crucial strategic plan to control the technology platforms and their
associated markets. I don‟t know if δ-cloud or OCCI will be an API reference of the next years. But I
have a clear idea. An open standard API should emerge. Now both have very good fundamentals and
are present in many discussion forums.
During the development of this project, we chose to use API OCCI. Now we see that the δ-cloud is
taking a lot of relevance. So we will keep an eye on the evolution of δ-Cloud and Libcloud.
We would like to highlight that our OCCI API is used in the project NUBA, VENUS-C, OPTIMIS and
our own project EMOTIVE. We presented our experience to adopt the OCCI API in the 2010 OGF30
OCCI conference in Belgium Chapter 5 .
The δ-cloud domain is not yet as rich as OCCI's, because it cannot manage storage and network yet.
But meanwhile it puts strong emphasis on virtual machine templates (or in δ-cloud terms: 'hardware
profiles').
Architecturally it follows a slightly different approach. Unlike OCCI which focuses on the
communication layer, δ-cloud seems to support a 2-layered approach and focuses on the API layer built
on top of the communication layer. Meanwhile it is not yet language-agnostic: currently it only
supports Ruby. The whole thing (the architectural concept) remembers me Libvirt (another RedHat lib).
13

Since RedHat recently submitted the library to Apache it might gain popularity.
Another interesting thing is that OpenNebula's δ-cloud driver is built on top of OCCI.
In conclusion, OCCI is supported by Open Grid Forum (OGF). There is so much duplicate work going
on in Cloud standards as: Cloud Computing Interoperability Forum, Open Grid Forum (OGF), Open
Cloud Computing, Interface Working Group, DMTF Open Cloud Standards Incubator, GoGrid API
(CC licensed), Sun Cloud API (CC licensed), Amazon Web Services as “de-facto” (i.e. as Eucalyptus
and Nimbus have proceeded).

I believe that the next step is creating another
standard working group that sits on top of them all.
OCCI and δ-cloud are in the first-line.

3.4.1 API OCCI
The initial goal of the API OCCI is to provide an
extensible interface to Cloud Infrastructure
Services (IaaS). The OCCI API is a RESTful
service, allowing for the development of
interoperable tools for common tasks including
deployment (create, control), autonomic scaling
and monitoring Cloud resources.

Figure 8 - OCCI interaction

This API allow for:
 Consumers to interact with Cloud computing infrastructure on an ad-hoc basis (e.g. deploy,
start, stop, restart)
 Integrators to offer advanced management services
 Aggregators to offer a single common interface to multiple providers
 Providers to offer a standard interface that is compatible with available tools
 Vendors of grids/Clouds to offer standard interfaces for dynamically scalable service delivery
in their products

Figure 9 – OCCI schema

OCCI have been as modular as possible to facilitate future extension.
The core protocol is completely generic, describing how to connect to a single entry point, authenticate,
14

search and CRUD operations (Create, Retrieve, Update and Delete resources) using existing standards
including HTTP (Plain Text), TLS, Oauth, JSON and Atom/Pub. State control (start, stop, restart),
billing, performance, etc. The scope of the specification will be all high level functionality required for
the life-cycle management of virtual machines (or workloads) running on virtualization technologies
(or containers) supporting service elasticity. Using a simplified service lifecycle model, it supports the
most common life cycle states offered by Cloud providers.
Simply by standardizing at this level OCCI may well become the HTTP of Cloud Computing. There is
a good article that compares OCCI API with HTML. Is OCCI the HTTP of Cloud Computing? (16)

Figure 10 OCCI Simple Life Cycle

RESTful web services
A RESTful Web Service offers these HTTP methods: GET, PUT, POST and DELETE. OCCI API is a
RESTful service and has methods associated with each resource type: Pool Resources (collection of
elements owned by a given user) and Entry Resources (single entry within a given collection).

Figure 11 – OCCI (Compute, Network and Storage)

GET PUT POST DELETE
Pool Resources (PR) to list all the entry x to create a new entry x
resources in that pool resource
resource owned by the user

Entry Resources to list the information to update the resource x to delete the resource
(ER) associated with that (only supported by the
resource COMPUTE resource)

XML format is used to represent COMPUTE, NETWORK and DISK resources; as well as the
collection of them (Pool Resources, PRs).
15

POOL RESOURCE
 References a URI for the ER.

Example:
<COMPUTES>
<COMPUTE href="http://www.opennebula.org/compute/234">
</COMPUTES>

NETWORK
 ID, the uuid of the network
 NAME, describing the network
 ADDRESS, of the network
 SIZE, of the network, defaults to C

Example:
<NETWORK>
<ID> 123 </ID>
<NAME> Blue Network </NAME>
<ADDRESS> 192.168.0.1 </ADDRESS>
<SIZE> C </SIZE>
</NETWORK>

STORAGE
 ID, the uuid of the image
 NAME, describing the image
 SIZE, of the image in MBs
 URL, pointer to the original image

Example:
<DISK>
<ID> 123 </ID>
<NAME> Ubuntu 9.04 LAMP </NAME>
<SIZE> 2048 </SIZE>
<URL> file:///images/ub untu/jaunty.img </URL>
</DISK>

COMPUTE RESOURCE
The compute element defines a virtual machine by specifying the configuration attributes. It is
more complex than previous resources commented:

 ID, the uuid of the virtual machine.
 NAME, describing the virtual machine.
 TYPE, a COMPUTE type specifies a CPU and memory capacity, valid types are small, medium
16

and large.
 STATE, the state of the COMPUTE. T
 DISKS, the block devices attached to the virtual machine (DISK, SWAP, FS)
 NICS, the network interfaces, defined with a list of NIC elements. (UUID, IP...)

Example:
<COMPUTE>
<ID>123AF</ID>
<NAME>Web Server</NAME>
<TYPE>small</TYPE>
<STATE>running</STATE>
<DISKS>
<DISK image=234 dev=sda1/>
<SWAP size=1024 dev=sda2/>
<FS size=1024 format=ext3 dev=sda3/>
</DISKS>
<NICS>
<NIC network=4567f ip="19.12.1.1"/>
<NIC network=0/>
</NICS>
</COMPUTE>

Return Codes
The OCCI Cloud API uses the following subset of HTTP Status codes.
200 OK: The request has succeeded. GET: an entity corresponding to the requested resource is sent in
the response.POST: an entity containing the result of the action. 201 Created, 202 Accepted, 204 No
Content.
The 4xx class of status code is intended for cases in which the client seems to have erred (400 Bad
Request, 401 Unauthorized, 403 Forbidden, 404 Not Found...).
And 5xx, the server failed to fulfill an apparently valid request. (500 Internal Server Error, 501 Not
Implemented ...)

Authentication
Authentication follows REST philosophy. It is recommended that the server-client communication is
performed over HTTPS to avoid sending user authentication information in plain text.

3.4.2 TCloud
TCloud API is based on vCloud API specification 0.8. In essence, compatibility for the main operations
and data types defined in vCloud is maintained by TCloud. TCloud provides extensions on advanced
Cloud Computing management capabilities including additional shared storage for service data,
network element provisioning, monitoring, snapshot management, and so on. TCloud API is focused on
adding network intelligence, reliability and security features to Cloud Computing. The goal of the
initiative is to provide the power of Cloud computing with the flexibility allowed by virtualization

Telefónica has released the TCloud API for Cloud Computing interoperability and submitted it to the
Distributed Management Task Force (DMTF). This shows Telefónica commitment to promote Cloud
17

interoperability and standardization. This releasing is good for telecommunications companies in
general, as it proves that telecommunications are skilled to define the technology in which its services
are based.

3.5 Green Computing
Nowadays the concept of Green Computing (or Green IT) (17) (18) is taking some relevance due to the
widespread concerns about issues such as climate change, recycling, biodegradability, etc. In this
chapter, we present the Green Computing focused on Cloud computing and virtualization. So we talk
about energy consumption that implies a certain level of CO2 emissions.
It is estimated that the IT sector causes 2% of global CO2 emissions and in some articles it is said that
the IT sector produces more carbon emissions than the world of aviation. For example some
information published in Daily Telegraph (19) talk about two Google searches produce the same CO2
as boiling a kettle.

Another important fact to keep in mind is that the IT sector is growing constantly, so this implies that
energetic consumption is growing. It is an important data that produces more CO2 emissions.
According to Greenpeace, world data centers to store Cloud computing services will triple emissions to
the atmosphere in 2020 (20). Currently there are several European researches to improve the
consumption-performance ratio, especially for datacenters and supercomputers. This master thesis
makes research about this subject.
The most part of the pollution produced by Cloud computing is caused by world datacenters. Power
usage effectiveness (PUE) is a measure of how efficiently is a computer datacenter using its power;
specifically, how much of the power is actually used by the computing equipment (in contrast to
cooling and other overhead). The overall average is 2, which means that for every watt used in running
the equipment, has spent other one to keep cooled this.
Virtualization technology enables the consolidation of multiple workloads and always increases
efficiency and most of them save energy on IT equipment, for example, using a smaller number of
machines. Nevertheless, virtualization also produces some additional overheads and as a result the rest
of the datacenter may be running less efficient. VM creation and VM migration is the most stand out
overhead in Virtualization moreover, Virtualization technology use a new layer called hypervisor this
means that a little high performance is lost. But virtualization overhead is insignificant in contrast to its
advantages.
BSC and UPC use EMOTIVE to research about green virtualization. Virtualization requires high
density datacenters. Remaining servers are running at higher power consumption levels but there are
different ways to deal with this. Energy-aware Scheduling in Virtualized Data Centers (21) is one
example. This presents a Scheduler that uses a mathematical algorithm to manage Virtual Machines.
Basically, the Scheduler makes a virtual machine consolidation in the minimum number of physical
machines and the unused machines will be shutdowned. Scheduler manages the Virtual Machines
migrating this to some physical machines to use the maximum possible performance of physical
machines. And if there are some physical machines without any virtual machine and in idle mode,
scheduler shutdown this server to consume less energy.

There are other solutions presented by others researches but the main idea is the same, the changes are
in the idea of the scheduler algorithm. Nowadays commercial middlewares use a generic scheduler as
backfilling that pack the VMs in the cluster nodes to reduce VM fragmentation and use the minimum
number of physical servers.
18

Other aspects to considerate that it is important to understand the impact of Virtualization on Data
Center Physical Infrastructure (22). The virtualization power savings can produce unexpected results
and probably you need to upgrade power and cooling infrastructure to take advantage of the savings
opportunity that virtualization offers. You need to worry about power and cooling when it is
virtualizing. You may need additional cooling in some physical areas to improve the power efficiency.
For example, it cools some areas dynamically depending on the load. Care should be taken to examine
the impact on power and cooling because each data center virtualization is different.

3.5.1 The greening of the Cloud
Traditional On-Premise vs Cloud Computing:
The Cloud computing business motivation is that the resources solutions on demand promise greater
flexibility, dynamic, timely and green solution than traditional on-premise computing.
Therefore, we must bear in mind that migrate certain parts or all of a classic on-premise IT to Cloud
can provide scalability, can reduce the costs of physical growth, reduce costs and reduce energy use (8).
On-premise computing needs an initial capital investment, maintenance and the costs of future updates.
In contrast Cloud does not need an important initial cost so it has a lower initial investment because
Cloud offer elasticity and pay-as-you go cost model.
It is interesting to find which solution is better. But it is more interesting the utilization into both
solutions together to keep the best features of each.
In the paper (23) there is an interesting analysis of cost and performance between “Traditional On-
Premise” with Cloud Computing classifying the various types of costs CapEX (CAPitalEXpenditures)
and OpEX (OPerationalEXpenditures) depending on the attribute to be analyzed (Infrastructure,
Business, Physical Resources, Network, Performance, Energy, budget, etc.). In short we can discuss
that in Cloud Computing there are more OpEx and in the traditional on-premise there are more CapEx.

Nowadays generally on-premise infrastructure run with low utilization, sometimes it goes down up to 5
to 10 percent of average utilization. Data centers that utilize Cloud technologies are more efficient than
traditional data centers. Energy is usually lost through server under utilization, because most of the
time these servers are just in idle mode. But in a Cloud environment the system is managed to run at
the highest efficiency. In addition, data center planning allows better power utilization. In traditional
data centers, they can have cooling problems and you can run out of space for more servers. There is
also a consortium of Cloud providers, who assure that its members optimize their data centers to mi-
nimize power consumption.

On-premise solution can be better, whenever if we have a constant full utilization of the IT
infrastructure. This often happens in large companies that offers constant services around the world.
For example, in their start Facebook was using Amazon services but finally due to their large increase
in business, Facebook built his own data center, adapted to their business needs.

Cloud solutions are highly recommended in most areas. But an important factor to consider is that
network latency influences negatively in the response time of the Cloud solutions. Traditional On-
Premise Computing usually have better network latency and therefore the response time gives better
results for the solution.
And also a lot of companies prefer to use on-premise infrastructure for its data privacy and protection.
In this project, however, we do not focus on Cloud security.
19

In conclusion, it is necessary to analyze the CapEx/OpEx balance and the consumption depending on
each own case. As we have said, what we study is the energy consumption and in the next chapter, we
would like to show our structure solution of hybrid architectures and how they are a solution to reduce
energy consumption, without losing too much performance.

Global vision in Green Computing:
We should have a global vision of both sides of the coin, Cloud computing and Green Computing.
Basically it is known the relationship between the Cloud and eco-efficient IT (24).

The end-user adoption of Cloud implies that there will be a turning point in 2011, Cloud Computing
utilization is increasing. Therefore, more power consumption in the Cloud.
There are several things to consider having good eco-efficiency in IT.
 Each year the number of datacenters is growing and also the power consumption.
 New legislation penalizes the excessive consumption of energy. New measures should be taken
into account.
 Expensive equipment uses more energy than what they truly need.
So the Cloud computing alleviates these things:
 This allows better management of internal uses of resources (sharing resources or virtualizing).
 Reduce peakload using schedulers to manage a rapid provisioning and deprovisioning.
 Reduce unnecessary use of resources in some points.
 When there is hardware (compute or storage) limits use outsourcing.
Green Clouds? Not all Cloud Computing cases are greener compared to in-house IT:
 Energy efficiency. Providers aim to have efficient operations.
 Sometimes Cloud Providers overlooked reporting metrics consumption of server resources.
 Recycling. Service providers would have to detail IT recycling policies.
Key points in the adoption and incentives: In 2008 many companies such as Yahoo, Google, Microsoft
and others, struggled to be in a good position as the greener operators of datacenters. But the thing is
that is not that easy to measure which data center is more efficient. In fact, several studies (commercial
and non commercial) have emerged to try to find a common standard measurement.
Follow the Moon:
Before finishing this chapter, we present an example of a very interesting technique. Follow the moon
concept means to reduce energy consumption and expenditure taking advantage of nighttime tempera-
tures and lower electricity rates, and so have their computing resources chase in day/night boundary.
i.e., migrate to data centers where it is night time. After all, always it is night somewhere in the world.
Although, these techniques have certain limitations because it is necessary to have similar settings, re-
quires visibility among others and the bandwidth latency presented increase. Well, this theory must be
carefully studied as many companies have done.

The Key technologies that can follow the moon are: virtualization, modularization, consolidation and
outsourcing appropriate. These are key strategies to achieve eco-efficient IT.
20

4 Contribution

4.1 EMOTIVE ORIGINAL
EMOTIVE (Elastic Management of Tasks in Virtualized Environments) (25) is the Barcelona
Supercomputing Center (BSC)‟s and Barcelona Tech (UPC) IaaS open-source solution for Cloud
Computing, which results from BSC‟s previous experience in European projects such as BREIN (1)
and SORMA (26). EMOTIVE provides users with elastic fully customized virtual environments
(supporting Xen hypervisor) in which to execute services. Further, it simplifies the development of new
middleware services for Cloud systems by supporting resource allocation and monitoring, data
management, live migration, and checkpoints.

EMOTIVE middleware can be categorized as an IaaS solution, since it provides the users with
virtualized environments where they can execute their tasks without any extra effort. These VMs,
which aim to fulfill the user requirements in terms of software and system capabilities, are
transparently managed by EMOTIVE in order to exploit the provider‟s resources. EMOTIVE can
easily be extended with multiple scheduling policies in order to manage the VMs using different
criteria.

Figure 12 - EMOTIVE Cloud architecture

Figure 12 illustrates the EMOTIVE Cloud architecture, which is mainly composed by three different
and modular layers: the fabrics infrastructure, the node management (VRMM), and the global
Scheduler. VRMM component is responsible for managing the life cycle of the virtual machines
(creation, destruction, submit tasks, etc.). Scheduler layer is responsible for distributing the tasks and
virtual machines between physical nodes. The Scheduler includes support for efficient virtual machines
migration, managing checkpoints, and system configuration and organization between different virtual
environments.

21

The Virtualization Fabrics layer comprises the physical resources where the VM will run. This layer
wraps the virtualized resources and offers them to the upper layers. EMOTIVE makes use of the Xen
API which makes it able to use Xen virtualization technologies. Furthermore, it implements a distributed
shared file system (DFS) that supports efficient VM creation, migration (to move VMs across provider‟s hosts
without stopping the execution), and checkpointing (to resume VM execution upon hardware failure). This file
system also supports a global repository where users can upload the input files needed by the applications (i.e.
data stage-in) and retrieve the resulting ones (i.e. data stage-out).
The data infrastructure offers a distributed storage for supporting virtualization capabilities such as
migration and checkpoint support, and it can use differents kinds of storage. It distributes the data
among the cluster nodes. It uses NFS in order to make the data of every node available from the other
nodes. Thanks to this technique, VMs can be moved between nodes without losing connection. This
capability allows new approaches such as consolidating the global system or giving more resources to a
given application if the node is not able to do this locally.
In addition, this data infrastructure allows each VM accessing data required by the user by using a
shared repository also distributed among the nodes. It also allows storing data in the system in order to
be reused later from other VMs.
The Virtual Machine Manager layer is implemented by means of the Virtualized Resource
Management and Monitoring (VRMM). This layer comprises all the local resource management
decisions (i.e. in a single node) and it is in charge of managing the physical resources of a node and
distributing these resources among all the VMs running on that node. In addition, it continuously
monitors the resource usage of these services and the fulfillment of their SLAs. If any SLA violation is
detected, an adaptation process for requesting more resources to the provider is started, first locally in
each node, then globally in the provider, and finally with other providers. This layer is also in charge of
creating and maintaining the whole virtual machine life cycle (create, destroy, migrate, etc.) and
executing tasks described by means of a JSDL file (27).
In addition, the VtM comprises all the local resource management decisions (i.e. in a single host): it is
in charge of managing the physical resources of a host and dynamically distributing these resources
among all the VMs running on that host in order to fulfill their respective Service Level Agreements
(SLAs). EMOTIVE allows specifying fine-grain resource-level guarantees in the SLA (e.g. amount of
computing power allocated to a given VM over time), which are clearly superior to the availability
guarantees supported by common providers such as Amazon EC2 (5).
Furthermore, EMOTIVE also has, by means of the VtME component, the capability to use external
resources, like the ones in public Cloud providers (i.e. Amazon EC2). This feature allows an
EMOTIVE-enabled provider to be involved in a Cloud federation (insourcing/outsourcing) and create
public, private, and hybrid clouds.

On the other side, the Resource Monitor (RM) component continuously monitors the status of tasks and
resources. This status is stored in a historical database, but it can be also used to assess the fulfillment
of the SLAs of the applications. If any SLA violation is detected, an adaptation process for requesting
more resources to the provider is started, first locally in each host, then globally in the provider, and
finally with other providers.

Finally, the Virtual Machine Scheduler layer comprises all the global VM placement decisions, both
among different providers in a Cloud federation and different hosts in a single provider. This layer is in
charge of deciding where a VM will be executed and managing its location during the execution (e.g.
migration of VMs across provider‟s hosts, cancellation of VMs, resumption of VM execution from a

22

checkpoint upon hardware failure, etc.). As a rule of thumb, the Scheduler tries to consolidate the VMs
in the provider‟s physical resources to optimize their use, while allocating enough resources to fulfill
the agreed SLAs.

Moreover, this framework allows multiple schedulers with different policies and capabilities such as
machine learning, prediction, economic, fault tolerance, semantic description, or SLA enforcement. In
this sense, it can use a simplistic Round Robin, or a consolidation-aware scheduling like Backfilling.
This is achieved thanks to the usage of a common interface (Web Service SOAP) that allows
developing new schedulers with different features and policies.

In conclusion, the main capabilities supported by EMOTIVE are summarized herewith:
· VMs creation on demand, according to application requirements.
· Monitoring of task and resource status, including historical information.
· Consolidation of VMs in the provider‟s physical resources to optimize their use.
· VM placement and fine-grain dynamic resource distribution based on Service Level Agreements
(SLAs).
· Efficient live migration of VMs across provider nodes.
· A checkpoint/recovery system to resume task execution upon hardware failure (thus achieving
fault tolerance).
· Ability to create additional VMs on external clouds when the local provider is overloaded.
· Data management services for supporting VM creation and the migration and checkpoint
mechanisms, and also to allow users to provide input (i.e. data stage-in) and retrieve output (i.e. data
stage-out).

4.2 The Evolution of EMOTIVE

4.2.1 Introduction
In this chapter, we will describe the main new functionalities added to EMOTIVE in this master thesis
and how they are implemented.
We can summarize these new features for EMOTIVE in the next schema:
 New support through Libvirt API.
 Replace Xen Api with Virsh API
 Support for KVM and Virtual Box hypervisors.
 Contextualization and easy installation. Virtual image management and creation from
the scratch.
 New modular architecture with Web Services RESTful. This makes EMOTIVE easier to
be extended.
 Compatibility with API OCCI thanks to new RESTful architecture. This allows
achieving interoperability in the Cloud.
 GUI adaption to new RESTful architecture.
 Initializing OVF support (alpha version)

23

 New EMOTIVE functionalities
 VLAN Network management
 Easy management and creation of Virtual Networks (VPN support).
 EMOTIVE evolution to be Greener
 EMOTIVE in new hybrid architectures.
 EMOTIVE adaption to NUBA National project.

4.2.2 New Modular Architecture
One of the main features of EMOTIVE is its modular and distributed architecture. EMOTIVE was
originally designed using a distributed SOAP architecture but now it uses Web Services RESTful
architecture. This architecture allows to use only some parts of EMOTIVE and supports agile and
dynamic construction of new Cloud environments.

Its Web Service REST interface makes EMOTIVE highly interoperable with other Cloud solutions. In
particular, we encourage using the Open Cloud Computing Interface (OCCI) (16), which allows
EMOTIVE to be interoperable with other Cloud middleware supporting this interface.

In addition to API OCCI compatibility, EMOTIVE can use the external Cloud Amazon EC2 via the
Web Service EC2 API and also it could interpret OVF files (28). The capability to use external
resources from Amazon EC2 allows to be involved in a Cloud federation (insourcing/outsourcing) and
create public, private, and hybrid clouds. Hybrid possibility allows to dynamically increase or decrease
the capacity of resources with the least effort possible thanks to its scheduler and to respond to
unpredictable demand curves.

Figure 13 - Internal EMOTIVE Architecture
24

With the new RESTful architecture we need to adapt every EMOTIVE module [Figure 13]: the GUI
module, Scheduler modules and VtM modules. In the chapter 4.5.2 we can see the full details of the
new architecture and the new interfaces used in EMOTIVE and in the chapter 4.3 we can see the evo-
lution of EMOTIVE API as we replaced XEN API to use Libvirt with API.

EMOTIVE is able to support different schedulers. All EMOTIVE Schedulers use Web Services
communications to receive an external client request or send requests to the Virtual Management (VtM)
component to manage virtual machines, submit jobs or other functionalities. A client request is a Web
Service RESTful with standard operations (like GET, POST, PUT, DELETE for HTTP). The client for
Scheduler uses this RESTful interface to request some work. This RESTful interface is very easy to
extend, being a powerful functionality.

4.2.3 Green IT evolution
In this master thesis, we analyze the energy impact of running virtualized environments using different
kind of computer architectures such as Xeon, Atom and hybrid approaches. Also we analyze the
different kind of hypervisors that now EMOTIVE can run thanks to new features added.
Additionally we exploit the EMOTIVE capability of supporting multiple scheduling policies in order to
compare the energy impact of some of them. These include a simplistic Scheduler that uses Round
Robin to balance VMs among nodes and a backfilling-alike Scheduler that does VM consolidation to
save energy. The latter is described in a previous work (29) (21). It basically uses a backfilling
scheduler to consolidate VMs in data center nodes according to multiple facets while optimizing the
provider‟s profit. In particular, it considers energy efficiency, virtualization overheads, fault tolerance,
and SLA violation penalties, while adding the ability to outsource resources to external providers. This
scheduler saves energy by shutting down the machines that are idle. In addition, we also compare the
power consumption of this EMOTIVE Scheduler with generic Schedulers solutions presented in other
middlewares solution such as OpenNebula.

We also added new features to simplify the process of virtual machines preparation and creation. For
instance, we can create Virtual Networks (VLAN) and VPNs (with SSL and PPTP protocol). With this
simple and dynamic network management we can set up dynamic Cloud environments on demand. We
can create VLANs or VPNs on demand at a certain instant of time, without the need of extend the
physical network infrastructure because it is not necessary to buy and put new hardware in the
infrastructure. In this sense, we can say that this feature has some ecological value.

4.3 Extended Virtualization support in EMOTIVE
We see that the EMOTIVE platform needs to expand and not depend only on Xen hypervisor. We want
it to be as generic as possible. Therefore we have redefined the architecture, rewritten the API and
made it compatible with Libvirt [Figure 13]. With this compatibility we have more possibilities to ex-
pand the virtualization architecture to use other hypervisors. So a part of XEN we extend to use EMO-
TIVE with KVM and Virtual Box virtualization thanks to the new API used (Libvirt), abandoning the
old API (Xen API). So now we replace Xen API by Libvirt API, therefore we create a new API plat-
form that uses Libvirt. To support the new hypervisors also we need to deploy the operating system
environment with the hypervisor and Libvirt installation and its configuration (25).

Apart from extending the number of hypervisors supported, Libvirt has other features, such as VLAN
management. We will talk about this in the next chapters.
25

Now we describe how API Libvirt is integrated in the EMOTIVE Cloud platform. The starting point is
the actual Xen monitoring API (XenMonitor.java). This API has been completely rewritten to be
adapted to Libvirt. (VirtMonitor.java). This represents a significant step forward in monitoring the plat-
form, as it extends the possibilities of the platform making it compatible with most existing virtualiza-
tion systems like Xen, KVM, VirtualBox, etc. Figure 14 summarizes the mapping between the methods
used in XenMonitor and VirtMonitor.java.

We choose JAVA language to develop the new monitoring API, because EMOTIVE is fully developed
in JAVA and bash shell. In the project we use the famous software development environment called
Eclipse. We use another tools and plugins such us Maven and Subversion. Maven dynamically down-
loads Java libraries and Maven plug-ins from one or more repositories. And Subversion (SVN) is a tool
used by EMOTIVE software developers to manage changes within their source code tree.

It is necessary to know that EMOTIVE Cloud uses important Linux bash scripts in the low level of its
architecture to do the virtual machines contextualization (for dynamically creating VMs disk images
from scratch and configuring some values, on demand). Libvirt can be used as API C, API Java and in
console mode. It is a great API.

EMOTIVE has been evolving together with API Libvirt. EMOTIVE started using Libvirt version 0.4.0
and now it is using Libvirt 0.8.3.

It is important to know that Libvirt Project and most technology used in the project (eg. KVM, OCCI),
is under development and some methods that the library offers are in beta version and have some bugs.
This technology is evolving and new functionalities are appearing. The new architecture has been
refined and is evolving. This has been an extensive period of development and testing on the platform.
Also you can see that in VirtMonitor there are absences of some methods that XenMonitor has [Figure
14]. But EMOTIVE only uses some basic methods therefore not noticed a difference in the EMOTIVE
usability with the API replacement. Equally EMOTIVE could extend new functionalities using all Java
methods presented.

In conclusion we see that before EMOTIVE virtualized only with XEN hypervisor. Now we can
virtualize with XEN, KVM, and Virtual Box. Therefore we needed to change the architecture and Java
code and also we needed to change the bash scripts due to the virtual machines contextualization so
that three hypervisors are 100 % compatible with the middleware. It is interesting to know that is very
hard to use VMware ESX in EMOTIVE although Libvirt is compatible with VMWARE ESX. VMware
uses a private architecture and it is necessary to do a lot of changes in EMOTIVE architecture. Libvirt
uses a remote communication with VMware in contrast with the others hypervisors. The others
hypervisors use local petitions and it is easier to be used in EMOTIVE. Full support for VMware is part
of our future work.

We will provide a quantitative and qualitative comparison of EMOTIVE running over different hyper-
visors (e.g. XEN, KVM and Virtual Box) in the section 4.6.1 . You could find the best solution for
every type of needs: performance, green, agility, usability, etc.

26

VirtMonitor.java (API Libvirt) XenMonitor.java (API Xen)

Compute Method Summary Compute Method Summary
int availableMemory() int availableMemory()
void checkAll() void checkAll()
int resume(java.lang.String name) int resume(java.lang.String name)
int save(java.lang.String name) int save(java.lang.String name)
int restore(java.lang.String name) int restore(java.lang.String name)
int start(java.lang.String name) int start(java.lang.String name)
int shutdown(java.lang.String name) int shutdown(java.lang.String name)
void unpause(java.lang.String name) void unpause(java.lang.String name)
void pause(java.lang.String name) void pause(java.lang.String name)
int pauseDomain(java.lang.String name)
int defineXML(java.lang.String name)
int availableCPUs() int availableCPUs()
void dummy() void dummy()
float getCPUAmount(java.lang.String name) float getCPUAmount(java.lang.String name)
int getCPUCapacity(java.lang.String name) int getCPUCapacity(java.lang.String name)
int getCPUFreq() int getCPUFreq()
int getCPUPriority(java.lang.String name) int getCPUPriority(java.lang.String name)
int getCPUSpeed() int getCPUSpeed()
int setCPUCapacity(java.lang.String name, int capacity)
int setCPUPriority(int id, int priority)
int setCPUPriority(java.lang.String name, int priority)
int getDiskRD(int vm) int getDiskRD(int vm, int vbd)
int getDisksRD(int vm) int getDisksRD(int vm)
int getDisksRW(int vm) int getDisksRW(int vm)
int getDiskWR(int vm) int getDiskRW(int vm, int vbd)
int getDomainId(java.lang.String name) int getDomainId(java.lang.String name)
String getDomainName(int domain) String getDomainName(int domain)
List<String> getDomains() List<String> getDomains()
String getIP(int vm) String getIP(int vm)
int getMemory(int vm) int getMemory(int vm)
int getMemory(java.lang.String name) int getMemory(java.lang.String name)
float getMemoryAmount(int vm) float getMemoryAmount(int vm)
float getMemoryAmount(java.lang.String name) float getMemoryAmount(java.lang.String name)
int getMemoryMax(java.lang.String name) int getMemoryMax(java.lang.String name)
int getMemoryStaticMax(java.lang.String name)
int getMemoryStaticMin(java.lang.String name)
int getMemoryDynamicMax(java.lang.String name)
int getMemoryDynamicMin(java.lang.String name)
int getMemoryDomU(int vm)
int freeMemory()
int getNetRX(int vm) int getNetRX(int vm)
int getNetTX(int vm) int getNetTX(int vm)
int getNumCPU(java.lang.String name) int getNumCPU(java.lang.String name)
int getNumDisks(int vm) int getNumDisks(int vm)
int getNumDomains() int getNumDomains()
String getState(java.lang.String name) String getState(java.lang.String name)
boolean migrate(java.lang.String name, ja- boolean migrate(java.lang.String name, ja-
va.lang.String destHost) va.lang.String destHost)
int pinCPU(java.lang.String name, int vcpu, int[] cpumap) int pinCPU(int vm, int vcpu, int cpu)

27

int setCPUCapacity(String name, int nvcpus) int setCPUCapacity(int id, int capacity)
int setHostMemoryDynamicMax(String name, int mem)
void setMemory(int id, int mem) setMemory(int id, int mem)
void setMemory(java.lang.String name, int mem)
void setMemoryFixed(int id, int mem) void setMemoryFixed(int id, int mem)
void setMemoryFixed(java.lang.String name, int mem)
int setMemoryMax(java.lang.String name, int mem)
void setNumCPU(java.lang.String name, int cpu)
void showMemory(java.lang.String name) void showMemory(java.lang.String name)

VirtMonitor.java (API Libvirt) XenMonitor.java (API Xen)
Network Method Summary Network Method Summary
XML defineNetwork(String name, String uuid, String bridge,
String address, String netmask, String dev, String mode,
String start, String end)
int createNetwork(XML)
int deleteNetwork(String network)
int listNetworks()
List<String> showNetwork(String network)
Figure 14- Mapping methods between VirtMonitor and XenMonitor

4.4 EMOTIVE Networks

4.4.1 VLAN
The acronym VLAN expands to Virtual Local Area Network. A VLAN is a logical local area network
(LAN) that extends beyond a single traditional LAN to a group of LAN segments, given specific confi-
gurations. Because a VLAN is a logical entity, its creation and configuration is done completely in
software.

Libvirt is the technology used to create VLANs in EMOTIVE. Libvirt allows virtual machines creation
and easy management but also virtual networks creation and easy management.
Libvirt allows using console mode to create and manage the VLANs or using the Java/C API. In
EMOTIVE we use the Java API (30). First of all, to create VLAN with Libvirt, it is necessary to create
the XML manually with the VLAN description and later we need to pass this XML information in the
specific Libvirt call function. We show in a XML description as example:
<network>
<name>private</name>
<bridge name="virbr2" />
<ip address="192.168.152.1" netmask="255.255.255.0">
<dhcp>
<range start="192.168.152.2" end="192.168.152.254" />
</dhcp>
</ip>
<ip family="ipv6" address="2001:db8:ca2:3::1" prefix="64" />
</network>

So in EMOTIVE we have automatized the XML parsing and creation to have better VLAN
management and creation. The steps to create VLAN with EMOTIVE are the next:

28

1 - Necessary information to create Network.java (Java object) it is a EMOTIVE class
name, id, uuid, ip address, gateway, netmask, dev, mode (route, nat, isolated, private), bridge name,
ip_start, ip_end

2 - EMOTIVE Parser to create XML file with the last information (object Network)
String xml = parsing.CreateXML(Network.java) or
String xml = parsing.CreateXML(name, uuid, bridge, address, netmask, dev, mode, start, end);

3 - Libvirt function to create Network with the XML created file
conn.networkCreateXML(xml);

Libvirt has a set of elements control how a virtual network is provided connectivity to the physical
LAN, these are: bridge, domain, forward (nat, route, bridge, private, vepa, passthrough). Now
EMOTIVE only uses route, nat, isolated and private but in a future we could evolve to use more…

Figure15 - EMOTIVE Cloud example with Virtual-machines with VLANs

The API OCCI is used to offer these network services to the upper layers. API OCCI is used to have a
common language interface with other Clouds such as OpenNebula. OCCI is very useful to manage,
create and remove network resources (in the Cloud). Basically OCCI allows to create, list, show, and
delete networks, similarly to virtual machines (OCCI compute).

OCCI (occi-network):
NETWORK Example:
 ID, the uuid of the network <NETWORK>
 NAME, describing the network <ID>123</ID>
 ADDRESS, of the network <NAME>Blue Network</NAME>
 SIZE, of the network, defaults to C <ADDRESS>192.168.0.1</ADDRESS>
<SIZE>C</SIZE>
FUNCTIONS: </NETWORK>
create, list, show, delete

The EMOTIVE Scheduler can use these functionalities via an OCCI web services restful call to create,
list, show … a VLAN. Later on, Scheduler communicates with VtM and VtM uses the networks
methods located in VirtMonitor.java to invoke the Libvirt network facilities.

The next screen [Figure 16] shows the help menu of EMOTIVE console. You can see the new networks
functionalities to network management. Functions as ($net-* …) are used for VLAN management

29

and ($vpn …) for VPN management.

localhost:~$ help
Help
server: change server to connect
create: create a VM with generic name
name: create a VM with specific name
set: set a property of the domain
+ (vmid/memory/cpu/home/exten/ip/name/taskid)
+ (id)
new: create a new domain
run [CMD]: run a task in a VM
status [TASKID]: show status of a task
pause [VMID]: pause VM
unpause [VMID]: unpause VM
destroy [VMID]: destroy VM
cancel [TASKID]: cancel a task
show
domain: domain in the system
task: tasks in the system
vm: VMs in the system
vmall: all VMs in the system
nodes: nodes in the system
vmothers: extern VM in the system (no EMOTIVE VM)
net-list: show network
net-create: create a new network
net-occi: create a new network
+ (id/name/ip/size(A,B,C)
net-destroy: destroy network
+ (name)
net-show show network
net-set: set a property of the Network
+ (name/ip/netmask/mode/start/end)
+ (quick [name-bridge] + [gateway+ip_range])
net-edit: In construction
vpn: create a vpn
multi + server + localip (openvpn)
multi + addclient + server + localip (openvpn)
ptp + server + localip (openvpn)
pptp + server + localip + user + password + tunnelname (pptp)
exit: quit this interactive terminal

Figure 16 - EMOTIVE help menu

The next screen [Figure 17] shows a demonstration to create a VLAN with EMOTIVE.

localhost:~$ net-show
Network:
Name: vlan01
Ip: 132.168.163.1
Mode: private
Netmask: 255.255.255.0
Bridge: vlan01

Range
IP start: 132.168.163.2
IP end : 132.168.163.254

Quickmode Example: >>'net-set quick vlan00 192.168.1'
name=vlan00 bridge=vlan00 ip=192.168.1.1 ip range [start
192.168.1.2 , end 192.168.1.254

30

localhost:~$ net-create

localhost:~$ net-list
Networks:
1: vlan01
localhost:~$
Figure 17- demostration to create a VLAN

4.4.2 VPN
Virtual Lan Networks allows the creation of isolated networks. We also wanted to create secure
networks with Virtual Private Networks (VPN). So we developed virtual networks creation between
VLANs or in the same network.

Virtual Private Networking is a solution that supports remote access and private data communications
over public networks that are cheaper alternatives to leased lines. VPN clients communicate with VPN
servers utilizing a number of specialized protocols.

To do this function we need to create a Java function to create VPN and have easy VPN management.
We automatize all necessary to created it: using OpenVPN open-source tool (31) and PPTP open-
source tool (32) with bash commands in Java and EMOTIVE Java functions. So to create a VPN, we
developed and automated VPN configuration as a System Administrator's would do manually. So first
of all we create the system config file (/etc/openvpn/openvpn.cfg or /etc/pptpd.conf) in the 2 nodes,
later we add the certificates in the 2 nodes and finally we launch the OpenVPN or PPTPD daemon in
each node.

The protocol used is PPTP with PPTPD application and SSL with OpenVPN open source application.
We use these two protocols in EMOTIVE Cloud because they have different interesting features.

OpenVPN is very useful in:
- Stronger Encryption: Some customers consider the more encryption the better; it would also
be fair to say that it's possible with PPTP for someone to get your password while connecting.
- No drop of packets (when using TCP): If you lose connection, you won't be thrown back on
the internet. This maybe important for you.
- Allows you access to more servers: With our Open VPN accounts you have access to all of our
servers, PPTP and Open. At any time if you want a server that is offering only PPTP accounts,
you can simply request it via our customer area. This means that our Open VPN accounts are
also PPTP if you need them to be. Simply login to your customer area and choose the PPTP
server, and a login and pass will be sent to you.
- Port Modification Allowed: If for some reason the standard configuration of our Open VPN
accounts still does not let you connect, our expert team can provide a custom configuration that
will go over whatever port you may have available. Not sure what all that means? No problem
we can login and do everything for you remotely.

And PPTP VPN Advantages:
- Works on Mobile Devices: Iphone, Android, Windows Mobile are just a few of the devices
that work with PPTP. These are very easily setup, and just a Host Name, Login and Password
you will be connected.
31

The Green Evolution of EMOTIVE Cloud EMOTIVE Cloud: The BSC’s IaaS open-source solution for Cloud Computing

The Green Evolution of EMOTIVE Cloud EMOTIVE Cloud: The BSC’s IaaS open-source solution for Cloud Computing

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (16)

Destaque

Destaque (7)

Semelhante a The Green Evolution of EMOTIVE Cloud EMOTIVE Cloud: The BSC’s IaaS open-source solution for Cloud Computing

Semelhante a The Green Evolution of EMOTIVE Cloud EMOTIVE Cloud: The BSC’s IaaS open-source solution for Cloud Computing (20)

The Green Evolution of EMOTIVE Cloud EMOTIVE Cloud: The BSC’s IaaS open-source solution for Cloud Computing