Market-Oriented Cloud Computing (as part of cloud symposium of ACM Compute 2009) Srikumar Venugopal Grid Computing and Distributed Systems (GRIDS) Laboratory Dept. of Computer Science and Software Engineering The University of Melbourne, Australia

1. Market-Oriented Cloud Computing
Srikumar Venugopal
Grid Computing and Distributed Systems (GRIDS) Laboratory
Dept. of Computer Science and Software Engineering
The University of Melbourne, Australia
E: srikumar@csse.unimelb.edu.au

2. Agenda
Motivation for Market-based Computing

A vision for Market-based Clouds

Current research in GRIDS Lab

Aneka: A resource provider

Brokering of resources and negotiation

Conclusion and Open Questions

2

3. Salient Features of Cloud Computing
Abstracted Infrastructure

Using Resources without reference to their location

Fully Virtualized

Servers are virtual instances

Dynamic

Can add, delete new instances dynamically

Pay by Consumption

No fixed long-term contracts

Configurable

Any application or OS can be provided

Forrester Research, “Is Cloud Computing Ready For The Enterprise?”,
3 March 2008

4. Yet..
Cloud SLAs are still in their infancy

Limited options for higher Quality of Service

Flat Pricing Model

Amazon

Cloud provisioning is not a core product

Amazon is an e-commerce company

Google is a search company

Need innovative business models

A larger marketplace

4

5. The Gridbus Project @ Melbourne:
Enable Leasing of ICT Services on Demand
WWG
Gridbus
Pushes Grid computing into
mainstream computing
5

6. http://www.gridbus.org
The Gridbus Project @ GRIDS Lab, The University of Melbourne:
The Gridbus Project @ GRIDS Lab, The University of Melbourne:
Toolkit for Creating and Deploying e-* Applications on Utility Grids
Toolkit for Creating and Deploying e-* Applications on Utility Grids
• Gridbus is a “open source” Grid R&D Distributed Data
project with focus on Grid Economy, Utility
Grids and Service Oriented Computing.
• Gridbus Middleware components include:
– Aneka: .NET-based Enterprise Grid
– Grid Market Directory and Web Services
Gridbus
– Grid Bank: Accounting and Transaction
Management
– Visual Tools for Creation of Distributed
Applications
– Grid Service Broker and Scheduling
– Workflow Management Engine
– Libra: SLA-based Resource Allocation
– GridSim Toolkit
6

7. 7

8. Participants, Goals, Requirements
Consumers: - minimize expenses, meet QoS

How do I express QoS requirements ?

How do I trade between timeframe & cost ?

How do I discover services and map jobs to meet my QoS needs?

How do I manage Grid dynamics and get my work done?

…

Providers:– maximise ROI and profit

How do I decide service pricing models ?

How do I specify them ?

How do I translate them into resource allocations ?

How do I enforce them ?

How do I advertise & attract consumers ?

How do I do accounting and handle payments?

…

They need mechanisms, tools and technologies that help them in value

expression, value translation, and value enforcement.
Service Level Agreements (SLAs)

8

9. Market Mechanisms for Clouds: A
vision
Broker1
Cloud
Consumer
Request Negotiate/Bid
Compute Storage
Resources
Cloud Cloud
Directory
....
Bank
Auctioneer
MARKET
Enterprise
Compute
Resource BrokerN
Cloud
Manager
(Proxy)
Storage
Cloud
9

10. Aneka: a resource provider for parallel
and distributed applications
Applications
Container
Thread Task Dataflow MPI Map Other SLA
Model Model Model Model Reduce Models Negotiation
Persistence
Allocation Manager
Message Handler / Dispatcher
Security
Communication Layer
10

11. Advance Reservations
Commitment of a guaranteed share of a

resource ahead of usage time
Resources : Nodes, Bandwidth, Storage

Advantages:

Lowers risk for user

Easier capacity planning for provider

Assured income

Applications : workflow, multimedia applications, etc.

Are a form of SLA

11

12. Aneka’s SLA-View for Resource Allocation
User/Broker
Enterprise Grid
Negotiation Protocol Engine
Master Node
Membership Scheduling
Reservation Service
Service Service
Node
Pricing
Membership Reservation Task
Selection
Policy
Store Store Store
Policy
Execution Execution
Execution Node
Node Node
Execution
Allocation Service
Service
Time Slot
Reservation Task
Selection
Store Store
Policy
Ack: C.S. Yeo
12

13. Pricing of Reservations
Dynamic pricing based on utilization level

p  ax  by
Where p is the unit price,

x is the static component (base price), and

y = load factor * z, is the dynamic component

a and b are the relative weights

b can be set higher when resource availability is low and vice

versa
Serves as a method of admission control

takes advantage of market conditions

C.S.Yeo, S. Venugopal, X. Chu, and R. Buyya, Autonomic Metered Pricing for a Utility
Service, Technical Report, GRIDS-TR-2008-16, GRIDS Laboratory.
13

14. Revenue vs Strategy
14

15. Cloud Provider Architecture
Users/
Brokers
Service R equest Examiner and
A dmission Control
- Customer-driven Service Management
- Computational R isk Management
- Autonomic Resource Management
SLA
Resource
Allocator Pricing Accounting
VM Service R equest
Monitor Monitor
Dispatcher
Virtual
Machines
(VMs)
Physical
Machines
15

16. Gridbus Broker: Abstracting Resource
Access
Home Node/Portal Credential Repository
MyProxy
Portlets
batch()
Gridbus -PBS
-Condor
Broker -SGE
fork()
Data Catalog
Globus
Aneka
Data Store Unicore SSH
Job manager
Access Technology fork()
Gateway
batch()
fork() batch() SRB -PBS
Grid FTP
-PBS Gridbus
Gridbus -Condor
-Condor agent
agent -SGE
-SGE
-XGrid
16

17. Broker-Provider Negotiation
Provider
Broker
Negotiation
Negotiation
Module
Module
Advance
Reservation
Resource Allocation
Scheduler Manager
Job Submission
and Monitoring
Broker acts as a user agent

Broker translates user requirements to resource requirements

However, the negotiation process is invisible to the end user.

17

18. The Negotiation Protocol
18

19. Effect of deadline urgency
S. Venugopal, X. Chu, and R. Buyya, “A Negotiation Mechanism for Advance Resource
Reservation using the Alternate Offers Protocol”, IWQoS 2008.
19

20. MetaCDN: Brokering Cloud Storage
Providers
Dr. James Broberg, University of Melbourne, http://www.metacdn.org
20

21. Open Questions
How to commoditise cloud services ?

What would be the structure of the Cloud

services market ?
What are the accounting and payment

mechanisms available ?
How to monitor and enforce the SLAs arrived at

by negotiation ?
Who arbitrates the process ?

21

22. Thank You
Questions?
http://www.gridbus.org