CDA for Grid and Cloud

Double Auction-based Scheduling of
Scientific Applications in Distributed
Grid and Cloud Environments

Francesc Lordan Gomis
Metodologia de Recerca en la Informàtica
25/06/2012

Outline

Introduction
Auction Theory
Model and strategies
Results
Conclusions

Double Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 1

Introduction
Grid and Cloud Computing allow user to execute applications on a third-party
infrastructure by paying for the access to the remote resources


Introduction
Users are represented by Schedulers that picks which resource a job will be run

The providers are represented by Resource Managers that try to maximize the
incomes of the provider by optimizing the resource usage

Resource Resource
Manager Manager

Scheduler
Resource Resource
Manager Manager


Introduction
When many users try to obtain the same resources, they compete for them.

Since access to these resources is paid, it can be seen as a market governed by
the supply-demand law

Economy-based negotiation techniques are attractive from bussiness point of view

Scheduler Resource
Manager
Scheduler
Scheduler
Market Resource

?
Manager
Scheduler
Scheduler Resource
Manager
Scheduler
Scheduler Resource
Manager
Scheduler


Auction Theory
Auctions are an economic mechanism used for allocating a set of resources
among a group of bidders.

Models:

Single-sided auctions: the seller submits a resource and the bidders submit their
offers to buy (bid)

- English auction: buyers constantly increase their offers until the highest one wins

- Dutch auction: the seller decreases the price of the resource until a buyer decides
to pay the proposed price

- Sealed auctions: buyers give their bids on an envelope
First-price: the highest bid wins
Vickrey second-price: the second highest bid wins


Auction Theory
Auctions are an economic mechanism used for allocating a set of resources
among a group of bidders.

Models:

Double-sided auctions: all the participant submit their offers to sell (asks) and
buy (bids)

- Call Auctions: during a timeframe all the participants submit their offers and
finally they try to match

- Continuous Double Auction (CDA): offers are continously submitted until a
match between the lowest ask and the highest bid is achieved or the auction
times out.

Combinatorial auctions: a single bid for multiple resource (NP-complete)


The limitation on resources converts the schedulers into competitors with
their own negotiation strategy.

Any strategy is based on 4 factors
Resource selection
Resource valuation
Auction participation
Bidding



Resource Selection

Before bidding, the scheduler decides for which resources it will participate
in an auction.

When a task is sumitted the scheduler looks for a pending auction. If there is
no such auction the scheduler postpones the bidding until there is one.

The selection is done with a lottery where probabilities take into account:
Availability of the resource
Execution time
Budget cost


Resource Valuation

User Valuation
2 steps:
- Update the value of the resource depending on the final price of its
last auction and a smoothing factor α.
- Increases the value depending on the number of lost auctions and a
revaluation factor β.

Provider Valuation
Depending on the success ratio on the last auctions for the resource
the price is increased or decreased by a resource revaluation factor γ



Auction participation

Scheduler
participates when there is at least one task waiting for that resource.
Tasks are kept in a priority queue sorted by the bottom levels

Resource Manager
The time between two auctions of the same resource are created
depend on the result for the last auctions.

If it was won, it wait until the resource has been used and adds a
delay
If it was lost, the delay is incremented exponentially with a delaying
factor δ



Bidding
Zero Intelligence: participants submit their offers periodically

Scheduler
tries to reduce the value of the resource by decreasing the distance
between its valuation and the highest bid

Resource Manager
tries to increase the value of the resource by reducing the distance
between its valuation and the lowest ask


Modifications to the base strategy:

Self-limitation
Reduce the price by reducing the competence
limit the number of auctions where the scheduler participates in based on:
The average number of retries
A resource limitation factor ζ

Pricing Agressiveness
Change the revaluation factor of a single scheduler
Value of the resource are increased faster or slower


Results
Resource Valuation


Results
Self-limitation


Results
Pricing aggressiveness

17

Conclusions
Creation of an open market of resources implemented as a CDA instance

Identification the general behaviour patters

Evaluation of strategies:
Self-limitation:
- has marginal improvements on specific configuration
- NOT RECOMENDED
Concessive strategy:
- very good for the common-drift case
Aggressive-strategy:
- reduces execution time with low impact on the budget on the single-user
deviation case
- higher budget and similar execution time on the common-drift case
- RISKY


Questions


Resource Selection
availability

Expected execution time and cost at any resource

Expected execution time

Resource value

0 <= rsfmin <=rsfmax<=1 Resource selection ratio

Double Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment

Resource Valuation
Scheduler Smoothing factor

Last final price Last valuation
Linear

Exponential Resource revaluation factor

# lost auctions for the resource

Resource Manager
Average success ratio

Resource revaluation factor


Auction Participation

Delay factor
# lost auctions for the resource


Self-limitation

Resource limitation factor

Average retry number


Workflows


Pricing aggressiveness


CDA for Grid and Cloud

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (7)

Semelhante a CDA for Grid and Cloud

Semelhante a CDA for Grid and Cloud (20)

Último

Último (20)

CDA for Grid and Cloud