Operating Systems - Queuing Systems

Operating Systems
CMPSCI 377
Queuing Systems
Emery Berger
University of Massachusetts Amherst

UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

Queuing Systems & Servers
Queuing systems


High-level model of concurrent applications


Flux


Language for building servers


UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 2

Queuing Networks
Model of tasks or services


Node includes queue (line) & server



Queuing Networks





Queuing Networks




arrival rate
()


Queuing Networks




waiting time


Queuing Networks




service time


Queuing Networks





Stable Systems
Stable queuing system:


arrival rate = departure rate


Stable Systems




Stable Systems



What happens if  > departure rate?



Stable Systems






Networks of Queues
Can build system from connected servers


Latency = time for one thing to get through


Throughput = service rate


5/sec 5/sec 5/sec

Throughput?



Networks of Queues
Can build system with numerous


connected servers
Latency = time for one thing to get through


Throughput = service rate


5/sec 1/sec 5/sec

Throughput?


Lowest throughput = bottleneck



Little’s Law
Little’s Law – applies to any “blackbox”


server
Queue length (N) =


arrival rate () * average waiting time (T)
N=T


N

 T


Applications of Little’s Law
Compute waiting time to get into


restaurant, bar, etc.
If N = 20 people in front of you,


 = departure rate = 1 / 5 min.,
how long will you wait in line?

N=T
N

 T


Required service time?


Arrival rate = one job @ 500 ms


Average queue length = 10


T=?


What’s the average latency?


N=T
N

 T


Required service time?


Arrival rate = one job @ 500 ms


Average queue length = 5


T=?


What’s the average latency?


N=T
N

 T


Motivating Example: Image Server
image
server
Client


Requests image @ desired


quality, size not found

Server


Images: RAW

http://server/Easter-bunny/
200x100/75
Compresses to JPG


Caches requests


Sends to client


client

Problem: Concurrency
image
Could write sequential code server

but…
More clients (latency)


Bigger server


Multicores, multiprocessors


One approach: threads


Limit reuse,


risk deadlock,
burden programmer
Complicate debugging


Mixes program logic &


concurrency control clients


The Flux Programming Language
High-performance & deadlock-free
concurrent programming w/ sequential components
Flux = Components + Flow + Atomicity


Components unmodified C, C++ (or Java)


Flow implicitly || path thru components


Atomicity high-level mutual exclusion


Compiler generates:


Deadlock-free, runtime-independent server


Threads, thread pools, events, …


Path profiling


Discrete event simulator



Flux Outline

Intro to Flux: building a server


Components


Flows


Atomicity


Performance results


Server performance


Performance prediction (QNMs)



Flux Server “Main”

Source nodes originate flows


Conceptually in separate thread


Executes inside implicit infinite loop


Here: initiates flow for each image request


source Listen  Image;

Listen
image server
ReadRequest Compress Write Complete


Flux Image Server
Basic image server requires:

HTTP parsing (http)


 Socket handling (socket)

 JPEG compression (libjpeg)

 All UNIX-style C libraries

Abstract node = flow across nodes

Concrete or abstract


Image = ReadRequest  Compress  Write  Complete;

image server

http libjpeg socket http


Control Flow
Direct flow via user-supplied predicate types


Type test applied to output


Note: no variables – dispatch on output “type”


Here: cache frequently requested images


typedef hit TestInCache;
Handler:[_,_,hit] = ;
Handler:[_,_,_] = ReadFromDisk  Compress  StoreInCache;

hit handler
ReadRequest CheckCache Write Complete
Listen

handler
StoreInCache
ReadInFromDisk Compress


Supporting Concurrency
Many clients = concurrent flows


Must keep cache consistent


Atomicity constraints


Same name = mutual exclusion


Apply to nodes or whole flow (abstract node)


atomic CheckCache {};
{, };
atomic Complete
atomic StoreInCache {};

CheckCache hit
ReadRequest Write Complete
CheckCache hit hit
ReadRequest Write Complete
CheckCache hit Writehandler
ReadRequest Complete
Listen ReadInFromDisk Compress StoreInCache
ReadRequest CheckCache StoreInCache Write Complete
ReadInFromDisk Compress StoreInCache

handler
StoreInCache


More Atomicity

Reader / writer constraints


Multiple readers or single writer (default)


atomic ReadList: {listAccess?};
atomic AddToList: {listAccess!};

Per-session constraints


User-supplied function ≈ hash on source


Added to flow ≈ chooses from array of locks


atomic AddHasChunk: {chunks(session)};

Preventing Deadlock

Naïve execution can deadlock


atomic A: {z,y}; atomic A: {y,z};
atomic B: {y,z}; atomic B: {y,z};

Establish canonical lock order


Partial order


Alphabetic by name



Preventing Deadlock, II
Harder with abstract nodes


A = B;
C = D;
A:{z}
A
A{z}
atomic A{z}; C{y}
C
B
atomic B{y};
atomic C{y,z};

Solution: Elevate constraints; fixed point


A = B;
C = D; A{y,z}
C
atomic A{y,z};
B
atomic B{y};
atomic C{y,z};


Handling Errors
What if image requested doesn’t exist?


Error = negative return value from component


Remember – nodes oblivious to Flux


Solution: error handlers


Go to alternate paths on error


Possible extension – can match on error paths


handle error ReadInFromDisk  FourOhFour;
hit
handler
ReadRequest CheckCache Write Complete

Listen handler
StoreInCache

FourOhFour


Almost Complete Flux Image Server
source Listen  Image;
Image =
ReadRequest  CheckCache  Handler  Write  Complete;
Handler[_,_,hit] = ;
Handler[_,_,_] = ReadFromDisk  Compress  StoreInCache;
atomic CheckCache: {cacheLock};
atomic StoreInCache: {cacheLock};
atomic Complete: {cacheLock};
handle error ReadInFromDisk  FourOhFour;
Concise, readable expression of server logic


No threads, etc.: simplifies programming, debugging


image hit
handler
server ReadRequest CheckCache Write Complete
Listen
handler
StoreInCache


Flux Outline

Intro to Flux: building a server


Components, flow


Atomicity, deadlock avoidance


Performance results


Server performance


Performance prediction


Future work



Flux Results

Four servers:

Image server (+ libjpeg) [23 lines of Flux]


Multi-player online game [54]


BitTorrent (2 undergrads: 1 week!) [84]


Web server (+ PHP) [36]


Evaluation

Benchmark: variant of SPECweb99


Three different runtimes here


Thread: one per connection


Thread pool: fixed max # threads


Event-driven: helper threads for blocking calls


Compared to Capriccio [SOSP03], SEDA [SOSP01]



Web Server


Performance Prediction

observed
parameters


The End


Operating Systems - Queuing Systems

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (20)

Semelhante a Operating Systems - Queuing Systems

Semelhante a Operating Systems - Queuing Systems (20)

Mais de Emery Berger

Mais de Emery Berger (20)

Último

Último (20)

Operating Systems - Queuing Systems