This document describes the design and experimental validation of a new bandwidth sharing scheme called UGUALE (User GUaranteed ALlocation Engine). UGUALE aims to guarantee fairness between users transmitting traffic over the same network link, ensuring each user obtains a guaranteed minimum rate plus an equal share of unused bandwidth, regardless of the number of TCP connections or round-trip times. The scheme was tested on a real network testbed and shown to achieve fair rates between users, unlike the standard TCP which favors users with more connections or lower RTTs. UGUALE implements meters and queues in switches to classify user traffic based on measured rates and prioritize lower-rate users, influencing end-to-end congestion control to converge

1. Dipartimento di Elettronica,
Informazione e Bioingegneria
Dipartimento di Fisica
Dipartimento di Energia
Design and experimental validation
of a new bandwidth sharing scheme
based on dynamic queue assignment
Supervisor: Prof. Antonio Capone
Supervisor: Prof. Brunilde Sans`o
Advisor: Eng. Carmelo Cascone
Luca Bianchi
Student ID 817219
A.Y. 2015-2016

2. Problem statement Approach Questions Testbed Results Conclusions
Introduction
In the Internet, fair bandwidth sharing is achieved mostly
thanks to end-to-end congestion control mechanisms
Most of the Internet traffic is TCP (Transmission Control
Protocol) compatible
For each connection, TCP estimates the network congestion
status and adjusts the CWND (Congestion Window) in order
not to overload the network
The CWND is the number of unacknowledged packets that
can be in transit end-to-end
2 / 29

3. Problem statement Approach Questions Testbed Results Conclusions
TCP congestion control
After the slow start region, TCP applies the Addictive
Increase / Multiplicative Decrease (AIMD) algorithm
3 / 29

4. Problem statement Approach Questions Testbed Results Conclusions
TCP per-flow fairness
When TCP flows transmit over a link, each flow obtains the
same bandwidth share ...
0 5 10 15 20 25 30
0.0
20.0M
40.0M
60.0M
80.0M
100.0M
bit/s
Per-user TCP/UDP raw rate
6 users
SUM
192.168.200.130
192.168.200.131
192.168.200.121
192.168.200.120
192.168.200.110
192.168.200.111
0 5 10 15 20 25 30
0.0
5.0M
10.0M
15.0M
20.0M
bit/s
Per-user smoothed rate (4s window)
time [s]
time [s]
4 / 29

5. Problem statement Approach Questions Testbed Results Conclusions
Per-user unfairness
... but users can instantiate an arbitrary number of parallel
TCP connections ...
0 5 10 15 20 25 30
0.0
20.0M
40.0M
60.0M
80.0M
100.0M
bit/s
Per-user TCP/UDP raw rate
6 users
SUM
192.168.200.130
192.168.200.131
192.168.200.121
192.168.200.120
192.168.200.110
192.168.200.111
0 5 10 15 20 25 30
0.0
5.0M
10.0M
15.0M
20.0M
25.0M
30.0M
35.0M
40.0M
bit/s
Per-user smoothed rate (4s window)
time [s]
time [s]
Users obtain rates directly proportional to their number of TCP connections
5 / 29

6. Problem statement Approach Questions Testbed Results Conclusions
RTT dependence
... and flows can have different Round Trip Times
0 5 10 15 20 25 30
0.0
20.0M
40.0M
60.0M
80.0M
100.0M
bit/s
Per-user TCP/UDP raw rate
6 users
SUM
192.168.200.130
192.168.200.131
192.168.200.121
192.168.200.120
192.168.200.110
192.168.200.111
0 5 10 15 20 25 30
0.0
10.0M
20.0M
30.0M
40.0M
50.0M
60.0M
bit/s
Per-user smoothed rate (4s window)
time [s]
time [s]
Flows obtain rates inversely proportional to their RTT
6 / 29

7. Problem statement Approach Questions Testbed Results Conclusions
Objective
A user is defined as an arbitrary aggregate of connection layer
flows
The objective of the work is to design and experimentally
validate a new bandwidth sharing scheme able to guarantee
fairness between users transmitting traffic over the same link
Every user u should obtain a guaranteed rate gu plus an equal
share of the unused bandwidth, independently from the
number of TCP connections and from the RTTs
Guaranteed rates are arbitrary values assigned such that
u∈U gu ≤ c, where U is the set of users routed on the same
egress interface of a switch whose link has capacity c
7 / 29

8. Problem statement Approach Questions Testbed Results Conclusions
Main idea
The main idea is to prioritize low-rate users w.r.t. high-rate
users. The faster a user transmits, the less priority its packets
obtain
The new approach, called UGUALE (User GUaranteed
ALlocation Engine), influences the end-to-end congestion
control mechanisms of traffic sources so that users’ served
traffic converge to a fair share of the link capacity
8 / 29

9. Problem statement Approach Questions Testbed Results Conclusions
UGUALE’s switch model
UGUALE uses three switch elements:
an aggregation table
a meter for each user
a Strict Priority (SP) scheduler with a few queues (w.r.t. the
number of served users)
Input Packet
Flow
aggregation
Metering Classification Scheduling Output
Traffic
Figure: UGUALE’s switch model
9 / 29

10. Problem statement Approach Questions Testbed Results Conclusions
Aggregation
A user is defined by an arbitrary n-tuple of header fields (e.g.
source and destination IP address)
Input packets are matched against an aggregation table
Every row is composed of:
a match rule (that defines a certain user)
an action to be applied to matching packets
Packets belonging to the same user are sent to a
user-dedicated meter
10 / 29

11. Problem statement Approach Questions Testbed Results Conclusions
Metering
A meter is a switch element that takes actions based on the
measured rate of packets passing through it
Each meter measures the offered rate of a certain user
Based on measured rates, subsequent packets of the same
user can be enqueued in different queues (classification)
User1
packets
Rate < Guaranteed
High Priority
Queue
User2
packets
Rate > Guaranteed
Low Priority
Queue
Meters
11 / 29

12. Problem statement Approach Questions Testbed Results Conclusions
Scheduling
Using a Strict Priority (SP) scheduler, the low priority queue
is served only when the high priority queue is empty
When a queue becomes full, tail-drop occurs
High Priority Queue
Low Priority Queue
Output link
SP
12 / 29

13. Problem statement Approach Questions Testbed Results Conclusions
UGUALE with more queues
In order to control rates with a finer granularity, the scheme
can be extended by using more queues
Schedulers generally have 8 queues and a meter can have
many thresholds.
Bands are interval of rates between two consecutive meter’s
thresholds. Packets are sent to different queues based on the
band in which the measured rate falls.
Band 3,1
Band 2,1
Band 2,2
Band 2,3
Band 2,4
Strict Priority
scheduler
Priority 4 queue
Priority 3 queue
Prority 2 queue
Priority 1 queue
User 1 packet
User 2 packet
User 3 packet
Band 3,2
Band 3,3
Band 3,4
Band 1,1
Band 1,2
Band 1,3
Band 1,4
Output
Traffic
t1,1
t1,2
t1,3
t1,4
t2,1
t2,2
t2,3
t2,4
t3,1
t3,2
t3,3
t3,4
a2
a3
a1
c
13 / 29

14. Problem statement Approach Questions Testbed Results Conclusions
Thresholds setting
For each user u, the first threshold corresponds to the user’s
guaranteed rate gu
Other thresholds are assigned such that the remaining
capacity is uniformly covered
In order to fairly share the excess bandwidth, band’s width
∆ref should be the same for each user
Band 1
Band 2
Band 3
Band 4
Band 5
Band 6
Band 7
Band 8
Band b 1,1
c
∆ref =c/|Q|
Band b 1,3
Band b 1,4
Band b 1,5
Band b 1,6
Band b 1,7
Band b 1,8
Reference
distribution
User u=1
g1 Band b 2,1
Band b 2,2
Band b 2,3
Band b 2,4
Band b 2,5
Band b 2,6
Band b 2,7
Band b 2,8
User u=2
g2
Band b 3,2
Band b 3,3
Band b 3,4
Band b 3,5
Band b 3,6
Band b 3,7
User u=3
g3Band b 1,2 Band b 3,1
∆ref
14 / 29

15. Problem statement Approach Questions Testbed Results Conclusions
Questions
How well does the system perform? Is it stable? Can it scale?
Since there are no theoretical models involving dynamic queue
assignment and end-to-end congestion control mechanisms,
we decided to first assess the approach using experiments
15 / 29

16. Problem statement Approach Questions Testbed Results Conclusions
Testbed
LAN
Redfox1
[User]
Redfox2
[User]
Redfox3
[User]
Redfox0
[Switch]
Redfox4
[Server]
The scheme was tested on a real
network composed of:
1 PC with an extra 4-NIC
(software switch)
3 PCs sending data
(clients)
1 PC receiving data
(server)
16 / 29

17. Problem statement Approach Questions Testbed Results Conclusions
Testbed setup
iPerf is used to transfer data
and measure rates
Each client PC can emulate
many users
Each user can have an
arbitrary number of TCP
connections and RTT
Users compete only for the
bottleneck link’s capacity, so
UGUALE is executed on this
port of the switch
17 / 29

18. Problem statement Approach Questions Testbed Results Conclusions
Implementation
UGUALE is implemented
using OpenFlow’s
abstractions
The used software switch
is Open vSwitch
Two different meter
implementations were tried:
Estimator meters
Token bucket meters
18 / 29

19. Problem statement Approach Questions Testbed Results Conclusions
Users with a different number of TCP connections
Fairness not enforced by the network
2 5 8
N. of TCP connections
0.10
0.05
0.00
0.05
0.10
DistancefromOFRnormalized
w.r.t.bottleneckcapacity
(c)
Mean/std
40.0
RTT [ms]
0.10
0.05
0.00
0.05
0.10 (d)
Mean/std
1 11 21
User ID
0.10
0.05
0.00
0.05
0.10
DistancefromOFRnormalized
w.r.t.bottleneckcapacity
(b)
Mean/std
-0.15 0 0.15
Distance from OFR normalized
w.r.t. bottleneck capacity
0
5
10
15
Discretedistribution
Mean: 0.0000299
Var: 0.0003949
Std: 0.0198714
MSE: 0.0003949
(a)
Mean USERS
User 1: 8c, 40ms
User 2: 4c, 40ms
User 3: 7c, 40ms
User 4: 6c, 40ms
User 5: 3c, 40ms
User 6: 6c, 40ms
User 7: 2c, 40ms
User 8: 2c, 40ms
User 9: 5c, 40ms
User 10: 5c, 40ms
User 11: 8c, 40ms
User 12: 5c, 40ms
User 13: 8c, 40ms
User 14: 4c, 40ms
User 15: 7c, 40ms
User 16: 4c, 40ms
User 17: 3c, 40ms
User 18: 7c, 40ms
User 19: 6c, 40ms
User 20: 3c, 40ms
User 21: 2c, 40ms
CONFIGURATION
Switch type: standalone
Switch queue lenght: 210 pkt
STATISTICS
Jain's Index mean: 0.8395898
Jain's Index var: 0.0004208
Throughput mean: 0.9870610
Throughput var: 0.0000002
Goodput mean: 0.9413611
Goodput var: 0.0003277
Ratio G/T mean: 0.9536995
Ratio G/T var: 0.0003347
19 / 29

20. Problem statement Approach Questions Testbed Results Conclusions
Users with a different number of TCP connections
Fairness enforced by UGUALE
2 5 8
N. of TCP connections
0.10
0.05
0.00
0.05
0.10
DistancefromOFRnormalized
w.r.t.bottleneckcapacity
(c)
Mean/std
40.0
RTT [ms]
0.10
0.05
0.00
0.05
0.10 (d)
Mean/std
1 11 21
User ID
0.10
0.05
0.00
0.05
0.10
DistancefromOFRnormalized
w.r.t.bottleneckcapacity
(b)
Mean/std
-0.15 0 0.15
Distance from OFR normalized
w.r.t. bottleneck capacity
0
10
20
30
40
50
60
70
Discretedistribution
Mean: -0.0000775
Var: 0.0000311
Std: 0.0055744
MSE: 0.0000311
(a)
Mean USERS
User 1: 7c, 40ms
User 2: 4c, 40ms
User 3: 2c, 40ms
User 4: 2c, 40ms
User 5: 5c, 40ms
User 6: 5c, 40ms
User 7: 6c, 40ms
User 8: 6c, 40ms
User 9: 8c, 40ms
User 10: 7c, 40ms
User 11: 3c, 40ms
User 12: 7c, 40ms
User 13: 4c, 40ms
User 14: 4c, 40ms
User 15: 6c, 40ms
User 16: 3c, 40ms
User 17: 2c, 40ms
User 18: 8c, 40ms
User 19: 3c, 40ms
User 20: 8c, 40ms
User 21: 5c, 40ms
CONFIGURATION
Switch type: uguale
Marking type: iptables
Switch queue lenght: 210 pkt
Unused capacity: 0.0
N. of bands: 8
N. of guard bands: -1
RTT compensation: False
STATISTICS
Jain's Index mean: 0.9902083
Jain's Index var: 0.0000112
Throughput mean: 0.9870587
Throughput var: 0.0000002
Goodput mean: 0.9393858
Goodput var: 0.0001669
Ratio G/T mean: 0.9517010
Ratio G/T var: 0.0001707
20 / 29

21. Problem statement Approach Questions Testbed Results Conclusions
Users with different RTTs
Fairness not enforced by the network
3
N. of TCP connections
0.10
0.05
0.00
0.05
0.10
DistancefromOFRnormalized
w.r.t.bottleneckcapacity
(c)
Mean/std
20.0 120.0 220.0
RTT [ms]
0.10
0.05
0.00
0.05
0.10 (d)
Mean/std
1 11 21
User ID
0.10
0.05
0.00
0.05
0.10
DistancefromOFRnormalized
w.r.t.bottleneckcapacity
(b)
Mean/std
-0.15 0 0.15
Distance from OFR normalized
w.r.t. bottleneck capacity
0
5
10
15
20
25
30
35
Discretedistribution
Mean: 0.0000139
Var: 0.0005084
Std: 0.0225478
MSE: 0.0005084
(a)
Mean USERS
User 1: 3c, 220ms
User 2: 3c, 60ms
User 3: 3c, 110ms
User 4: 3c, 160ms
User 5: 3c, 170ms
User 6: 3c, 150ms
User 7: 3c, 100ms
User 8: 3c, 40ms
User 9: 3c, 200ms
User 10: 3c, 70ms
User 11: 3c, 180ms
User 12: 3c, 20ms
User 13: 3c, 130ms
User 14: 3c, 210ms
User 15: 3c, 30ms
User 16: 3c, 120ms
User 17: 3c, 140ms
User 18: 3c, 90ms
User 19: 3c, 80ms
User 20: 3c, 50ms
User 21: 3c, 190ms
CONFIGURATION
Switch type: standalone
Switch queue lenght: 126 pkt
STATISTICS
Jain's Index mean: 0.8002089
Jain's Index var: 0.0005750
Throughput mean: 0.9870048
Throughput var: 0.0000000
Goodput mean: 0.9414874
Goodput var: 0.0000238
Ratio G/T mean: 0.9538836
Ratio G/T var: 0.0000244
21 / 29

22. Problem statement Approach Questions Testbed Results Conclusions
Users with different RTTs
Fairness enforced by UGUALE
3
N. of TCP connections
0.10
0.05
0.00
0.05
0.10
DistancefromOFRnormalized
w.r.t.bottleneckcapacity
(c)
Mean/std
20.0 120.0 220.0
RTT [ms]
0.10
0.05
0.00
0.05
0.10 (d)
Mean/std
1 11 21
User ID
0.10
0.05
0.00
0.05
0.10
DistancefromOFRnormalized
w.r.t.bottleneckcapacity
(b)
Mean/std
-0.15 0 0.15
Distance from OFR normalized
w.r.t. bottleneck capacity
0
20
40
60
80
100
Discretedistribution
Mean: 0.0000074
Var: 0.0000324
Std: 0.0056928
MSE: 0.0000324
(a)
Mean USERS
User 1: 3c, 180ms
User 2: 3c, 60ms
User 3: 3c, 170ms
User 4: 3c, 40ms
User 5: 3c, 200ms
User 6: 3c, 160ms
User 7: 3c, 220ms
User 8: 3c, 30ms
User 9: 3c, 50ms
User 10: 3c, 150ms
User 11: 3c, 140ms
User 12: 3c, 70ms
User 13: 3c, 190ms
User 14: 3c, 100ms
User 15: 3c, 90ms
User 16: 3c, 120ms
User 17: 3c, 20ms
User 18: 3c, 110ms
User 19: 3c, 80ms
User 20: 3c, 210ms
User 21: 3c, 130ms
CONFIGURATION
Switch type: uguale
Marking type: iptables
Switch queue lenght: 126 pkt
Unused capacity: 0.0
N. of bands: 2
N. of guard bands: -1
RTT compensation: False
STATISTICS
Jain's Index mean: 0.9868650
Jain's Index var: 0.0000236
Throughput mean: 0.9870126
Throughput var: 0.0000000
Goodput mean: 0.9411788
Goodput var: 0.0001692
Ratio G/T mean: 0.9535632
Ratio G/T var: 0.0001732
22 / 29

23. Problem statement Approach Questions Testbed Results Conclusions
Experimental results
Users’ rates have small oscillations around optimal fair rates
and the efficiency is maximized
Guaranteed rates are assured to every user
The fairness in the assignment of the spare capacity depends
on the number and width of used bands
If thresholds are close to the used rate interval, performance
improves
Bands cannot be too small, otherwise the system becomes
unstable
A single threshold in the optimal fair rate is the best solution
23 / 29

24. Problem statement Approach Questions Testbed Results Conclusions
Experimental results - Meters
The two types of meters obtain different statistics:
”Estimator” meters allow the maximum goodput
”Token buckets” meters reduce remarkably the variance of
rates
2 5 8
N. of TCP connections
0.10
0.05
Distance
w.r.t.
40.0
RTT [ms]
0.10
0.05 RTT compensation: True
STATISTICS
Jain's Index mean: 0.9992814
Jain's Index var: 0.0000002
Throughput mean: 0.9870042
Throughput var: 0.0000000
Goodput mean: 0.8932797
Goodput var: 0.0000206
Ratio G/T mean: 0.9050411
Ratio G/T var: 0.0000212
Figure 5.14: The token bucket implementation of meters improves remarkably the variance but
causes a reduction of goodput, so all rates are a bit lower than expected.
highlighted by the results presented in table 5.1. The table shows the average
statistics obtained in a set of tests executed with both meter types.
Meter JFI mean Goodput mean Distr. mean Distr. var
Token B. 0.985 0.908 −0.00153 3.27 · 10−5
Estimator 0.969 0.937 −0.00015 8.42 · 10−5
Table 5.1: Average statistics of different meter implementations.
89
Table: Average statistics obtained in a set of tests
executed with both meter types
24 / 29

25. Problem statement Approach Questions Testbed Results Conclusions
Long switch queues
Fairness enforced by UGUALE
3
N. of TCP connections
0.10
0.05
0.00
0.05
0.10
DistancefromOFRnormalized
w.r.t.bottleneckcapacity
(c)
Mean/std
20.0 120.0 220.0
RTT [ms]
0.10
0.05
0.00
0.05
0.10 (d)
Mean/std
1 11 21
User ID
0.10
0.05
0.00
0.05
0.10
DistancefromOFRnormalized
w.r.t.bottleneckcapacity
(b)
Mean/std
-0.15 0 0.15
Distance from OFR normalized
w.r.t. bottleneck capacity
0
5
10
15
20
25
30
Discretedistribution
Mean: -0.0004652
Var: 0.0002766
Std: 0.0166302
MSE: 0.0002768
(a)
Mean USERS
User 1: 3c, 190ms
User 2: 3c, 220ms
User 3: 3c, 160ms
User 4: 3c, 150ms
User 5: 3c, 130ms
User 6: 3c, 70ms
User 7: 3c, 180ms
User 8: 3c, 90ms
User 9: 3c, 100ms
User 10: 3c, 200ms
User 11: 3c, 40ms
User 12: 3c, 50ms
User 13: 3c, 170ms
User 14: 3c, 120ms
User 15: 3c, 210ms
User 16: 3c, 20ms
User 17: 3c, 140ms
User 18: 3c, 80ms
User 19: 3c, 30ms
User 20: 3c, 60ms
User 21: 3c, 110ms
CONFIGURATION
Switch type: uguale
Marking type: iptables
Switch queue lenght: 1000 pkt
Unused capacity: 0.0
N. of bands: 8
N. of guard bands: -1
RTT compensation: False
h: 1
STATISTICS
Jain's Index mean: 0.9249796
Jain's Index var: 0.0010794
Throughput mean: 0.9869978
Throughput var: 0.0000000
Goodput mean: 0.9314254
Goodput var: 0.0035937
Ratio G/T mean: 0.9436952
Ratio G/T var: 0.0036887
25 / 29

26. Problem statement Approach Questions Testbed Results Conclusions
Short switch queues
Fairness enforced by UGUALE
3
N. of TCP connections
0.10
0.05
0.00
0.05
0.10
DistancefromOFRnormalized
w.r.t.bottleneckcapacity
(c)
Mean/std
20.0 120.0 220.0
RTT [ms]
0.10
0.05
0.00
0.05
0.10 (d)
Mean/std
1 11 21
User ID
0.10
0.05
0.00
0.05
0.10
DistancefromOFRnormalized
w.r.t.bottleneckcapacity
(b)
Mean/std
-0.15 0 0.15
Distance from OFR normalized
w.r.t. bottleneck capacity
0
20
40
60
80
100
Discretedistribution
Mean: -0.0000068
Var: 0.0000412
Std: 0.0064221
MSE: 0.0000412
(a)
Mean USERS
User 1: 3c, 210ms
User 2: 3c, 160ms
User 3: 3c, 30ms
User 4: 3c, 150ms
User 5: 3c, 100ms
User 6: 3c, 140ms
User 7: 3c, 200ms
User 8: 3c, 110ms
User 9: 3c, 130ms
User 10: 3c, 170ms
User 11: 3c, 190ms
User 12: 3c, 180ms
User 13: 3c, 80ms
User 14: 3c, 50ms
User 15: 3c, 70ms
User 16: 3c, 40ms
User 17: 3c, 90ms
User 18: 3c, 20ms
User 19: 3c, 220ms
User 20: 3c, 120ms
User 21: 3c, 60ms
CONFIGURATION
Switch type: uguale
Marking type: iptables
Switch queue lenght: 126 pkt
Unused capacity: 0.0
N. of bands: 8
N. of guard bands: -1
RTT compensation: False
h: 1
STATISTICS
Jain's Index mean: 0.9832987
Jain's Index var: 0.0000392
Throughput mean: 0.9868859
Throughput var: 0.0000005
Goodput mean: 0.9409834
Goodput var: 0.0001106
Ratio G/T mean: 0.9534885
Ratio G/T var: 0.0001135
26 / 29

27. Problem statement Approach Questions Testbed Results Conclusions
Experimental results - Length of switch queues
The length of switch queues impacts performance:
Long queues cause wide rate oscillations and high latency.
Nevertheless, when the fairness is not enforced by the network,
the impact of different RTTs is reduced.
Short queues keep the variance and the latency low, so fairness
and stability are enhanced
27 / 29

28. Problem statement Approach Questions Testbed Results Conclusions
Conclusions
The proposed allocation engine proved to be very effective in
guaranteeing minimum rates and in the fair repartition of the
excess bandwidth among users.
UGUALE allows an easy assignment of bandwidth shares to
users and the maximum link utilization using only the
instruments already available in modern switches.
The promising results obtained by UGUALE makes us believe
that such an approach is worth of further analysis.
28 / 29

29. Problem statement Approach Questions Testbed Results Conclusions
Future works
Future works include:
The development of an analytical model
The execution of experiments with real traffic traces (users
with different end-to-end congestion control mechanisms, short
and long lived flows, on-off traffic sources). Other QoS
parameters can be evaluated, such as delay, jitter and Flow
Completion Time (FCT)
The definition of a network architecture embracing UGUALE
29 / 29

30. Thanks for the attention