Model predictive (MP) control as a novel active queue management (AQM) algorithm in dynamic computer networks is proposed. According to the predicted future queue length in the data buffer, early packets at the router are dropped reasonably by the MPAQM controller so that the queue length reaches the desired value with minimal tracking error. The drop probability is obtained by optimizing the network performance. Further, randomized algorithms are applied to analyze the robustness of MPAQM successfully, and also to provide the stability domain of systems with uncertain network parameters. The performances of MPAQM are evaluated through a series of simulations in NS2. The simulation results show that the MPAQM algorithm outperforms RED, PI, and REM algorithms in terms of stability, disturbance rejection, and robustness.

1. ISA Transactions 51 (2012) 120–131
Contents lists available at SciVerse ScienceDirect
ISA Transactions
journal homepage: www.elsevier.com/locate/isatrans
Design and analysis of a model predictive controller for active queue
management
Ping Wang, Hong Chen∗
, Xiaoping Yang, Yan Ma
Department of Control Science and Engineering, Jilin University, Campus NanLing, 130025 Changchun, PR China
a r t i c l e i n f o
Article history:
Received 27 March 2010
Received in revised form
11 May 2011
Accepted 25 August 2011
Available online 1 October 2011
Keywords:
Network congestion control
Active queue management
Model predictive control
Stability
Randomized algorithms
a b s t r a c t
Model predictive (MP) control as a novel active queue management (AQM) algorithm in dynamic
computer networks is proposed. According to the predicted future queue length in the data buffer,
early packets at the router are dropped reasonably by the MPAQM controller so that the queue length
reaches the desired value with minimal tracking error. The drop probability is obtained by optimizing the
network performance. Further, randomized algorithms are applied to analyze the robustness of MPAQM
successfully, and also to provide the stability domain of systems with uncertain network parameters. The
performances of MPAQM are evaluated through a series of simulations in NS2. The simulation results show
that the MPAQM algorithm outperforms RED, PI, and REM algorithms in terms of stability, disturbance
rejection, and robustness.
© 2011 ISA. Published by Elsevier Ltd. All rights reserved.
1. Introduction
With the rapid development of the Internet, network con-
gestion is occurring more frequently. Active queue management
(AQM) has become an attractive control strategy for congestion
avoidance [1,2]. The random early detection (RED) [3] method,
popular at the beginning of studying AQM controllers, however,
is too sensitive to parameter configuration. A deep insight of
the system dynamics is helpful to find new control algorithms.
To this end, a fluid-based dynamic model of the transmission
control protocol (TCP) was developed by using stochastic theory
in [4]. Based on this TCP model, the fundamentals of control the-
ory have been used to analyze and develop new AQM schemes.
Proportional–integral (PI) controllers [5] and some advanced pro-
portional–integral–derivative (PID) controllers [6] have been pro-
posed to improve the performance of TCP/AQM systems. Although
these controllers enhance the performance of network systems in
wide network scenarios, the control parameters configured in par-
ticular network scenarios are not able to be adjusted. Model pre-
dictive control (MPC) [7–9] predicts the system dynamics in the
future and then determines the optimal control signal during each
sampling time. Generalized predictive control (GPC), belonging to
the MPC family, has been initially used for network congestion con-
trol [10]. Predictive function control (PFC), which is one of the MPC
∗ Corresponding author. Tel.: +86 431 85094831; fax: +86 431 85095243.
E-mail addresses: wangping08@mails.jlu.edu.cn (P. Wang), chenh@jlu.edu.cn
(H. Chen), yxp@jlu.edu.cn (X. Yang), yma@jlu.edu.cn (Y. Ma).
variants, has also been proposed as an AQM method in [11]. The
time delay in the forward/backward channel can actively be com-
pensated using a model predictor.
Based on control theory, the stability is first considered in
the system design. Stability analysis plays an important role in
selecting the parameters of AQM controllers. The authors of [5]
showed that the RED method cannot guarantee stability when
network delays are taken into account. A linearized stability
analysis based on the simplified model used in [4] has been
developed by taking a proportional controller as the AQM strategy
in [12]. The necessary and sufficient conditions of asymptotic
stability of the linearized models are obtained. In addition, the
stability of congestion control of networks with large round-trip
communication delays was addressed in [13]. The authors of [10]
analyzed the nominal stability of GPC as an AQM algorithm.
In fact, the robustness of the congestion control algorithm is
still not well addressed under a dynamic network environment.
In the process of improving AQM algorithms, the objective is to
establish stability in a parameter space taken as large as possible.
A PI controller [5] has stronger robustness than RED in buffer
queue regulation. However, it performs poorly when round-trip
time (RTT) and load variations occur in real networks [14]. In
most cases, the parameters of the linearized model are nonlinearly
related, so local stability and robustness with respect to parameter
variations cannot be accomplished analytically. The study reported
in [15] resorted to non-analytic tools, in which randomized
algorithms were used to analyze the stability and robustness of
the network system. The study of randomized algorithms has
aroused great interest in the systems and control community [16].
0019-0578/$ – see front matter © 2011 ISA. Published by Elsevier Ltd. All rights reserved.
doi:10.1016/j.isatra.2011.08.006

2. P. Wang et al. / ISA Transactions 51 (2012) 120–131 121
They provide extensive insight into local stability of nonlinear
control algorithms. Furthermore, one can obtain accurate results
on stability margins. These randomized algorithms are efficient
and of low complexity.
The main contributions of this paper are as follows. First,
we propose a new congestion control algorithm based on MPC,
called MPAQM. In this scheme, the queue length is predicted
based on the extended TCP/AQM system model and the state
estimator. And the network performance index is converted to
the objective function of optimal control. The drop probability
in the bottleneck link is obtained by solving the optimization
problem. Second, properties of the closed-loop system controlled
with MPAQM are presented. Furthermore, the margins of robust
stability are given by randomized algorithms under uncertain
network parameters. Finally, extensive simulations are conducted
to demonstrate the effectiveness of MPAQM. The simulation results
show that the MPAQM algorithm achieves shorter response time,
smaller overshoot, and better stability than RED, PI, and REM.
MPAQM also has better responsiveness and robustness.
The paper is organized as follows. In Section 2, the fluid
flow model and the extended model of the network system are
discussed. In Section 3, the MPAQM scheme and some guidelines
for parameter settings are explained. Stability and robustness
analyses based on randomized algorithms are presented in
Section 4. The simulation results of the performance of MPAQM
are illustrated by using the network simulator 2 (NS2) in Section 5.
Finally, we present our conclusions in Section 6.
2. Modeling of TCP/AQM
In this section, we discuss the model of a TCP/AQM intercon-
nection system, and the extended system model considering input
delay.
2.1. Fluid flow model
Consider a single link of capacity C shared by N TCP Reno
sources. Ignoring the TCP timeout mechanism, a fluid model of
TCP behavior is described by the following coupled and nonlinear
differential equations [6]:



˙w(t) =
w(t − R(t))
w(t)R(t − R(t))
(1 − p(t − R(t)))
−
w(t)w(t − R(t))
2R(t − R(t))
p(t − R(t)),
˙q(t) = −C + N
w(t)
R(t)
,
(1)
where w denotes the average TCP window size (packets), q the
average queue length (packets), C the link capacity (packets/s),
R(t) = q(t)/C + Tp the transmission RTT (s), Tp the propagation
delay (s), N the number of TCP sessions, and p the drop probability
of a packet, respectively.
In [5], it is assumed that (w(t−R(t))/w(t))(1−p(t−R(t))) ≈ 1
with a dropping probability p(·) which is close to zero, so that its
additive increase multiplicative decrease (AIMD) model is
˙w(t) =
1
R(t)
−
w(t)w(t − R(t))
2R(t − R(t))
p(t − R(t)). (2)
We now approximate the dynamic model (1) by small-signal
linearization about an operating point. Take (w, q) as the state, p as
the input, and q as the output. For a given desired queue length q0,
we can derive the following operating point (w0, q0, p0) of model
(1):
w2
0p0 = 2(1 − p0), w0 =
R0C
N
, R0 =
q0
C
+ Tp. (3)
See Appendix for further details. Ignoring the dependence of the
time-delay argument t −R(t) on the queue length, we linearize (1)
at the operating point to obtain
δ¨q(t) = A1δq(t) + A2δ˙q(t) + B1δp(t − R0), (4)
where δq = q − q0, δp = p − p0, δ¨q = ¨q, δ˙q = ˙q, A1 =
− 2CN
R0(2N2+C2R2
0
)
, A2 = −
2CNR0+2N2+C2R2
0
R0(2N2+C2R2
0
)
, B1 = −
2N2+C2R2
0
2R2
0
N
. The details
of the derivation can be found in [6].
The AQM control strategy introduced in this paper is based on
model (4).
2.2. Extended system model
Model (4) can be represented as the following state-space
model:
˙x(t) = Ax(t) + Bu(t − R0), (5a)
y(t) = Csx(t), (5b)
where
x =
[
δq
δ˙q
]
, A =
[
0 1
A1 A2
]
, B =
[
0
B1
]
,
u = δp, Cs =

1 0

.
By discretizing (5) at the sampling instants kTs [17], the
discrete-time model is consequently given as
x(k + 1) = Adx(k) + Bd1u(k − τ) + Bd2u(k − τ − 1),
y(k) = Cdx(k),
(6)
where, Ad = eATs , Bd1 =
 R0−τTs
0
eAs
Bds, Bd2 =
 Ts
R0−τTs
eAs
Bds, Cd =
Cs, τ =

R0
Ts

(

R0
Ts

rounds
R0
Ts
to the nearest integer less than or
equal to
R0
Ts
).
To guarantee regulation with zero steady-state error for the
constant set-point, let the control variable u be the following
discrete-time integrator:
u(k) = u(k − 1) + u(k). (7)
Because model (6) highlights explicitly the presence of the system
time delay, we define the state vector X(k) and matrices ˜Ad, ˜Bd, ˜Cd
as follows [18]:
X(k) =






x(k)
u(k − 1)
u(k − 2)
...
u(k − τ − 1)






,
˜Ad =








Ad 0 0 · · · Bd1 Bd2
0 1 0 · · · 0 0
0 1 0 · · · 0 0
0 0 1 · · · 0 0
...
...
...
...
...
...
0 0 0 · · · 1 0








,
˜Bd =






0
1
0
...
0






, ˜Cd =






Cd
0
0
...
0






T
.
Hence, system (6) can be extended to
X(k + 1) = ˜AdX(k) + ˜Bd u(k), (8a)
y(k) = ˜CdX(k). (8b)
Till then, the network system dynamics is described by (8), which
is the basic model of the MPC controller designed in the next
section.

3. 122 P. Wang et al. / ISA Transactions 51 (2012) 120–131
3. Model predictive algorithm of AQM
In this section, a new AQM algorithm, MPAQM, based on MPC is
designed. The objective is to stabilize the queue length as a target
value for improving the network performance and smoothing out
the burst traffic under arbitrary feedback delays.
The key idea of MPAQM is described in three steps. First,
the state vector of model (8) is estimated. We begin with the
estimated state vector, and predict the queue length in the future
finite horizon based on model (8). Second, the drop probability
is determined by solving the optimization problem in order to
achieve faster responses and prevent excessive control effort.
Finally, the optimal drop probability is applied to the network
system as the feedback control signal. According to the basic
principles of predictive control, this process is repeated at each
sampling time, which makes queue length reach the desired value
quickly. The design process of each part is introduced as follows.
3.1. State estimator design
The state vector X(k) of model (8) has to be estimated because
the second component in x(k) is not available. Here, the estimator
is designed by the state-space method. When the estimated state
ˆX(k) and the control input increment u(k) at time k are given,
state ˆX(k + 1) is calculated by (8a)
ˆX(k + 1) = ˜Ad
ˆX(k) + ˜Bd u(k) + Kf (y(k) − ˜Cd
ˆX(k)), (9)
where Kf is the gain matrix of the estimator. The design criteria
for Kf are to guarantee the stability of the estimator and the fast
convergence of estimation error.
In this study, Kf is based on the reginal pole placement theory
and LMI (linear matrix inequality) techniques, such that the
distribution of eigenvalues of ˜Ad −Kf
˜Cd is a disk region with center
at the origin and radius r0 (r0 < 1). The condition of the reginal
pole placement is given as follows.
Lemma 1 ([19]). The square matrix A has all its eigenvalues σ(A) ⊂
D(r, o) if and only if there exists a symmetric X > 0 such that
[
−rX AX − oX
XAT
− oX −rX
]
< 0, (10)
where D(r, o) is a disk with center at o and radius r.
From Lemma 1, we derive the following LMI for estimator (9).
Corollary 1. For the square matrix ˜Ad − Kf
˜Cd, σ(˜Ad − Kf
˜Cd) ⊂
D(r0, 0) if and only if there exists a symmetric X > 0 and Q , such
that
[
−r0X ˜AT
d X − ˜CT
d Q T
X ˜Ad − Q ˜Cd −r0X
]
< 0, (11)
when the solutions of Q and X of LMI (11) exist, Kf = X−1
Q is one
of the appropriate gain matrices of estimator (9).
Remark 3.1. It should be pointed out that, as long as (˜Cd, ˜Ad) is
detectable, the solution of LMI (11) exists for any given r0 < 1,
i.e., one can find a Kf through (11) such that the estimator (9) is
stable.
3.2. Future queue length prediction
According to the principles of predictive control [7], at time k,
the coming queue length is predicted on the basis of model (8).
Here, Nc is defined as the control horizon, and Np is defined as
the prediction horizon; hence, the prediction function of the queue
length is as follows:
YNp (k + 1|k) = Sx
ˆX(k) + Su U(k), (12)
where
YNp (k + 1|k)




y(k + 1|k)
y(k + 2|k)
...
y(k + Np|k)



 ,
U(k)




u(k)
u(k + 1)
...
u(k + Nc − 1)



 ,
Sx =





˜Cd
˜Ad
˜Cd
˜A2
d
...
˜Cd
˜A
Np
d





,
Su =





˜Cd
˜Bd 0 0 · · · 0
˜Cd
˜Ad
˜Bd
˜Cd
˜Bd 0 · · · 0
...
...
... · · ·
...
˜Cd
˜A
Np−1
d
˜Bd
˜Cd
˜A
Np−2
d
˜Bd · · · · · · ˜Cd
˜A
Np−Nc
d
˜Bd





.
3.3. Optimization and feedback control
As mentioned above, the main control requirement of AQM
algorithms is to make the queue length q stabilize near the target
value q0 as soon as possible to avoid network congestion. But
it is difficult to control the TCP sending window because of the
time delay. MPAQM determines the drop probability based on
the predicted future queue length YNp (k + 1|k). The reference
sequences of the queue length are defined as
Re(k + 1) =

r(k + 1) r(k + 2) · · · r(k + Np)
T
,
where r(k + i) = 0, i = 1, 2, . . . , Np (because the origin is the
operating point of model (4)). The queue length q can converge
quickly to q0 by minimizing
J1 = ‖YNp (k + 1|k) − Re(k + 1)‖2
.
In addition, network fluctuations should be prevented for
guaranteeing the quality of the network service. That is to say,
the variation range of drop probability should be limited in
AQM algorithms. In MPAQM, this requirement is formulated as
minimizing J2 = ‖ U(k)‖2
.
Because minimizing J1 and J2 simultaneously is contradictive,
the weighting factors for different request are given. The MPAQM
algorithm is described by the following optimization problem:
min
U(k)
J(y(k), U(k), Nc , Np), (13a)
J(y(k), U(k), Nc , Np) = ‖Γy(YNp (k + 1|k) − Re(k + 1))‖2
+ ‖Γu U(k)‖2
, (13b)
where Γy and Γu are the weighting matrices.
By solving the optimization problem (13), the optimal control
sequences at time k are derived:
U∗
(k) = (ST
u Γ T
y ΓySu + Γ T
u Γu)−1
ST
u Γ T
y ΓyEp(k + 1|k), (14)
where Ep(k + 1|k) = Re(k + 1) − Sx
ˆX(k).
The first component of U∗
(k) is effectively used to compute
the control signal u(k) according to (7). Hence, the closed-loop
control law is defined as
u(k) = KmpcEp(k + 1|k), (15)
where Kmpc =

1 0 · · · 0

(ST
u Γ T
y ΓySu + Γ T
u Γu)−1
ST
u Γ T
y Γy.

4. P. Wang et al. / ISA Transactions 51 (2012) 120–131 123
4. Closed-loop properties
In this section, we use the Routh–Hurwitz theorem, a common
technique in control theory, to analyze the stability of the MPAQM
algorithm, and we use randomized algorithms to analyze the
robustness under uncertain network parameters.
4.1. Stability analysis
In order to analyze the stability of the estimator-based MPAQM
algorithm, we first discuss the stability of the state feedback
MPAQM. To do this, it is assumed that all states are measured, i.e.,
ˆX = X. Then, the feedback control (15) becomes
u(k) = Kmpc (Re(k + 1) − SxX(k)) , (16)
and the corresponding closed-loop system is given as
X(k + 1) =

˜Ad − ˜BdKmpcSx

X(k) + ˜BdKmpcRe(k + 1). (17)
It is then clear that the state feedback MPAQM algorithm is
stable if and only if ˜Ad − ˜BdKmpcSx is stable, i.e., all its eigenvalues
lie in the unit circle.
Now we come back to the estimator-based case and need to
obtain the corresponding closed-loop system. The network system
is still described by (8a), and the estimator is given by (9). By
substituting (15) into (9) and (8a), we have
X(k + 1) = ˜AdX(k) − ˜BdKmpcSx
ˆX(k) + ˜BdKmpcRe(k + 1), (18a)
ˆX(k + 1) = Kf
˜CdX(k) + (˜Ad − Kf
˜Cd − ˜BdKmpcSx)ˆX(k)
+ ˜BdKmpcRe(k + 1). (18b)
Hence the estimator-based closed-loop system is given as
[
X(k + 1)
ˆX(k + 1)
]
=
[
˜Ad −˜BdKmpcSx
Kf
˜Cd
˜Ad − Kf
˜Cd − ˜BdKmpcSx
]
×
[
X(k)
ˆX(k)
]
+
[
BdKmpc
BdKmpc
]
Re(k + 1). (18c)
It is then clear that system (18c) is stable if and only if all the
eigenvalues of
[
˜Ad −˜BdKmpcSx
Kf
˜Cd
˜Ad − Kf
˜Cd − ˜BdKmpcSx
]
(19)
lie in the unit circle.
By taking a similarity transformation with
P =
[
I 0
−I I
]
, P−1
=
[
I 0
I I
]
(20)
on (19), we have
P
[
˜Ad −˜BdKmpcSx
Kf
˜Cd
˜Ad − Kf
˜Cd − ˜BdKmpcSx
]
P−1
=
[
˜Ad − ˜BdKmpcSx −˜BdKmpcSx
0 ˜Ad − Kf
˜Cd
]
, (21)
where ˜Ad − ˜BdKmpcSx is the closed-loop system matrix when all
states are assumed to be measured, and ˜Ad − Kf
˜Cd is the system
matrix of the state estimator. Hence, we arrive at the following
separation principle.
Theorem 1. The estimator-based MPAQM is stable if and only if the
state feedback MPAQM and state estimator are stable, respectively.
As an example for analyzing the stability of closed-loop system
(18c), we consider a dumbbell topology with a single common
bottleneck link of 15 Mb/s capacity (see Fig. 1). The other
parameters in model (4) are as follows: the round-trip propagation
delay Tp is 50 ms, the TCP flows N is 120, and the mean packet size is
500 bytes. Note that the operating point (w0, p0, q0) in (3) depends
Fig. 1. The system network model.
on the desired queue length q0, which is usually less than one half
the buffer size [13,14].
We set q0 = 200 packets, and obtain the operating point R0 =
0.1033 s, p0 = 0.175. We set Ts = 0.01 s; then τ = 10 is obtained.
For a better result of estimation, r0 in LMI (11) is set to be 0.4. By
solving LMI (11), we obtain
Kf =

2.9317 234.7818 − 0.1642 − 0.1642 − 0.1642
−0.1642 − 0.1642 − 0.1642 − 0.1642
−0.1642 − 0.1642 − 0.1642 − 0.1643
T
.
The maximum modulus of the eigenvalues of matrix ˜Ad − Kf
˜Cd is
0.2563. For the MPC controller, the weights of the cost function are
chosen to be Γu = 2000I15, Γy = I15 (I15 is the unit matrix in the
15 × 15-dimensional space); the prediction and control horizons
are set to be Np = 15 and Nc = 15, respectively. The maximum
modulus of the eigenvalues of matrix (19) is 0.984, which implies
that the closed-loop system (18c) is stable.
Remark 4.1. The sampling interval affects the discrete-time
model (6) and the calculation burden of the MPAQM algorithm.
Among existing AQM schemes, the control mechanism of the PI
scheme is most similar to that of MPAQM scheme. In [5], it is
advisable to operate the PI controller at 10–20 times the loop
bandwidth, and its sampling frequency is 160 Hz. We assume that
the MPAQM and PI methods have the same bandwidth. To reduce
the computation burden, the sampling frequency in our study is set
to be 100 Hz (Ts = 0.01 s).
4.2. Robustness analysis
The nominal stability of closed-loop system (18c) has just been
analyzed when the network parameters are constant. Actually,
the TCP loads are time-varying loads, which directly affects
the congestion degree, and R0 is the approximate RTT value in
equilibrium. Moreover, the model mismatch problem does exist in
model (4), where high-frequency dynamics are not strictly taken
into account. So we further discuss the robustness of closed-loop
system (18c).
Because of the random characteristics of the network, random-
ized algorithms are used to analyze the robustness. Considering the
uncertainty of parameters N and R0, we derive the following model
from model (6):
x(k + 1) = Ar
dx(k) + Br
d1u(k − τ) + Br
d2u(k − τ − 1), (22)
where Ar
d, Br
d1, Br
d2 are random variables. In order to simplify the
derivation, the input delay is assumed to be constant.
Following the derivation in Section 2.2, we obtain the
corresponding extended system:
X(k + 1) = ˜Ar
dX(k) + ˜Br
d u(k), (23a)
y(k) = ˜CdX(k). (23b)
By substituting (15) into (23a), we have
X(k + 1) = ˜Ar
dX(k) − ˜Br
dKmpcSx
ˆX(k)
+ ˜Br
dKmpcRe(k + 1). (24)

5. 124 P. Wang et al. / ISA Transactions 51 (2012) 120–131
Fig. 2. The first 800 samples of various two-dimensional uniformly distributed
pseudorandom sequences.
The corresponding closed-loop system is
[
X(k + 1)
ˆX(k + 1)
]
=
[
˜Ar
d −˜Br
dKmpcSx
Kf
˜Cd
˜Ad − Kf
˜Cd − ˜BdKmpcSx
]
×
[
X(k)
ˆX(k)
]
+
[
Br
dKmpc
BdKmpc
]
Re(k + 1). (25)
Then, we analyze the stability of
L =
[
˜Ar
d −˜Br
dKmpcSx
Kf
˜Cd
˜Ad − Kf
˜Cd − ˜BdKmpcSx
]
(26)
in the probabilistic sense based on Monte Carlo methods.
Step 1. The parameters N and R0 are taken to be random, with
uniform density functions fN and fR0
. Then, we generate M inde-
pendent identically distributed samples according to the density
function fN and fR0
: N1
, N2
, . . . , NM
, R1
0, R2
0, . . . , RM
0 , respectively.
Subsequently, we compute Ar
d(Ni
, Ri
0), Br
d1(Ni
, Ri
0), Br
d2(Ni
, Ri
0) in
(22) for i = 1, 2, . . . , M, where we have suppressed the de-
pendence on the capacity C. By the derivation from (22) to (25),
L(Ni
, Ri
0) is computed for the robustness analysis.
Step 2. Construct the indicator function
I(Ni
, Ri
0) =

1 If L(Ni
, Ri
0) is Schur
0 otherwise.
(27)
The estimated probability of stability is readily given by
ˆpM =
1
M
−M
i=1
I(Ni
, Ri
0). (28)
The estimation ˆpM is usually referred to as the empirical
probability. We need to know the samples size M to obtain a
‘‘reliable’’ probabilistic estimation ˆpM . Thus, a Chernoff bound is
used. The Chernoff bound [20] states that, for any ϵ ∈ (0, 1) and
δ ∈ (0, 1), if
M ≥
1
2ϵ2
ln

2
δ

(29)
then, with probability greater than 1 − δ, we have |ˆpM − ptrue| <
ϵ, where ptrue denotes the real probability of stability. When the
samples size M is determined, another important issue in Monte
Carlo methods is how to choose good random number generators.
We use a multiplicative congruential generator. The quasi-random
sequences generated by the multiplicative congruential method
are shown in Fig. 2.
We analyze the robustness of the concrete network described
in Section 4.1 by Monte Carlo methods. The parameters in (29) are
set as follows: δ = 0.001 and ϵ = 0.008. We choose the number
of samples as M = 60 000, based on the Chernoff bound. We want
Fig. 3. The stability probability with uncertain N and R0.
Fig. 4. The stability probability versus capacity.
to find the stability domain in the parameter space (N, R0) with
the MPAQM algorithm. Fig. 3 plots the stability probability when
the uncertainties of N and R0 change from 5% to 65% around the
normal values 120 and 0.1033, respectively. Fig. 3 shows that the
system is stable with ±55% uncertain N and ±50% uncertain R0.
We further discuss the influence of different link capacity C
on the system stability. We perform simulations with parameter
ranges
N ∈ [54, 186], R0 ∈ [0.0517, 0.155]
and 20 fixed values of capacity:
C = 9.6, 10.2, 10.8, 11.4, . . . , 20.4, 21 Mb/s.
The samples are generated by the multiplicative congruential and
grid method, respectively. The stability probability is shown in
Fig. 4. The simulation result shows that the stability of the system
with MPAQM decreases, i.e., the queue length in the buffer of
bottleneck link cannot reach the target value of 200 packets, when
the capacity C is greater than 15 Mb/s.
5. Simulation results
In this section, we evaluate the performance and robustness
of the proposed MPAQM algorithm by a number of simulations
performed in NS2 [21]. The well-known RED, PI, and REM methods
are also simulated for the purpose of comparison.

6. P. Wang et al. / ISA Transactions 51 (2012) 120–131 125
a b
Fig. 5. Queue length and drop probability of MPAQM.
Fig. 6. Queue length and drop probability of RED.
5.1. Performance evaluation
The performance of MPAQM is compared with that of RED,
PI, and REM under FTP flows and mixture flows, respectively.
The network topology and the model parameters are given in
Section 4.1. Some other parameters used in NS2 are set as follows:
the buffer size B is 500 packets, and the window size of each TCP
connection is 30 packets.
5.1.1. Under FTP flows only
First, we investigate the stability and responsiveness of MPAQM
under long-lived FTP flows only. Each sender–receiver pair has TCP
connections as cross traffic. In both scenarios, TCP Reno is used as
the transport agent. The parameters of MPAQM are set to be the
values calculated in Section 4.1.
For a fair comparison in some sense, the settings of the
parameters for the following AQM schemes are based on their
authors’ recommendations and have some refinements for this
network topology. The basic parameters of RED (see the notation
in [1]) are set at intervaltime = 0.5 s, minth = 100 packets,
maxth = 280 packets, maxp = 0.01, and wq = 0.002. For the
PI scheme, the control parameters are fixed following the rule
design recommended in the original work [5]: a = 4.389 ×
10−5
, b = 4.346×10−5
. The settings of the parameters for the REM
scheme are based on author recommendations [22]: α = 0.1, φ =
1.002, γ = 0.001, b∗
= 200 packets.
Figs. 5–8 show the queue lengths and the drop probability ob-
tained with different AQM schemes. It can be seen that the queue
lengths in Figs. 5, 7 and 8 stabilize after about 4.6 s, 10.3 s, and
7.8 s, respectively, while the queue length of RED fail to do so (it
does at about 230 packets). It should be noted that the RED, PI,
and REM methods are designed based on the nominal values of
network parameters to regulate the queue length; therefore, the
best results are expected in this experiment. Other performance
measures are given in Table 1. The variation of queue length in the
MPAQM method is low and it is the highest in the PI method. Al-
though the bottleneck link utilization of the various AQM meth-
ods is rather similar, the drop probability in RED has the highest
value and in the MPAQM method it has the lowest value. Through
comparing, MPAQM gives less overshoot, lower drop probabil-
ity, shorter response time,and better stability than the other
methods.
5.1.2. Under mixture flows
The unresponsive constants bit rate (CBR) flows are like
a constant disturbance. It can influence the control effect of
AQM algorithms as a result of queue oscillations or unstable
queue evolution. In this simulation, we use a mixture of FTP
and CBR flows to simulate a more realistic network scenario.
In Fig. 1, the number of FTP flows and CBR flows are 120
and 15, respectively. The Internet protocol used by CBR flows
is user data protocol (UDP). The inter-packet gap of a CBR

7. 126 P. Wang et al. / ISA Transactions 51 (2012) 120–131
Table 1
Performance measures under FTP flows.
Average q (packets) STD q (packets) Drop probability (%) Link utilization (%)
MPAQM 199.657 28.6048 7.2467 99.9012
RED 233.75 57.72 10.23 99.9891
PI 199.739 63.5286 8.7433 99.4178
REM 205.82 46.27 7.36 99.9015
Table 2
Performance measures under mixture flows.
Average q (packets) STD q (packets) Drop probability (%) Drop probability (10–30 s)(%) Link utilization (%)
MPAQM 196.848 56.1698 15.8145 24.071 99.5156
RED 252.924 110.3782 12.66 17.32 99.9807
PI 200.508 67.72 18.1347 20.4133 98.5732
REM 197.66 147.725 17.47 21.93 91.0271
Fig. 7. Queue length and drop probability of PI.
Fig. 8. Queue length and drop probability of REM.
flow is 0.005 s, and the mean packet size is 500 bytes. The
CBR flows start at 10 s and stop at 30 s. The parameters for
MPAQM, RED, PI, and REM are unchanged from the values used in
Section 5.1.1.
From Figs. 9–12, we see that MPAQM can robustly stabilize
the queue length around 200 packets, while the queue length
of PI keeps the peak value for around 8 s. REM requires much
longer time to decrease its queue size from the buffer top. The
queue length in the RED method fluctuates and is far from the
desired value when UDP flows exist. Other performance measures
are shown in Table 2. As is shown in this table, the queue length
variation in the MPAQM method is less than in the others. The
performances of RED and REM degrade noticeably. The variation of
queue length in the REM method has the highest value. This leads
to the underutilization of resources and poor quality of service.
Although the loss rate in the RED method has the lowest value,
the average queue length is very far from the desired value. The
results also show that the queue evolutions of the MPAQM and PI

8. P. Wang et al. / ISA Transactions 51 (2012) 120–131 127
Fig. 9. Queue length and drop probability of MPAQM under mixture flows.
Fig. 10. Queue length and drop probability of RED under mixture flows.
Fig. 11. Queue length and drop probability of PI under mixture flows.
schemes are based on higher drop probability, but the performance
of MPAQM is superior to that of PI.
5.2. Robustness evaluation
The robustness of various AQM methods is evaluated under
network uncertainty. According to the analysis in Section 4.2, we
conduct a set of simulations to investigate the robustness through
the TCP connections from 60 to 180 and the round-trip propagation
delay from 30 ms to 80 ms. Details of the network setting are given
in Table 3.
5.2.1. Robustness with variables N and Tp
The bottleneck link capacity and the parameters of MPAQM are
the same as the values used in Section 4.1. The parameters N and
Tp are set according to Table 3.

9. 128 P. Wang et al. / ISA Transactions 51 (2012) 120–131
Fig. 12. Queue length and drop probability of REM under mixture flows.
Fig. 13. Queue length and drop probability of MPAQM with variables N and Tp.
Fig. 14. Queue length and drop probability of RED with variables N and Tp.
Fig. 13 shows that the queue length in the MPAQM algorithm
robustly stabilizes around the target value, and the drop probabil-
ity increases to some degree, while the number of TCP connections
increases. Moreover, MPAQM has a rapid regulating process when
new TCP connections start to send packets and old TCP connections
stop sending packets. From 40 to 60 s, larger values of the propaga-
tion delay are considered together with load variations represent-
ing a strong congestion scenario. Notice that, in Fig. 14, RED does
not achieve a good queue stabilization from a medium load value
of about 150 sessions. In Fig. 15, the PI method shows consistent

10. P. Wang et al. / ISA Transactions 51 (2012) 120–131 129
Fig. 15. Queue length and drop probability of PI with variables N and Tp.
Fig. 16. Queue length and drop probability of REM with variables N and Tp.
Table 3
Network setting in robustness evaluation.
Number of active TCP sessions N 30 30 30 30 30 30
Propagation delay Tp (ms) 30 40 50 60 70 80
Starting time (s) 0 0 10 20 30 40
Stopping time (s) 100 100 90 80 70 60
variations of the queue length and drop probability. We observe
that the PI takes the longer time to settle down the equilibrium
point. It can be seen from Fig. 16 that REM shows an acceptable
behavior when we vary N from 30 to 150 sessions. But it performs
poorly for the reverse process. From these figures, we can find that
MPAQM achieves a short response time, good stability, and good
robustness.
5.2.2. Robustness with different C
Based on the results in Section 5.2.1, the experiments in this
subsection aim to investigate the ability of the MPAQM scheme
under varying bottleneck link capacities. For sake of brevity, only
the performance of the MPAQM controller with different C is
shown because other AQM schemes presented worse behavior.
Four fixed values of bottleneck link capacity are set: C = 5, 10,
20, 25 Mb/s, respectively. The other parameters of the network and
MPAQM algorithm are still the values used in Section 5.2.1.
Fig. 17 plots the queue evolution of the MPAQM method under
different C, which shows that the queue length fails to stabilize
around 200 packets over a period of time when C = 20 Mb/s and
C = 25 Mb/s. The stability with 25 Mb/s capacity is worse than
that with 20 Mb/s. The simulation results verify the analysis result
indicated in Fig. 4.
5.2.3. Multiple bottleneck topology with cross traffic
Using the multiple bottleneck network topology depicted in
Fig. 18, we study the behavior of different AQM algorithms in the
presence of cross traffic. We set 120 FTP flows with senders at
the left-hand side and receivers at the right-hand side, and 30 FTP
flows with each cross sender–receiver pair. The simulated network
has two bottleneck links (link R2–R3, link R4–R5). Because link
R2–R3 and link R4–R5 exhibit similar trends, we do not plot the
data of link R4–R5. The queues of R2–R3 for different AQMs are
depicted in Fig. 19.
In Fig. 19, we can see that all the AQM schemes have noticeable
oscillations in this case with multiple bottlenecks and cross traffic;
yet MPAQM controls the queue length significantly better than
others. REM and PI exhibit larger queue oscillation than MPAQM.
RED oscillates largely in the congested routers, and frequently
results in poor link utilization and continuous packet losses. The
queue lengths of the PI and RED schemes are still frequently below
the expected value (200 packets). In contrast, MPAQM can regulate
the queue length around the expected value and obtain a better
control effect than the other schemes.

11. 130 P. Wang et al. / ISA Transactions 51 (2012) 120–131
Fig. 17. Queue length of MPAQM with different capacity C.
Fig. 18. The system network model.
6. Conclusions
In this paper, we have proposed a novel AQM scheme based
on MPC, i.e., MPAQM. The components of the basic framework
are the extended state-space model of the network system, a
state estimator based on the state-space method, prediction of the
queue length, and an optimal controller for improving the network
performance.
The stability analysis gives the guidelines on how to select the
parameters of the MPAQM algorithm to ensure the stability of
the network system. Considering the uncertainty of the network
parameters, a robustness analysis based on Monte Carlo methods
provides the accurate margins of stability.
We conducted extensive simulations and have comprehen-
sively compared the performance of MPAQM with RED, PI, and
REM. The performance analysis and simulation results illustrate
the effectiveness of the MPAQM algorithm, which provides bet-
ter performances than RED, PI, and REM. The queue length of
MPAQM converges to the desired value more quickly. The dynamic
response of the queue length has smaller oscillations in the pres-
ence of unresponsive UDP flows or parameter variations. In par-
ticular, the queue length dynamics is shown to exhibit good
robustness and fast system response under multiple bottleneck
link scenarios.
Future work will cover the extension of the simulation
environment from numerical simulations in NS2 to real network
experiments and the explicit consideration of time-domain
constraints on the drop probability and queue length.
Appendix. Derivation of (3)
For a given desired queue length q0, the operating point
(w0, q0, p0) of model (1) is defined by ˙w = 0 and ˙q = 0, so that



˙q(t) = 0 ⇒ R0 =
q0
C
+ Tp,
⇒ −C + N
w0
R0
= 0,
˙w(t) = 0 ⇒
1 − p0
R0
−
w2
0
2R0
p0 = 0.
(A.1)
Then, we get (3).

12. P. Wang et al. / ISA Transactions 51 (2012) 120–131 131
Fig. 19. Queue evolution under multiple bottleneck topology.
References
[1] Floyd S, Jacobson V. Random early detection gateways for congestion
avoidance. IEEE/ACM Transactions on Networking 1997;4(1):1–22.
[2] Clark DD, Fang W. Explicit allocation of best effort packet delivery service.
IEEE/ACM Transactions on Networking 1998;6(4):362–73.
[3] Branden B, Clark D, Crowcroft J. Recommendations on queue management and
congestion avoidance in the internet. In: RFC2309. 1994.
[4] Misra V, Gong WB, Towsley D. Fluid-based analysis of a network of AQM
routers supporting TCP flows with an application to RED. In: Proceedings of
ACM/SIGCOMM. 2000.
[5] Hollot CV, Misra V, Towsley D, Gong WB. Analysis and design of controllers for
AQM routers supporting TCP flows. IEEE Transactions on Automatic Control
2002;6(47):945–59.
[6] Kim KB. Design of feedback controls supporting TCP based on the state-space
approach. IEEE Transactions on Automatic Control 2006;7(51):1086–99.
[7] Lee JH, Morari M. Model predictive control: past, present and future.
Computers and Chemical Engineering 1999;23:667–82.
[8] Rawlings JB. Tutorial overview of model predictive control. IEEE Control
Systems Magazine 2000;3(20):38–52.
[9] Veselý V, Rosinová D, Foltin M. Robust model predictive control design with
input constraints. ISA Transactions 2010;49:114–20.
[10] Zhang XH, Zou KS, Chen ZQ, Deng ZD. Stability analysis of AQM algorithm
based on generalized predictive control. In: Proceedings of the 4th inter-
national conference on intelligent computing. Berlin (Heidelberg): Springer-
Verlag; 2008. p. 1242–8.
[11] Bigdeli V, Haeri M. Predictive functional control for active queue management
in congested TCP/IP networks. ISA Transactions 2009;48:107–21.
[12] Michies W, Melchor-Aguilar D, Niculescu SI. Stability analysis of some classes
of TCP/AQM networks. International Journal of Control 2006;9(79):1136–44.
[13] Wang JX, Rong L, Liu YH. Design of a stabilizing AQM controller for large-delay
networks based on internal model control. Computer Communications 2008;
31:1911–8.
[14] Quet PF, Ozbay H. On the design of AQM supporting TCP flows using robust
control theory. IEEE Transactions on Automatic Control 2004;6(49):1031–6.
[15] Tempo R. Randomized algorithms for stability and robustness analysis of high-
speed communication networks. IEEE Transactions on Neural Networks 2005;
5(16):1229–41.
[16] Tempo R, Calafiore G, Dabbene F. Randomized algorithms for analysis and
control of uncertain systems. New York: Springer-Verlag; 2005.
[17] Nilsson J. Stochastic analysis and control of real-time systems with random
time delays. Automatica 1998;1(34):57–64.
[18] Clarke DW, Scattolini R. Constrained receding horizon predictive control.
Proceedings of IEE 1992;(Part D):347–54.
[19] Boyd S, El Ghaoui L, Feron E, Balakishnan V. Linear matrix inequalities in
system and control theory. Philadelphia: SIAM; 1994.
[20] Hellekalek P. Good random number generators are (not so) easy to find.
Mathematics and Computers in Simulation 1998;46(5–6):485–505.
[21] USC/ISI, Angeles L., CA. The NS simulator and the documentation. Available
from: http://www.isi.edu/nsnam/ns/.
[22] Athuraliya S, Low S, Li V, Yin Q. Rem: active queue management. IEEE Network
Magazine 2001;15:48–53.