VIEWS: 3 PAGES: 14 CATEGORY: College POSTED ON: 4/28/2009
1 Exponential-RED: A Stabilizing AQM Scheme for Low- and High-Speed TCP Protocols ¸ Shao Liu, Student Member, IEEE, Tamer Basar, Fellow, IEEE, and R. Srikant, Senior Member, IEEE (shaoliu@ifp.uiuc.edu, tbasar@control.csl.uiuc.edu, rsrikant@uiuc.edu) Abstract— This paper introduces and analyzes a decentral- stable [22], [21]. The main beneﬁt of this algorithm is that it ized network congestion control algorithm which has dynamic achieves arbitrary fairness among the users and leads to full adaptations at both user ends and link ends, a so-called general link utilization. This idea was adopted in [30] to modify the primal-dual algorithm. We obtain sufﬁcient conditions for local stability of this algorithm in a general topology network with dual algorithm to allow slow adaptation at the sources and heterogeneous round-trip delays. Then, as an implementation of achieve the same beneﬁts as the AVQ algorithm. this algorithm in the Internet, we introduce an AQM (Active Both the modiﬁed primal algorithm and the modiﬁed dual Queue Management) scheme called Exponential-RED (E-RED), algorithm have dynamic adaptations at both sources and which outperforms RED and is inherently stable when combined routers, and thus can be regarded primal-dual. However, all with TCP-Reno or its variants for high-speed networks. Keywords: Congestion Control, Primal Algorithm, Dual Algo- the algorithms in the primal family relate the network con- rithm, Primal-Dual Algorithm, TCP, RED. gestion measure directly with the link aggregate rate, which corresponds to averaging the feedback from the network at the sources; and all the algorithms in the dual family relate the I. I NTRODUCTION source rate directly with the route congestion measure, which Recently, there has been a ﬂurry of research activity on de- corresponds to averaging the source rates at the links before centralized end-to-end network congestion control algorithms. the feedback of more explicit congestion information to the A widely-used framework, introduced in [15], is to associate sources [14]. a utility function with each ﬂow and maximize the aggregate In this paper, we ﬁrst brieﬂy review the above developments system utility function subject to link capacity constraints—an and identify their role in allocating resources fairly in a network optimization problem known as Kelly’s System Problem [15]. of competing users. We then generalize the class of primal- Congestion control schemes can be viewed as decentralized dual algorithms, and provide design guidelines to stabilize source and router algorithms to drive the system operating these algorithms in general topology networks with heteroge- point to the optimum or some suboptimum solution of this neous feedback delays. In this class of algorithms, the source maximization problem. dynamics are similar to those in the primal algorithm in [15] Congestion control schemes can be divided into three and [33] while the link dynamics are similar to those in the dual classes: primal algorithms, dual algorithms and primal-dual algorithm [25]. Following the approach introduced in [33], we algorithms. In primal algorithms, the users adapt the source obtain a local stability result which shows that stability does rates dynamically based on the route prices, and the links select not depend on the source adaptation speed, but depends only a static law to determine the link prices directly from the arrival on the link adaptation speed. This result subsumes the dual rates at the links [15]. In dual algorithms, on the other hand, the algorithm as its limiting case when the source adaptation speed links adapt the link prices dynamically based on the link rates, approaches inﬁnity. and the users select a static law to determine the source rates From the stability analysis of the general primal-dual algo- directly from the route prices and the source parameters [15], rithm, we also show that RED (Random Early Detection) could [25], [36]. Primal-dual algorithms combine these two schemes stabilize TCP-Reno if modiﬁed slightly. Our modiﬁcation to and dynamically compute both user rates and link prices [1], RED sets the packet marking probability to be an exponential [35]. For a comprehensive survey of these algorithms, see [31]. function of the length of a virtual queue whose capacity is A modiﬁed primal algorithm, called the AVQ (Active Virtual slightly smaller than the link capacity. Due to the exponential Queue) algorithm, was introduced in [20]. Here the link prices marking proﬁle, we call it Exponential-RED (E-RED). From in the original primal algorithm [15] are slowly adjusted so our analysis, it can be shown that E-RED stabilizes TCP-Reno that asymptotically in time, the link prices become equal to and all its packet loss/mark based variations. Compared with the Lagrange multipliers in Kelly’s system problem [20]. More other queue-length-based AQM schemes, like RED [7], REM importantly, in the presence of feedback delays, the parameters [2], PI [12] and BLUE [6], E-RED is the ﬁrst such scheme that of this algorithm can be chosen such that the network is locally can be proved to stabilize TCP-Reno for a general topology network with heterogeneous delays. In parallel with our work, An earlier version of this paper appeared in the Proceedings of the 42nd IEEE Conference on Decision and Control, 2003; see [24]. another recent paper [10] has also obtained scaling rules for Research supported by the NSF ITR Grant CCR 00-85917, DARPA Grant RED and PI to stabilize TCP-Reno. F30602-00-2-0542 and AFOSR URI F49620-01-1-0365. We ﬁnally perform some ns-2 and Matlab simulations to Department of Electrical and Computer Engineering and Coordinated Sci- ence Laboratory, University of Illinois, 1308 West Main Street, Urbana, IL compare E-RED with RED and to discuss the dependence 61801-2307, USA. of E-RED’s performance on the network scenario and the E- 2 RED parameter choices. The simulation results show that E- A. Primal Algorithm RED outperforms RED when combined with TCP-Reno in An issue of importance, driven by both theoretical and the sense that it achieves less queue length oscillation, higher x ˆ practical considerations, is the distributed computation of (ˆ, p) bandwidth utilization, and lower queueing delay at the same using only the realistic decentralized information available time. The simulation results also show that E-RED works well to individual users. The congestion control problem deals with HighSpeed TCP [8] and Scalable TCP [18]. precisely with the tasks of ﬁnding such algorithms that are II. OVERVIEW OF P RIOR W ORK decentralized, and selecting among them the one with the best performance. The users adapt xr with respect to qr , and the Consider a set of users/routes, R, and a set of links adapt pl with respect to yl . The TCP algorithms are user links/resources, L. For each user r ∈ R, its route consists adaptation laws, and AQM algorithms are link adaptation laws. of a set of links, which is a subset of L, denoted Lr . Each link For the logarithmic utility function, Ur (xr ) = wr log xr , l ∈ L may be used by several routes; accordingly, we write Kelly et al. introduced the Primal Algorithm [15], where each l ∈ r if l ∈ Lr . Each user r ∈ R has an associated ﬂow rate user r implements the following algorithm: xr and a utility function Ur (xr ). Furthermore, each link l ∈ L has an associated link aggregate rate yl = r∈R:l∈r xr , and ˙ xr = kr (wr − xr qr ) , (6) a ﬁxed capacity cl . Introduce the source rates (column) vector where kr is a scaling factor or the step size. Each Link l, on x := (xr , r ∈ R), the link rate (column) vector y := (yl , l ∈ the other hand, computes its price as L), and the routing matrix A := (Alr , l ∈ L, r ∈ R), where Alr = 1 if l ∈ r and 0 otherwise; then we have y = Ax. pl = fl (yl ), where fl > 0, fl > 0 . (7) We impose the standard conditions on each user’s utility Without incorporating delay into the algorithm, it was shown function Ur (xr ), namely that it be increasing, strictly concave, in [15] that this primal algorithm is globally stable. If we and continuously differentiable in xr for xr ≥ 0. Further include propagation delay and neglect queueing delay, thus assume that Ur (xr ) → −∞ as xr → 0, and that the utilities treating delay as a constant, then we have the relationships: are additive so that the aggregate utility for the system is f U (x) = r∈R Ur (xr ). The system problem, ﬁrst introduced yl (t) = xr (t − τlr ) , (8) r:l∈r in this context by Kelly [16], is the concave programming problem: b qr (t) = pl (t − τlr ) , (9) max U (x) subject to y ≤ c over x ≥ 0 , (1) l∈r x f b where τlr is the forward delay from source r to link l, and τlr where c = (cl , l ∈ L), a column vector. The Lagrangian is: is the backward delay from link l back to source r, and we f b L(x, p) = Ur (xr ) + pl (cl − yl ) , (2) have round trip delay Tr := τlr + τlr , for any l ∈ r. r∈R l∈L Denote the Laplace transforms of x(t), y(t), p(t), and q(t), where pl is the Lagrange multiplier associated with link l, l ∈ respectively, by X(s), Y (s), P (s), and Q(s), and deﬁne the L. This multiplier is the shadow price of the link, and in many routing matrix with delay in the frequency domain as A(s) = algorithms it also summarizes the link congestion information. (Alr (s), l ∈ L, r ∈ R) , where Associate with each user r an aggregate route price qr deﬁned Alr (s) = f exp(−sτlr ) if l ∈ r , as qr := l∈r pl . Introduce the link price (column) vector p = = 0 otherwise. (pl , l ∈ L) and the route price (column) vector q = (qr , r ∈ R). Then we have q = AT p, and noting that pT y = pT Ax = Then we have the relationships [33] xT AT p = xT q, the Lagrangian can be rewritten as: Y (s) = A(s)X(s) , (10) L(x, p) = (Ur (xr ) − xr qr ) + pl c l . (3) Q(s) = diag(e−sTr )AT (−s)P (s) , (11) r∈R l∈L From the Kuhn-Tucker theorem [3] of nonlinear programming, and the primal algorithm with delay is given as follows [13]: ˆ ˆ x solves (1) if and only if there exists a p such that the pair User end: ˙ xr (t) = kr (wr − xr (t − Tr )qr (t)) . (12) x ˆ (ˆ, p) constitutes a saddle point for the function L(x, p), where x is the maximizer and p is the minimizer, with x ≥ 0 and Link end: pl (t) = fl (yl (t)) . (13) p ≥ 0. Making the natural assumption that the constraint set Conditions for local stability were ﬁrst conjectured in [13], and for x is not empty and is bounded (and thus compact), the strict then proved and extended in [32], [33], [34]. For a certain class concavity of the aggregate utility function leads to the existence of functions fl (·), global stability has been established in [37]. ˆ of a unique optimal solution x to (1). This then guarantees the Note that the link price computation in the primal algorithm ˆ ˆ existence of p, and thus the existence of q . The uniqueness of above is static. An algorithm called AVQ was introduced ˆ ˆ x determines the uniqueness of q . If A has full row rank, then in [20], where users choose the same dynamic adaptation, but ˆ p is uniquely determined from q, and thus p is also unique. A at the links, pl (t) is a function of both yl (t) and a virtual queue x ˆ pair (ˆ, p) is in saddle-point equilibrium if and only if [16] capacity cl which is a time-varying link parameter. The global Ur (ˆr ) = qr ∀r ∈ R , x ˆ (4) stability of this algorithm has been established in [23], [35] without delay, and local stability has been established in [22], ˆ ˆ ˆ ˆ pl (cl − yl ) = 0 and pl ≥ 0, and cl ≥ yl ∀l ∈ L . (5) [21] in the presence of feedback delays. 3 B. Dual Algorithm and = cl if pl > 0 , The dual algorithm introduced in [25] employs the following yl (19) ≤ cl if pl = 0 , algorithms at the link and user ends: for all l, r. So the equilibrium point solves the system problem Links: ˙ pl = γl (yl − cl ) , (14) (1) exactly. In [35], a passivity approach was used to show where γl > 0 is an adaptation parameter. In [30], γl was chosen global stability of this equilibrium without considering delay. to be 1/cl . In the next section, we will embed this algorithm within a more general class of primal-dual algorithms, and for this general −1 Users: xr = xr (qr ) = Ur (qr ) . (15) class we will establish local stability in the presence of delay. The above dual algorithm is a special case of the dual algorithm D. Remarks on the Link Model in [15], [14]. It was also derived in [36] for the special case As we have already stated, most of the AQM algorithms of logarithmic utility functions. choose the marking probability as a function of the queue For this dual algorithm, global stability was proved in [25], length. So it is intuitive to choose pl to be a function of while local stability was proved for a particular choice of the queue length bl , and not as a function of the aggregate utility functions in [29]. The restriction on the choice of utility link arrival rate yl . This is the model chosen by the dual and functions means that arbitrary relative fairness among the users primal-dual approaches. However, in the primal approach, pl cannot be achieved using the algorithm in [29]. Following the is chosen to be a function of yl . Does this then mean that time-scale decomposition idea in [23], [22], [21], to track an one has to measure the link arrival rate, and not the queue arbitrary utility function, the dual algorithm was modiﬁed in length, to compute the marking probability? In general, the [30] by introducing slow time-scale dynamics at the user end. answer seems to be no when there are stochastic disturbances In other words, xr = xr (ξr , qr ), where ξr is a slowly adjusted in the network. In [17], Kelly argues that if the queue length source parameter, just like the virtual queue capacity cl in the hits zero several times within a round-trip time, even if the AVQ algorithms. This ensures that the dual algorithm can also marking is queue-length-based, the source acts as though the allocate the network resources fairly. marking probability is a function of the stochastically averaged version of the queue length. Thus, in the deterministic model C. Primal-Dual Algorithm of congestion control, the source only sees the price pl as a Both the primal algorithms and the dual algorithms have static function of yl . However, the conditions under which the dynamics at one end (user end for the primal algorithm and link queue length hits zero frequently have not been provided in end for the dual algorithm) and static adaptation at the other. [17]. Recently, in [4], it has been shown that the parameters of However, in the current Internet, TCP is a dynamic source popular AQM schemes determine whether the queue length hits algorithm, and most AQM algorithms, like RED, Vegas, etc, zero frequently or not. Speciﬁcally, if both the queue length and relate the price to the queue length or queueing delay, and the capacity are scaled in proportion to the number of users thus are dynamic link algorithms. This motivates the work on in the network, and the marking probability is based on the primal-dual algorithms, to directly relate xr to qr and pl to yl . ˙ ˙ queue length divided by the number of users in the network, As already mentioned, x solves the system problem if and ˆ then we have a queue-based model; else, the queue length hits only if there exists a p such that (ˆ, p) is the saddle point of ˆ x ˆ zero frequently and we have the rate-based model. Thus, the L(x, p). So we can approach this optimization problem from a primal-dual algorithm considered here requires that the AQM game theoretical point of view: L(x, p) is the gain for player 1 parameters are chosen such that the queue length does not hit (P 1) who controls x, and loss for player 2 (P 2) who controls zero frequently. Since we also would like to have negligible p. So P 1 is the maximizer and P 2 is the minimizer, and we queueing delay, this suggests that the AQM algorithm based have a 2-player zero-sum game, with (ˆ, p) the unique saddle x ˆ on the primal-dual algorithm analysis must adjust its marking point: probability according to the occupancy of a virtual queue, whose capacity is slightly smaller than that of the real link ˆ x ˆ x L(x, p) ≤ L(ˆ, p) ≤ L(ˆ, p) ∀ x ≥ 0, p ≥ 0 . [9]. Thus, we can maintain a large virtual queue, which always To reach this unique saddle point equilibrium, we can choose stays nonzero, as well as a small real queue, yielding negligible iterative algorithms with dynamic adaptations from both play- queueing delay. We will discuss this further in later sections. ers. The simplest such algorithm is the gradient algorithm: III. A G ENERAL P RIMAL -D UAL A LGORITHM AND I TS ∂L(x, p) L OCAL A NALYSIS ˙ xr = kr = kr (Ur (xr ) − qr ) , (16) ∂xr The primal-dual algorithm (16)-(17) is too speciﬁc and still ∂L(x, p) far from being interpreted as a TCP-AQM combination. We ˙ pl = −γl = γl (yl − cl ) . (17) introduce here a general primal-dual algorithm, whose source ∂pl law includes current versions of TCP as special cases and Such algorithms were considered in [1] and [35], which can be whose link law includes most current AQM algorithms as ˙ regarded as primal-dual algorithms in the sense that xr depends special cases. Deﬁne [x]+ := max(0, x) and ˙ on qr and pl depends on yl . At equilibrium, we have: h(x) if x > 0 , −1 [h(x)]+ := xr = U r (qr ) , (18) x [h(x)]+ if x = 0 . 4 The general primal-dual algorithm is given by Let us now linearize (22). At the equilibrium point, we have + ˆ x qr = Ur (ˆr ). Linearization around it gives: xr = fr (xr ) Ur (xr ) − ˙ qr x , (20) r ˆ xr (t) = fr (Ur (ˆr )xr (t) − qr (t)) , ˙ x + ˙ pl = gl (pl ) yl − cl p , (21) ˆ l x ˆ where fr = fr (ˆr , xr ). Note that in the above equation and where fr (xr ) and gl (pl ) are continuous functions, lower in the rest of the paper, by slightly abusing notation, we use bounded by constants fr > 0 and gl > 0, respectively. xr , qr , yl , and pl also to represent the perturbations from their ˆ ˆ ˆ ˆ equilibrium values xr , qr , yl , and pl , respectively. A. Global Stability without Delay In the Laplace domain, we have It immediately follows that the equilibrium point of the ˆ ˆ x sXr (s) = −fr Qr (s) + fr Ur (ˆr )Xr (s) . general primal-dual algorithm (20)-(21) solves the system Solving for Xr (s), yields: problem exactly. Using an approach similar to the one in [35], one can show that the general primal-dual algorithm without ˆ 1 1 θr Xr (s) = −fr Qr (s) = − Qr (s) , delay is globally asymptotically stable (GAS). Deﬁne1 ˆ s − fr Ur (ˆr ) x x ur (ˆr ) sTr + θr xr (24) ˆ u − xr Hr (xr ) := du , where xr ˆ fr (u) ˆ x θr = fr Tr (−Ur (ˆr )) . (25) pL ˆ v − pl Note that θr > 0, since Ur (xr ) is strictly concave and thus Jl (pl ) := dv , ˆ pl gl (v) Ur (xr ) < 0. and introduce For the link dynamics, linearization yields: V (x, p) = Hr (xr ) + Jl (pl ) . ˙ p pl = gl (ˆl )yl . r∈R l∈L In the Laplace domain, we have: Using these, we can establish the following theorem on p gl (ˆl ) global stability of the system; see [31] for a proof. Pl (s) = Yl (s) . (26) s Theorem III.1. V (x, p) is a Lyapunov function for the sys- The return ratio of the closed-loop feedback system (10), tem described by (20)-(21), and thus the system is globally (11), (24) and (26) is given by asymptotically stable. 1 θr e−sTr p gl (ˆl ) L(s) = diag AT (−s)diag A(s) . −Ur (ˆr ) sTr + θr x s B. Local Stability Analysis with Delay (27) With feedback delay, we can modify (20)-(21) to: Notice that we can rewrite L(s) as: xr (t) = fr (xr (t), xr (t − Tr ))[Ur (xr (t)) − qr (t)]+r , ˙ (22) Tr θr e−sTr x L(s) = diag −Ur (ˆr ) sTr (sTr + θr ) x (28) pl (t) = gl (pl (t))[yl (t) − cl ]+ , ˙ pl (23) ×AT (−s)diag(gl (ˆl ))A(s) . p where, by slightly abusing notation, we allow fr to be a This now leads to the following theorem on stability, whose continuous function of both xr (t) and xr (t − Tr ), which is proof can be found in the Appendix. again lower bounded by the constant fr > 0. Theorem III.2. The closed-loop system described by (22), Before we proceed with the local analysis by linearizing (23), (8), and (9) is locally asymptotically stable around the the algorithm around the equilibrium values, we ﬁrst note the equilibrium point if the following two conditions hold: following on the boundary conditions in (22) and (23) and the constant delay assumption in the system modelling. For ˆ qr the users, xr > 0, ∀r ∈ R, since Ur (xr ) → ∞ as x → 0. ˆ x −Ur (ˆ) ≥ , (29) ˆ ar x r For the links, we restrict the link set L to the active link set ˆ La := {l ∈ L : pl > 0}, i.e., we consider only the bottleneck p gl (ˆl ) 1 1 ≤ , (30) links. Since the set L \ La does not affect the local analysis, ˆ pl 2cl al Tm for the rest of this paper we will assume that L = La . Then, where ar is a positive constant for each r, and al = the boundary conditions in (22) and (23) can be ignored for maxr∈l ar , Tm = maxr∈R Tr . local analysis. Also, we assume that the queueing delay can Remark 1. Condition (29) is satisﬁed for the general class of be neglected when compared with propagation delay, and thus utility functions introduced in [28]. For example, if Ur (xr ) is of the round trip time can be taken to be a constant. This is a the form wr log xr (logarithmic utility function), we have ar = reasonable assumption if one assumes that each router uses a 1. If Ur (xr ) is of the form −wr /xnr (power utility function of r virtual queue to compute link prices [9]. order nr ), we have ar = 1/(1 + nr ). 1 The deﬁnitions of H (x ) and J (p ) were suggested by Murat Arcak r r l l in his correspondence with John Wen and the third author of this paper in a Remark 2. From (28) and the proof of Theorem III.2 in the different context. Appendix, we see that the sufﬁcient condition for local stability 5 (l.s.) depends only on the link price adaptation speed gl (pl ) and mr ir −dr i and the user utility function Ur (xr ), and not on the source Ur (xr (t)) = x (t)Tr r −dr . nr r rate adaptation speed fr (xr (t), xr (t − Tr )). If we adapt the source rate inﬁnitely fast (i.e., choosing fr (·) ≡ κr and letting Thus, κr → ∞), we arrive at the dual algorithm. The corresponding ˆ ir − d r d fr = nr xdr +1 Tr r −1 ˆr and Ur (xr (t)) = Ur (xr (t)) sufﬁcient condition for stability of the dual algorithm also xr (t) depends only on the link adaptation speed and the user utility So, we have function. If gl (pl ) is ﬁxed to be 1/cl as in the dual algorithm [25], then the utility function must be speciﬁed as in [29]. To q (dr − ir )ˆr 1 x Ur (ˆr ) = − and ar = . allow arbitrary utility functions, we could either add a slow ˆ xr dr − i r adaptation at the source [30] or choose a different gl (pl ). In particular, TCP-Reno and HighSpeed TCP correspond to a Thus, the interesting observation from our analysis is that slow power utility function of order 1 and ar = 1/2, while Scalable- adaptation at the source (as in [30]) is not necessary for the TCP corresponds to a logarithmic utility function and ar = 1. dual algorithm to be stable, and achieve full utilization and arbitrary fairness. The dual algorithm can also be stabilized B. Exponential RED (E-RED) with slow adaptation at the links since the l.s. result does not From Theorem III.2, we know that to stabilize the window depend on fr (xr ). adaptation source law in the TCP family, the link dynamic IV. I MPLEMENTATION IN THE I NTERNET AND E-RED adaptation should satisfy AQM S CHEME p gl (ˆl ) 1 ≤ . (33) In the current Internet, the source law is TCP and the link ˆ pl 2al Tm cl law is an AQM scheme, such as DropTail or RED. We wish Consider the link dynamics to make as little changes to the current protocols as possible, pl but still achieve guaranteed stability and better performance. In ˙ pl = max(gl , βl )[yl − cl ]+ , pl (34) cl this section we utilize the local stability analysis for the primal- with βl < 1/(2al Tm ). It satisﬁes the stability condition (33) dual algorithm to modify TCP+RED such that the combination ˆ ˆ if pl is not close to zero (more speciﬁcally, pl > gl cl /βl ). is locally stable. Throughout this paper, we assume that the above condition on ˆ pl is satisﬁed. Equation (34) can be implemented at the packet A. General Form of TCP level by setting the packet marking probability (the link price) Consider the general form of TCP introduced in [33], which pl as an exponential function of the virtual queue length bl : contains all possible TCP variants. Let Wr represent the window size of source r. Suppose the window size increases 0 if 0 ≤ bl < thmin,l , βl i pl = (bl −thmin,l ) by mr Wr r for each unmarked acknowledgement and decreases pmin,l e if thmin,l < bl < thmax,l , cl dr by nr Wr for each marked acknowledgement, where mr , nr , 1 if bl ≥ thmax,l , ir and dr are constants, with mr and nr being positive and (35) ir < dr being both integers. We note that the choice of ir = −1 where thmin,l < thmax,l are the two queue length thresholds and dr = 1 corresponds to TCP-Reno (mr = 1, nr = 2/3 for between which the exponential marking is selected and pmin,l the standard version), and the choice of ir = 0 and dr = 1 is the marking threshold when bl = thmin,l . Let pmax,l denote corresponds to the Scalable TCP in [18]. the marking probability when bl = thmax,l ; then, thmin,l , This window based adaptation can be written in differential thmax,l , and pmin,l must be such that pmax,l < 1. If pmin,l and equation form as ˆ pmax,l are selected properly such that pmin,l < pl < pmax,l , ˙ i d then between the two probability thresholds, the derivative of Wr = xr (t − Tr )(mr Wr r (t) − nr Wr r (t)qr (t)) . (31) pl in (35) is just (34) and the linearization satisﬁes (33). Then, We note here the implicit assumption that qr is sufﬁciently this marking scheme locally stabilizes TCP given that the ﬂuid small so that we do not distinguish between 1 and 1 − qr (t). model of the system dynamics is valid. We call the above This is widely accepted in the modelling of TCP ([19], [27]). marking scheme Exponential-RED or E-RED in short, since The source rate is xr = Wr /Tr , so we have: the marking proﬁle is an exponential function of the queue xr (t − Tr ) length. We note that the form of (35) is similar to the fair dual xr ˙ = mr Tr r xir (t) − nr Tr r xdr (t)qr (t) i r d r algorithm in [14]. However, the source control laws are static Tr = n r Tr dr −1 dr xr (t − Tr )xr (t) in the fair dual algorithm, and hence the stability conditions in ×( mr xir −dr (t)Tr r −dr − qr (t)) . r i [14] are different than the ones in this paper. n r (32) In the TCP-AQM scheme, the prices pl and qr are expressed Comparing this with the source law (22) of the primal-dual al- in terms of the marking probabilities, which is why we use gorithm, and considering the fact that fr (xr ) is lower bounded pl , qr to represent both the prices and the marking probabilities. by fr > 0 and that (32) represents the dynamics of TCP only However, the price feedback from pl to qr is linear in (9). while when xr is not close to zero, we have: the probabilistic feedback is actually not linear: d fr (xr (t), xr (t − Tr )) = max(fr , nr xr (t)xdr (t − Tr )Tr r −1 ) , r qr = 1 − (1 − pl ) . (36) 6 We notice, however, that if pl and qr are sufﬁciently small for ing loop function: all l ∈ L, r ∈ R, we can approximate (36) by (9). In this Tr θr e−sTr and the next subsection, we assume that the small probability L(s) = diag AT (−s) condition holds; subsequently in subsection IV-D, we consider x −U (ˆr ) Tr (sTr + θr ) ˆ βl p l κl the stability issue for the general case. ×diag A(s) cl s(s + κl ) Given the linear feedback from pl to qr , from Theorem III.2, Tr θr e−sTr the condition βl < 1/(2al Tm ) guarantees the local stability of = diag AT (−s) E-RED combined with any window adaptation source law in x −U (ˆr ) sTr (sTr + θr ) β l p l κl ˆ the TCP family. Note that E-RED simply uses a marking proﬁle ×diag A(s) different than RED, and employs a virtual queue. cl s + κ l (41) Remark 3. The slow link adaptation makes the queue length This now leads to the following theorem on the stability very large. To maintain a small queue length, so that the condition for the source algorithm (20) and E-RED-aq: queueing delay is negligible and thus the RTT can be modeled Theorem IV.1. The closed-loop system composed of the source as a constant, we need to implement the E-RED algorithm in algorithm (20) and E-RED-aq is locally asymptotically stable ˜ a virtual queue whose capacity is cl = γl cl , where 0 < γl < 1 around the equilibrium if the two conditions in (29) and (30) is the link utilization parameter. Then, bl is the queue length are satisﬁed. in that virtual queue, and we can have a large bl while maintaining a small real queue. Proof: See the Appendix. This theorem suggests that E-RED-aq requires the same link adaptation as E-RED-iq, so the averaging of the queue length C. E-RED with the Average Queue Length as in RED is not detrimental to stability. In RED, the marking probability is a linear function of the average queue length. In E-RED, we can also take the marking D. Probabilistic Feedback probability to be an exponential function of the average value In this subsection, we will show that even under exact of the virtual queue length. We call this variant E-RED-aq and probabilistic feedback, the local stability results still hold. We the original E-RED without averaging E-RED-iq (iq stands for introduce two pseudo price variables vr and zl , deﬁned as: instantaneous queue length). vr = − log(1 − qr ) , zl = − log(1 − pl ) . For this variant, we replace bl with rl in the marking equation (35), where rl is the exponentially weighted moving Then (36) leads to the linearized relationship: average of bl based on samples taken every δl seconds. Thus, b vr (t) = zl (t − τlr ) . (42) rl ((k + 1)δl ) = r(kδl )(1 − αl ) + αl bl , (37) l∈r Let v and z be the column vectors of all vr and zl , respectively, where 0 < αl < 1 is the weight. As in [27], we use the and let V (s) and Z(s) be Laplace transforms of v(t) and z(t). following continuous approximation to (37): Then, we have a relationship similar to (9): rl = −κl (rl − bl ) , ˙ (38) V (s) = diag(e−sTr )AT (−s)Z(s) . (43) where κl = − log(1 − αl )/δl . In the Laplace domain, we have: Comparing (28) with (41) and noticing that for E-RED, gl (pl ) = βl pl /cl , we see that the loop function of either E- κl RED-iq or E-RED-aq combined with (20) can be written as r(s) = yl (s) . s(s + κl ) ˆ xr ˜ ˆ pl L(s) = diag fr (s)e−sTr AT (−s)diag ˜ gl (s) A(s) , The dynamics of E-RED-aq yields ˆ qr cl (44) βl ˜ ˜ where fr (s) and gl (s) are two functions of s. The stability ˙ pl = p l ˙ rl . cl results build on the fact that the spectral radius of the matrix ˆl diag( pl )R(s)diag( xr ) is bounded by 1, and thus we can y ˆ ˆr qˆ ˙ Since at equilibrium rl = 0, linearization yields: select the parameters such that the eigenloci of L cross the real βl p l ˆ line to the right of −1. Then, the stability condition is obtained ˙ pl = ˙ rl , (39) from the generalized Nyquist Criterion. cl Now, for the probabilistic feedback, with the relationships and in the Laplace domain, we have: (10) and (43), we have: βl p l ˆ κl xr ˜ ˆ xr ˜ ˆ ∂qr Xr (s) = qr fr (s)Qr (s) qr fr (s) ∂vr Vr (s) Pl (s) = Yl (s) . (40) ˆ xr ˜ ˆ ˆ cl s(s + κl ) = ˆ qr fr (s)(1 − qr )Vr (s) , ˆ Combining the source algorithm (20) with logarithmic or ∂zl 1 1 ˆ pl Zl (s) = Pl (s) = Pl (s) = ˜ gl (s)Yl (s) . power utility function and E-RED-aq, we arrive at the follow- ∂pl ˆ (1 − pl ) ˆ (1 − pl ) cl 7 S1 D1 So the new loop function is in the following form: d 11 d12 S2 D2 L(s) = ˆ q ˜ diag xr (1−ˆr ) fr (s)e−sTr AT (−s) R1 R2 D3 qr ˆ (45) S3 d ˆ pl ×diag cl (1−pl ) gl (s) A(s) . ˆ ˜ Thus for the probabilistic feedback law (42), the same stability SN d N1 dN2 DN conditions as those in the price feedback law (9) are obtained if we could prove that the spectral radius of the matrix Fig. 1. The network topology: a single bottleneck link and multiple ﬂows with diag( cl (1−pl ) )A(s)diag( xr (1−ˆr ) ) is also bounded by 1. ˆ pl ˆ ˆ qr ˆ q heterogeneous delays ˆ Since r:r∈l xr ≤ cl , ∀l, we just need to show that pl qr ≤ (46) 1 − pl 1 − qr that RED becomes unstable when the capacity or the round l∈r trip delay becomes large. In the ﬁrst simulation scenario, Theorem IV.2. The inequality (46) holds, and thus the stability we choose a large link capacity network with c = 1 Gbps, results for the linear price feedback case also hold under N = 2000, d = 10ms, and the delays of all other links probabilistic feedback. being uniformly distributed between 1 and 20 ms. So the Proof: See the Appendix. maximum round trip time is Tm = 100ms. For the E-RED parameters, we set pmin,l = 0.0005, pmax,l = 0.1, γ = 0.95, V. S IMULATIONS ξ = 1. Even though our sufﬁcient stability condition suggests ξ ≤ 1/2, we have chosen ξ = 1 in order to understand how We implemented the E-RED algorithm in the ns-2 package conservative our stability results are. For RED, we monitor and performed simulations on a network whose sources use the instantaneous queue length, and measure the throughput TCP-Reno, and whose routers choose E-RED-iq, E-RED-aq or with a time granularity of 0.1s; they are shown in Fig. 2. RED. Let ξl := βl Tm /2; then, ξl ≤ 1/2 is the local stability From the graphs, we see that RED is unstable in the sense condition for E-RED-iq with TCP-Reno. We make all sources that both the queue length and the throughput show signiﬁcant ECN-capable, and set the maximum window size to be 1000 oscillations, and the queue length hits zero and the full buffer instead of 64, so the maximum window size will not be a size frequently. For E-RED, we monitor the real queue length threshold for the ﬂow rate in all scenarios. We also let all and the virtual queue length, and measure the throughput with sources send packets of identical size, 1040 bytes, including the same time granularity; they are shown in Fig. 3. From the the IP and TCP header. For E-RED, we choose the parameters graphs, we see that E-RED is stable in the sense that the virtual ξl , pmin,l , pmax,l , thmin,l , γl , and αl (αl = 1 corresponds to queue length oscillates around its equilibrium very slightly, the E-RED-iq and αl < 1 corresponds to E-RED-aq), and compute throughput is less oscillatory than that of RED, and the real thmax,l from pmin,l , pmax,l , thmin,l . queue size is always small after the transient phase. So E- For most simulations in this section, we consider a network RED stabilizes TCP-Reno in large capacity scenarios while with a single bottleneck link and multiple ﬂows, as shown in RED does not. To compare the performances numerically, Fig. 1. Since there is a large population of users in the Internet we measure the average queue length (Avg(qlen)), standard and the ﬂuid model makes sense only when the number of deviation of queue length (Std(qlen)), and average throughput users is large, we choose a sufﬁciently large number of users (Avg(thr)) for both algorithms after the transient phase (over for most of the simulations. We vary the values of N , c, d, the interval between 10s and 39s for this scenario); these values d11 to dN 2 to obtain different network conﬁgurations. We set are listed in Table I. In the second scenario, we choose a large the buffer limit (of the real queue) always to be max(0.3c, 25) round trip delay network with c = 300 Mbps, N = 1000, packets, where c is measured in Mbps, which guarantees that d = 10ms, and the delays of all other links being uniformly the bottleneck queueing delay for sufﬁciently large capacity distributed between 45 and 95 ms. So the maximum round trip is bounded by 2.5ms (actually, we will see that if E-RED time is Tm = 400ms. We set pmin,l = 0.00067, pmax,l = 0.1, stabilizes the system, then the real queue length is much γ = 0.95, ξ = 1. The queue length and throughput for RED smaller than its buffer limit and thus the queueing delay is are shown in Fig. 4, and we see that RED is unstable. The much smaller than this bound). As for the parameter settings, real queue length, virtual queue length and throughput for E- for RED, we set thresh to be 1/5 of the buffer limit and RED are shown in Fig. 5; we see that E-RED is stable and maxthresh to be 3 ∗ thresh, keep all the other parameters achieves a higher and less oscillatory throughput and lower and at their default settings, and choose ECN marking instead of less oscillatory queueing delay than RED. For the numerical dropping; for E-RED, we set thmin,l to be 1/5 of the buffer comparison, the average throughput, average queue length, and limit (of the real queue). The other parameters for E-RED will standard deviation of queue length (measured after the transient be varied for different scenarios. phase) for both algorithms are listed in Table I. So for both large capacity and large delay networks, given A. Stability and Performance Comparison of RED and E-RED that the number of users is sufﬁciently large, E-RED stabilizes TCP-Reno while RED does not. Furthermore, E-RED achieves We ﬁrst compare the stability and performances of E-RED higher bandwidth utilization and smaller queueing delay. and RED. From the analysis in [11] and [26], we know One may argue that the advantage of E-RED over RED is 8 Queue Length (packets) Vs Time (seconds) Queue Length (packets) Vs Time (seconds) 300 90 80 250 70 200 60 queue length (packets) queue length (packets) 50 150 40 100 30 20 50 10 0 0 0 5 10 15 20 25 30 35 40 0 10 20 30 40 50 60 70 80 time (seconds) time (seconds) Throughput (bps) Vs Time (seconds) Throughput (bps) Vs Time (seconds) 1.2e+09 3.5e+08 3e+08 1e+09 2.5e+08 8e+08 Throughput (bps) Throughput (bps) 2e+08 6e+08 1.5e+08 4e+08 1e+08 2e+08 5e+07 0 0 0 5 10 15 20 25 30 35 40 0 10 20 30 40 50 60 70 80 time (seconds) time (seconds) Fig. 4. RED in a large delay network: instantaneous queue length (top graph) Fig. 2. RED in a large capacity network: instantaneous queue length (top and throughput (bottom graph). graph), and throughput (bottom graph). Real Queue Length (packets) Vs Time (seconds) Real Queue Length (packets) Vs Time (seconds) 300 90 ’vqresult’ using 1:2 80 250 70 real queue length (packets) real queue length (packets) 200 60 50 150 40 100 30 20 50 10 0 0 0 5 10 15 20 25 30 35 40 0 20 40 60 80 100 120 140 160 180 200 time (seconds) time (seconds) Virtual Queue Length (packets) Vs Time (seconds) Virtual Queue Length (packets) Vs Time (seconds) 30000 45000 ’vqresult’ using 1:3 40000 25000 35000 virtual queue length (packets) virtual queue length (packets) 20000 30000 25000 15000 20000 10000 15000 10000 5000 5000 0 0 0 5 10 15 20 25 30 35 40 0 20 40 60 80 100 120 140 160 180 200 time (seconds) time (seconds) Throughtput (bps) Vs Time (seconds) Throughtput (bps) Vs Time (seconds) 1.2e+09 3.5e+08 ’vqthroughput’ using 1:2 3e+08 1e+09 2.5e+08 8e+08 throughput (bps) throughput (bps) 2e+08 6e+08 1.5e+08 4e+08 1e+08 2e+08 5e+07 0 0 0 5 10 15 20 25 30 35 40 0 20 40 60 80 100 120 140 160 180 200 time (seconds) time (seconds) Fig. 5. E-RED in a large delay network: real queue length (top graph), virtual Fig. 3. E-RED in a large capacity network: real queue length (top graph), queue length (middle graph), and throughput (bottom graph). virtual queue length (middle graph), and throughput (bottom graph). 9 TABLE I Virtual Queue Length (packets) Vs Time (seconds): alpha=0.001 70000 N UMERICAL COMPARISON OF RED AND E-RED IN LARGE CAPACITY 60000 (LC) AND LARGE DELAY (LD) NETWORK SCENARIOS 50000 virtual queue length (packets) AVQ/Scenario Avg(qlen) (pkts) Std(qlen) (pkts) Avg(thr) (Mbps) 40000 RED/LC 117 131 939 E-RED/LC 9.74 11.8 951(≈ γc) 30000 RED/LD 27.7 34.3 278 20000 E-RED/LD 11.8 16.2 285(≈ γc) 10000 0 0 10 20 30 40 50 60 70 80 time (seconds) mainly due to the use of a virtual queue. We have performed 45000 Virtual Queue Length (packets) Vs Time (seconds): alpha=0.001 simulations with RED in a virtual queue (REDvq) and found 40000 that REDvq is unstable if the virtual queue limit and the REDvq 35000 virtual queue length (packets) parameters are taken to be the same as those of RED in the real 30000 queue. The reason for the instability is that the link adaptation 25000 20000 speed is much larger than what Theorem III.2 suggests. From 15000 Fig. 3, we see that at equilibrium, bl ≈ 27000 packets. Since 10000 we have chosen ξ = 1, we get 5000 0 2(27000 − 60) × 8320 0 50 100 150 200 time (seconds) ˆ pl = 0.0005 exp( ) = 0.0442 , 0.1 × 109 Fig. 6. The virtual queue of E-RED-aq in large capacity (top graph) and large delay networks (bottom graph): αl = 0.001. and the following stability condition on link adaptation speed: 2 × 0.0442 × 8320 gl (ˆl ) ≤ p = 7.36 × 10−6 C. Sudden Trafﬁc Changes 0.1 × 109 To study the transient response of the proposed algorithm, For RED, the default setting for pmax is 0.1 and this requires we consider a scenario where a large trafﬁc source suddenly joins the network and leaves after a while. We choose c = 0.1 maxthresh − thresh ≥ = 13584 , 100Mbps and N = 200, and the round trip delays to be from 7.36 × 10−6 20ms to 100ms. One UDP ﬂow, with ﬁxed rate of c/3 joins at and hence a much larger virtual queue length. If we increase 50 and leaves at 70. The virtual queue is shown in Fig. 7. We the virtual queue limit 120 fold so that the difference between see that the transient response is fast. the two thresholds is now 14400 (which is larger than 13584), Virtual Queue Length (packets) Vs Time (seconds): UDP In and Out then the simulations (not shown here due to space limitations) 7000 indicate that the system is stable under these new parameter 6000 settings. So both linear and exponential marking proﬁles can 5000 virtual queue length (packets) stabilize the system if the adaptation speed is chosen according 4000 to the condition in Theorem III.2. However, for linear proﬁle 3000 to satisfy the stability condition, we need the knowledge of ˆ pl , while E-RED does not require that. This auto-conﬁguration 2000 is one of the reasons for choosing the exponential marking 1000 proﬁle. 0 0 10 20 30 40 50 60 70 80 90 time (seconds) Remark 4. From the above simulation results, we see that once the virtual queue becomes stable (oscillates around its Fig. 7. UDP trafﬁc in and out, virtual queue equilibrium point slightly), the system will become stable, and the real queue size will be small and the throughput will oscillate around γl cl slightly. In the following simulations, to D. Dependence of Stability on the Parameters: ξ, γ and pmin,l . save space, we provide the graphs of only the virtual queues for the E-RED algorithm. We then study the inﬂuence of the parameters on stability and performance. We have shown that ξ = 1 also stabilizes the system while ξ ≤ 1/2 is suggested in Theorem III.2. To further B. E-RED-iq and E-RED-aq test how conservative the condition is, we have chosen ξ to be 2, 4 and 8. The network parameters are chosen as c = 200 We then test the stability and performance of E-RED-aq. We Mbps, N = 200, and RT T between 20 and 100 ms. The E- still choose the topologies as in the previous subsection, and RED parameters are chosen as pmin,l = 0.0005, pmax,l = 0.1, choose αl = 0.001 and ξ = 0.9. The virtual queue size of the γ = 0.9. We have plotted the virtual queue lengths for the two network scenarios are shown in Fig. 6. From the graphs, three ξ values in Fig. 8. We see that as ξ increases, the virtual we see that E-RED-aq is also stable with TCP-Reno. queue shows higher oscillation, and when ξ = 8, the system is 10 unstable. So this simulation shows that the slow link adaptation E. TCP-Reno, HighSpeed-TCP, and Scalable-TCP condition is not very conservative. We studied the combination of E-RED with two pop- ular TCP variants for high-speed networks: HighSpeed- Virtual Queue Length (packets) Vs Time (seconds): beta=2 TCP [8] and Scalable-TCP [18]. We performed ns-2 simulation 5000 for HighSpeed-TCP, and performed Matlab simulation for 4500 4000 Scalable-TCP since it is not included in the current ns-2 package. For both simulations, the scenario is the following: virtual queue length (packets) 3500 3000 N = 3, the capacity per user is 10 Mbps, and the round trip 2500 delay for each user is 200ms. The window behavior of the 2000 1500 three users under HighSpeed TCP is shown in Fig. 9. The 1000 average throughput per user measured over 75 seconds is 0.8c. 500 For a small number of sources, the use of ﬂuid models is 0 0 5 10 15 20 time (seconds) 25 30 35 40 questionable since there are not enough sources for the law of 3000 Virtual Queue Length (packets) Vs Time (seconds): beta=4 large numbers to hold [31]. Despite this, the combination of E- RED with HighSpeed TCP gives a reasonably high throughput. 2500 The fact that the ﬂuid model does not apply was noticed in the simulations from the behavior of the queue length. virtual queue length (packets) 2000 1500 The queue length does not converge to its equilibrium value. However, simulations (not shown here) have indicated that if 1000 N is sufﬁciently large, the system converges to its equilibrium 500 under E-RED. 0 Window Size (packets) Vs Time (seconds) 0 5 10 15 20 25 30 35 40 500 time (seconds) window size 1 window size 2 Virtual Queue Length (packets) Vs Time (seconds): beta=8 450 window size 3 1400 400 1200 350 window size (packets) 300 1000 virtual queue length (packets) 250 800 200 600 150 100 400 50 200 0 0 10 20 30 40 50 60 70 time (seconds) 0 0 5 10 15 20 25 30 35 40 time (seconds) Fig. 9. The window sizes of the 3 HS-TCP ﬂows with E-RED. Fig. 8. The virtual queue length of E-RED, ξ = 2, 4, 8 (in top, middle and bottom graphs respectively). The Matlab simulation for Scalable-TCP shows that if ξ = 1, both the source rates (shown in Fig. 10) and the link marking In simulations not shown here, we have also studied the probability (not shown here) oscillate, without a trend of either inﬂuence of γ on the system performance. We studied two divergence or convergence. If ξ = 0.9, all rates and probability scenarios, one with c = 150 Mbps and N = 100, and another converge. These results suggest that ξ = 1 is the critical point one with the same c but N = 250. For the ﬁrst scenario, we between stability and instability. This Matlab simulation also have seen that the system is unstable for γ > 0.93, and is supports our earlier claim that the slow adaptation condition stable for γ ≤ 0.93. For the second scenario, we have seen in Theorem III.2 is not only sufﬁcient but also very close to that the system is unstable for γ > 0.97, and is stable for being necessary. γ ≤ 0.97. For both scenarios, once γ is small enough for the system to be stable, the oscillation level does not change much F. Multi-link Network Simulations as γ decreases. Thus we know that for a ﬁxed network scenario, All the simulations above were for a single-link network. there is a threshold value of γ, under which the system is stable. Our theoretical analysis applies to a general topology network This threshold value can be larger if the number of sources is and in this subsection we will examine the performance of E- increased while keeping the per ﬂow throughput constant. This RED in a multi-link network. The network is composed of 6 is to be expected since what we have observed is essentially a routers and 500 source-destination pairs (users), as shown in statistical multiplexing effect. Fig. 11. The 500 ﬂows from the 500 users are all FTP trafﬁc In simulations not shown here, we have also seen that and they go along 5 different paths: R1 − R2 − R4 − R5 (L1), ˆ if pmin,l is smaller than the equilibrium value p, the value R3 − R2 − R4 − R6 (L2), R1 − R2 − R3 (L3), R6 − R4 − R5 of pmin,l only changes the initial setup time, and does not (L4), and R6 − R4 − R2 − R3 (L5), with each path containing inﬂuence the stability and transient response. However, if 100 ﬂows. All the inter-router links have a capacity of 100 ˆ pmin,l > p, we will never reach the operating point and the Mbps and a delay of 10 ms; and all the user-router links have system cannot be stable. a capacity of 10 Mbps (large enough for these links not to be 11 scalable TCP: rates Virtual Queue Length (packets) Vs Time (seconds) 2500 7000 6000 2000 5000 virtual queue length (packets) 1500 4000 x(t) 3000 1000 2000 500 1000 0 0 0 20 40 60 80 100 120 140 160 180 200 0 5 10 15 20 25 30 35 40 time t time (seconds) scalable TCP: rates Throughput (bps) Vs Time (seconds) 2500 1.4e+08 1.2e+08 2000 1e+08 Throughput (bps) 1500 8e+07 x(t) 6e+07 1000 4e+07 500 2e+07 0 0 0 20 40 60 80 100 120 140 160 180 200 0 5 10 15 20 25 30 35 40 time t time (seconds) Fig. 12. The virtual queue length (top graph) and throughput (bottom graph) Fig. 10. Source rates for Scalable TCP+E-RED: ξ = 1 (top graph) and ξ = 0.9 of the link between R2 and R4 in a multilink network. (bottom graph). • The choice of pmin,l does not inﬂuence the stability and bottlenecked) and a delay uniformly distributed between 0 ms ˆ transient response, given that pmin,l < p, and 10 ms. We plot the virtual queue length and throughput of • If ξ > 1/2, the system might be unstable. If ξ ≤ 1/2 and the link between R2 and R4 in Fig. 12; those of the other inter- the system is unstable because N is not large enough, the router links show similar behavior. From the graphs, we see system cannot be stabilized by making ξ even smaller. that E-RED stabilizes TCP-Reno for this multi-link multi-path This suggests that the large oscillations for small N multi-ﬂow network and achieves satisfactory performance. are due to the TCP’s AIMD behavior and cannot be eliminated through slow adaptation speed. R1 R5 VI. C ONCLUSION L1 path 1: flow 1-100 L4 In this paper, we have introduced a general class of primal- path 3: path 4: flow L3 flow dual algorithms and the E-RED algorithm. This class of primal- 201- R2 R4 300 301- 400 dual algorithms has been shown to be locally stable in the path 2: flow 101-200 L5 presence of delay. The E-RED algorithm has been shown to L2 path 5: flow 401-500 R3 stabilize TCP-Reno and its high speed variants, both analyt- R6 ically as well as through simulations. The simulations were performed under different scenarios and different parameter Fig. 11. The multilink network topology. choices. These simulations clearly show that E-RED outper- forms RED with larger throughput and lower queueing delay. They also suggest guidelines for tuning the E-RED parameters. G. Summary of the Simulations VII. A PPENDIX We summarize here our results from the simulations: In this appendix we provide proofs for Theorems III.2, IV.1, • For the general network with heterogeneous delays, E- and IV.2. First, we present the following lemmas as preliminary RED stabilizes TCP-Reno, even when the capacity or the to the proof of Theorem III.2 and Theorem IV.1: round trip delays are very large, given the condition that the number of ﬂows is sufﬁciently large. Lemma VII.1. If θ1 > 0, θ2 > 0, T1 > 0, T2 > 0, Tm = • E-RED-aq also stabilizes TCP-Reno, by reducing the link max(T1 , T2 ), λ1 ≥ 0, λ2 ≥ 0, λ1 + λ2 ≤ 1, then adaptation speed (Theorem IV.1). T1 θ1 e−jωT1 T2 θ2 e−jωT2 • The transient response is fast when there are sudden f (ω) = λ1 + λ2 Tm jωT1 (jωT1 + θ1 ) Tm jωT2 (jωT2 + θ2 ) changes in the available bandwidth. crosses the real line at the right of −2 as ω varies from −∞ • For a ﬁxed network scenario, there is a threshold value of to ∞. γ to guarantee stability, and for most realistic scenarios, this value of γ could be between 0.9 and 0.95. As N Proof. It is easy to see that f (−ω) = f ∗ (ω), where f ∗ (ω) is increases, this value of γ can be made very close to 1.0. the convex conjugate of f (ω), so we only need to consider the 12 case of ω > 0. Without loss of generality, we assume T1 ≤ T2 , If φ2 + x2 > φ1 , then φ2 > φ1 − x2 > 0, since φ1 ≥ π/2 − then Tm = T2 . Let x1 = ωT1 , x2 = ωT2 , then x2 ≥ x1 > 0, 1/2 > 1/2 ≥ x2 . So cos φ2 < cos(φ1 − x2 ) = cos φ1 cos x2 + T1 T2 and Tm = x1 , Tm = 1. Deﬁne x2 sin φ1 sin x2 and thus we have 1 θi e−jxi 1 |R| = x2 [λ1 cos φ1 sin(φ1 + x1 ) + λ2 cos φ2 sin(φ2 + x2 )] Ci (xi ) := = cos φi e−jxi −φi −π/2 , ≤ 1 jxi (jxi + θi ) xi x2 [λ1 cos φ1 sin(φ1 + x1 ) +λ2 (cos φ1 cos x2 + sin φ1 sin x2 ) sin(φ2 + x2 )] where φi = arctan(xi /θi ) ∈ (0, π/2). Then, f (ω) = 1 ≤ x2 [λ1 cos φ1 + λ2 (cos φ1 + sin x2 )] x1 λ1 x2 C1 (x1 ) + λ2 C2 (x2 ). Let R := Re(f (ω)) and I := < 1 x2 [λ1 x2 + λ2 (x2 + x2 )] ≤ 2 . Im(f (ω)). Then, So under case 2, we still have R > −2. Then, we have λ1 cos φ1 sin(φ1 + x1 ) + λ2 cos φ2 sin(φ2 + x2 ) proved that R > −2 if I = 0 and thus the curve crosses the R=− , x2 real line to the right of −2. and Lemma VII.2. If λi ≥ 0, θi > 0, Ti > 0, ∀i = 1, 2, 3 and λ1 cos φ1 cos(φ1 + x1 ) + λ2 cos φ2 cos(φ2 + x2 ) Tm = max(T1 , T2 , T3 ), λ1 + λ2 + λ3 = 1, then I=− . x2 3 3 Ti θi e−jωTi f (ω) = λi gi (ω) = λi To show that the curve crosses the real line to the right of Tm jωTi (jωTi + θi ) i=1 i=1 −2, it is sufﬁcient to show that R > −2 if I = 0. Since R > −2 if x2 > 1/2, we only need to consider the case of crosses the real line to the right of −2 as ω varies from −∞ x1 ≤ x2 ≤ 1/2. From I = 0, we get to ∞. Proof. Supppose f (ω) crosses the real line to the left of λ1 cos φ1 cos(φ1 + x1 ) = λ2 cos φ2 (− cos(φ2 + x2 )) −2; then there exists an ω ∗ such that Im(f (ω ∗ )) = 0 and 3 We consider the following two cases: Re(f (ω ∗ )) ≤ −2. If s := i=1 λi < 1, then we can pick 3 Case 1: cos(φ1 + x1 ) > 0 and cos(φ2 + x2 ) < 0, which λi = λi /s, and the new combination f (ω ∗ ) = i=1 λi gi (ω ∗ ) lead to π > φ2 + x2 > π/2 > φ1 + x1 > 0. Then, we have also lies on the real line to the left of −2. So we only 3 cos φ2 = sin(π/2 − φ2 ) < sin x2 < x2 , and 0 < − cos(φ2 + need to consider the case when i=1 λi = 1, under which ∗ x2 ) = sin(φ2 + x2 − π/2) < sin x2 < x2 . f (ω ) lies in the triangle formed by A1 , A2 and A3 , where If φ1 + x1 ≤ φ2 , then cos(φ1 + x1 ) ≥ cos φ2 . So we have Ai , i = 1, 2, 3 are the points in the complex plan corresponding to gi (ω ∗ ), i = 1, 2, 3. From the triangle property, there exists λ2 − cos φ2 cos(φ2 +x2 ) sin(φ1 +x1 ) some point on one of the three edges of the triangle that is on |R| = x2 [ cos(φ1 +x1 ) + cos φ2 sin(φ2 + x2 )] the real line to the left of −2. From Lemma VII.1, we know ≤ λ2 that this cannot be true. x2 [− cos(φ2 + x2 ) sin(φ1 + x1 ) + cos φ2 sin(φ2 + x2 )] Lemma VII.3. For any integer N > 0, if λi ≥ 0, θi > 1 x2 +x2 ≤ x2 [(− cos(φ2 + x2 )) + cos φ2 ] < x2 =2 0, Ti > 0, ∀i = 1, 2, ..., N and Tm = max(Ti : i = N 1, 2, ..., N ), i=1 λi ≤ 1, then If φ1 + x1 > φ2 , then φ1 > φ2 − x1 ≥ φ2 − x2 > 0, since φ2 > π/2 − 1/2 > 1/2 ≥ x2 . So cos φ1 < cos(φ2 − x2 ) = N N Ti θi e−jωTi cos φ2 cos x2 + sin φ2 sin x2 and thus we have f (ω) = λi gi (ω) = λi i=1 i=1 Tm jωTi (jωTi + θi ) 1 |R| = x2 [λ1 cos φ1 sin(φ1 + x1 ) + λ2 cos φ2 sin(φ2 + x2 )] 1 crosses the real line at a point to the right of −2 as ω varies ≤ x2 [λ1 (cos φ2 cos x2 + sin φ2 sin x2 ) sin(φ1 + x1 ) from −∞ to ∞. +λ2 cos φ2 sin(φ2 + x2 )] 1 Proof. From Caratheodory’s Theorem ([3], Proposition B.6, ≤ x2 [λ1 (cos φ2 + sin x2 ) + λ2 cos φ2 ] < 1 Page 679), let X be a subset of Rn ; then every element of x2 [λ1 (x2 + x2 ) + λ2 x2 ] ≤ 2 . conv(X) can be represented as a convex combination of no So under case 1, we have R > −2. more than n + 1 elements of X. So the convex combination of Case 2: cos(φ1 + x1 ) ≤ 0 and cos(φ2 + x2 ) ≥ 0, which gi (ω)’s can be represented as a convex combination of three lead to π > φ1 + x1 ≥ π/2 ≥ φ2 + x2 > 0. Then, we have gi (ω)’s. From Lemma VII.2, we know that it is true for any cos φ1 = sin(π/2 − φ1 ) ≤ sin x1 < x1 < x2 , and − cos(φ1 + integer N . x1 ) = sin(φ1 + x1 − π/2) < sin x1 < x1 ≤ x2 . If φ2 + x2 ≤ φ1 , then cos(φ2 + x2 ) ≥ cos φ1 . So we have With Lemma VII.3 at hand, we can now prove Theorem III.2 and Theorem IV.1: λ1 ˜ q x x Proof of Theorem III.2. Let ar := −ˆr /(ˆr Ur (ˆr )). Then, |R| = x2 [cos φ1 sin(φ1 + x1 ) + cos φ1 (− cos(φ1 +x1 )))sin(φ2 +x2 ) ] ˜ from (29), ar ≤ ar . cos(φ2 +x2 λ1 p p Let βl := (gl (ˆl )cl /ˆl ). Then from (28) and (29), we have ≤ x2 [cos φ1 sin(φ1 + x1 ) +(− cos(φ1 + x1 )) sin(φ2 + x2 )] Tr a r x r ˜ ˆ θr e−sTr ˆ βl p l ≤ 1 x2 +x2 L(s) = diag AT (−s)diag A(s) . x2 [cos φ1 + (− cos(φ1 + x1 ))] < x2 =2 ˆ qr sTr (sTr + θr ) cl 13 Since the open-loop system is stable, by the generalized where Nyquist Criterion [5] the closed-loop system is asymptotically stable if the eigenloci of L(jω) do not encircle −1. Now the ˆ ˆ βl p l 2Tm ar xr jω + κr ˜ ˆ A(jω) = diag A(jω)diag . eigenvalues of L(jω) are identical with those of cl ˆ qr κr Tr θr e−jωTr Given the condition in (30), we have: ˆ L(jω) = diag ˆ ˆ AT (−jω)A(jω) , Tm 2jωTr (jωTr + θr ) ˆ ˆ ˆ σ 2 (A(jω)) = ρ(AT (−jω)A(jω)) where a ˆ 2˜r Tm xr T ˆ βl p l κl = ρ(diag( )A (−jω)diag( )A(jω)) ˆ qr cl |jω + κl | βl p l ˆ ˜ ˆ 2Tm ar xr 2ar Tm T κl ˆ A(jω) = diag A(jω)diag . ≤ diag( )A (−jω)diag(βl pl | ˆ |) cl ˆ qr ˆ qr jω + κl 1 × diag( yl )A(jω)diag(ˆr ) ≤ 1 × 1 = 1 . ˆ x Let σ(Z) denote the largest singular value of a square matrix Z and ρ(Z) its spectral radius. Given the condition in (30), we The last inequality above uses (47) and have: κl 1 βl | | ≤ βl ≤ , ∀r, l : l ∈ r , ˆ 2 ˆT ˆ σ (A(jω)) = ρ(A (−jω)A(jω)) jω + κl 2ar Tm a ˆ 2˜r Tm xr T ˆ βl p l and thus the absolute row sums of these matrices are all less = ρ(diag( )A (−jω)diag( )A(jω)) ˆ qr cl than or equal to 1. 2ar Tm T Now, if λ is an eigenvalue of L(jω), then ≤ diag( ˆ )A (−jω)diag(βl pl ) ˆ qr 1 θr e−jωTr x × diag( )A(jω)diag(ˆr ) λ ∈ Co(0 ∪ { }) . yl ˆ 2jωTm (jωTr + θr ) ≤ 1 × 1 = 1. From Lemma VII.3, the eigenloci of L(jω) do not encircle -1; The last inequality above uses the facts that thus by the generalized Nyquist Criterion [5], the closed-loop system is stable. ˆ ˆ xr = yl = cl , ∀l ∈ L , ˆ ˆ pl = qr , ∀r ∈ R , (47) We ﬁnally prove Theorem IV.2. r:l∈r l:l∈r Proof of Theorem IV.2. Note that: and pl (1 − pl ) ≤ qr (1 − qr ) 1 l∈r βl < , ∀r : l ∈ r , ⇐⇒ zl ≤ e vr − 1 2ar Tm l∈r (e − 1) ⇐⇒ a l∈r (˜l − 1) ≤ ˜ l∈r al − 1 and thus the absolute row sums of these matrices are all less ⇐⇒ N ai ≤ N i=1 i=1 ai + N − 1 , than or equal to 1. Now, if λ is an eigenvalue of L(jω), then where ai := ezl ≥ 1 for all l, and (ai , 1 ≤ i ≤ N ) is a ˜ a permutation of (˜l , l ∈ r). Let us now prove the last inequality Tr θr e−jωTr by induction: it is obviously true for N = 1 and N = 2. Let λ ∈ Co(0 ∪ { }) . 2Tm jωTr (jωTr + θr ) us suppose that it is true for N . Then, for N + 1, we have From Lemma VII.3, the eigenloci of L(jω) do not encircle -1; N +1 ai ≤ N i=1 i=1 ai + N − 1 + aN +1 thus by the generalized Nyquist Criterion [5], the closed-loop = N +1 N i=1 ai + i=1 ai (1 − aN +1 ) system is stable. +N + (aN +1 − 1) N +1 N θr e−jωTr = i=1 ai + N − ( i=1 ai − 1)(aN +1 − 1) Remark 5. We can also show that N +1 jωTr (jωTr + θr ) ≤ i=1 ai + ((N + 1) − 1) always crosses the real axis to the left of the point −2/π, which suggests that our local stability condition will not be Thus the inequality also holds for N + 1, and therefore for all too conservative regardless of the value θ takes in [0, ∞). N. ˜ Proof of Theorem IV.1. Deﬁne ar as in the proof of Theo- R EFERENCES ˜ rem III.2. Then, ar ≤ ar , ∀r, and (41) becomes ¸ [1] T. Alpcan and T. Basar, “A game-theoretic framework for congestion Tr a r x r ˜ ˆ θr e−sTr control in general topology networks,” in Proc. 41th IEEE Conf. Decision L(s) = diag AT (−s) and Control, Las Vegas, Nevada, Dec. 2002. ˆ qr sTr (sTr + θr ) [2] S. Athuraliya, V. H. Li, S. H. Low, and Q. Yin, “REM: Active queue ˆ β l p l κl management,” IEEE Network, 15(3):48-53, May/June 2001. ×diag A(s) . cl s + κ l [3] D. Bertsekas, Nonlinear Programming. Athena Scientiﬁc, Belmont, MA, 1999, 2nd Edition. Similar to the proof of Theorem III.2, we need to show that the [4] S. Deb and R. Srikant, “Rate-based versus queue-based models of eigenloci of L(jω) do not encircle −1. Now, the eigenvalues congestion control, ” in Proc. ACM Sigmetrics, June 2004. [5] C. A. Desoer and Y. T. Wang, “On the generalized Nyquist stability of L(jω) are identical with those of criterion,” IEEE Trans. Automatic Control, 25:187–196, 1980. [6] W.-C. Feng, K. G. Shin, D. D. Kandalur, and D. Saha, “The BLUE ˆ Tr θr e−jωTr ˆ ˆ L(jω) = diag AT (−jω)A(jω) , active queue management algorithms,” IEEE/ACM Trans. Networking, Tm 2jωTr (jωTr + θr ) 10(4):513–528, Aug. 2002. 14 [7] S. Floyd and V. Jacobson, “Random early detection gateways for [36] H. Yaiche, R. R. Mazumdar and C. Rosenberg, “A game-theoretic congestion avoidance,” IEEE/ACM Trans. Networking, Aug. 1993. framework for bandwidth allocation and pricing in broadband networks,” [8] S. Floyd, “Highspeed TCP for large congestion windows,” Internet draft, IEEE/ACM Trans. Networking, 8(5):667–578, Oct. 2000. Aug. 2003. Available at http://www.icir.org/ﬂoyd/hstcp.html. [37] L. Ying, G. Dullerud and R. Srikant, “Global stability of Internet [9] R.J. Gibbens and F.P. Kelly, “Resource pricing and the evolution of congestion controllers with heterogeneous delays,” in Proc. American congestion control,” Automatica, 35:1969–1985, 1999. Control Conf., Boston, MA, June 2004. [10] H. Han, C. V. Hollot, Y. Chait, and V. Misra, “TCP network stabilized by buffer-based AQMs,” in Proc. IEEE Infocom, Hong Kong, March 2004. [11] C.V. Hollot, V. Misra, D. Towsley, and W. Gong, “A control theoretic analysis of RED,” in Proc. IEEE Infocom, Anchorage, Alaska, April 2001. Shao Liu (S’05) received B.S. degree from Peking [12] C.V. Hollot, V. Misra, D. Towsley, and W. Gong, “Analysis and design University, Beijing, and M.S. degree in electrical of controllers for AQM routers supporting TCP ﬂows,” IEEE Trans. engineering from University of Illinois at Urbana- Automatic Control, 47(6):945–959, June, 2002. Champaign, where he is currently working toward the [13] R. Johari and D. Tan, “End-to-end congestion control for the Internet: Ph.D. degree in electrical engineering and performing Delays and stability,” IEEE/ACM Trans. Networking, 9(6):818–832, Dec. research on congestion control for communication 2001. networks, and contention and congestion control for [14] F. Kelly, “Fairness and stability of end-to-end congestion control,” wireless ad hoc networks. His email address is: European Journal of Control, pages 149–165, 2003. shaoliu@ifp.uiuc.edu. [15] F. P. Kelly, A. Maulloo, and D. Tan, “Rate control in communication networks: shadow prices, proportional fairness and stability,” Journal of the Operational Research Society, 49:237–252, 1998. [16] F.P. Kelly, “Charging and rate control for elastic trafﬁc,” European Trans. Telecommunications, 8:33–37, 1997. [17] F.P. Kelly, “Models for a self-managed Internet,” Philosophical Transactions of the Royal Society, A358:2335–2348, 2000. [18] T. Kelly, “Scalable TCP: Improving performance in highspeed wide area networks,” Dec. 2002. Available at http://www- ¸ Tamer Basar (S’71-M’73-SM’79-F’83) received lce.eng.cam.ac.uk/ ctk21/papers/. B.S.E.E. degree from Robert College, Istanbul, and [19] S. Kunniyur and R. Srikant, “End-to-end congestion control: utility M.S., M.Phil, and Ph.D. degrees in engineering and functions, random losses and ECN marks,” in Proc. IEEE Infocom, Tel applied science from Yale University. After stints Aviv, Israel, March 2000. at Harvard University and Marmara Research Insti- [20] S. Kunniyur and R. Srikant, “Analysis and design of an adaptive virtual tute (Gebze, Turkey), he joined the University of queue algorithm for active queue management,” in Proc. ACM Sigcomm, Illinois at Urbana-Champaign in 1981, where he is pages 123–134, San Diego, CA, Aug. 2001. currently the Fredric G. and Elizabeth H. Nearing Professor of Electrical and Computer Engineering. [21] S. Kunniyur and R. Srikant, “Designing AVQ parameters for a general He has published extensively in systems, control, topology network,” in Proc. Asian Control Conf., Singapore, Sept. 2002. communications, and dynamic games, and has cur- [22] S. Kunniyur and R. Srikant, “Note on the stability of the AVQ scheme,” rent interests in modeling and control of communication networks, control in Proc. Conf. Information Sciences and Systems, Princeton, NJ, March over heterogeneous networks, resource management and pricing in networks, 2002. and robust identiﬁcation and control. [23] S. Kunniyur and R. Srikant, “A time-scale decomposition approach to ¸ a Dr. Basar is the Editor-in-Chief of Automatica, Editor of the Birkh¨ user adaptive ECN marking” IEEE Trans. Automatic Control, 47(6):882–894, Series on Systems & Control, Managing Editor of the Annals of the Inter- June 2002. national Society of Dynamic Games (ISDG), and member of editorial and ¸ [24] S. Liu, T. Basar and R. Srikant, “Controlling the Internet: a survey and advisory boards of several international journals. He has received several some new results,” in Proc. IEEE Conf. Decision and Control, Maui, awards and recognitions over the years, among which are the Medal of Science HI, Dec. 2003. of Turkey (1993), Distinguished Member Award (1993), Axelby Outstanding [25] S. H. Low and D. E. Lapsley, “Optimization ﬂow control, I: Basic Paper Award (1995) and Bode Lecture Prize (2004) of the IEEE Control algorithm and convergence,” IEEE/ACM Trans. Networking, pages 861– Systems Society (CSS), and the Quazza Medal (2005) of IFAC. He is a member 875, Dec. 1999. of the National Academy of Engineering, a member of the European Academy [26] S. Low, F. Paganini, J. Wang, S. Adlakha and J. Doyle, “Dynamics of of Sciences, a Fellow of IEEE, a past president of CSS, and a past (founding) TCP/RED and a scalable control,” in Proc. IEEE Infocom, June 2002. president of ISDG. His email address is: tbasar@control.csl.uiuc.edu. [27] V. Misra, W. Gong, and D. Towsley, “A ﬂuid-based analysis of a network of AQM routers supporting TCP ﬂows with an application to RED,” in Proc. ACM Sigcomm, Stockholm, Sweden, Sept. 2000. [28] J. Mo and J. Walrand, “Fair end-to-end window-based congestion control,” IEEE/ACM Trans. Netowrking, 8(5):556–567, Oct. 2000. [29] F. Paganini, J. Doyle, and S. Low, “Scalable laws for stable network congestion control,” in Proc. IEEE Conf. Decision and Control, Dec. R. Srikant (S’90-M’91-SM’01) received his B.Tech. 2001. from the Indian Institute of Technology, Madras in [30] F. Paganini, Z. Wang, J. Doyle, and S. Low, “A new TCP/AQM for stable 1985, his M.S. and Ph.D. from the University of Illi- operation in fast networks,” in Proc. IEEE Infocom, San Francisco, CA, nois in 1988 and 1991, respectively, all in Electrical April 2003. Engineering. He was a Member of Technical Staff at AT&T Bell Laboratories from 1991 to 1995. He a [31] R. Srikant, The Mathematics of Internet Congestion Control. Birkh¨ user, is currently with the University of Illinois, where he Boston, 2004. is a Professor in the Department of Electrical and [32] G. Vinnicombe, “On the stability of end-to-end congestion control Computer Engineering and a Research Professor in for the Internet,” University of Cambridge Technical Report CUED/F- the Coordinated Science Lab. INFENG/TR.398, 2001. Available at http://www.eng.cam.ac.uk/˜gv. He was an associate editor of Automatica, and is [33] G. Vinnicombe, “On the stability of networks operating TCP-like currently on the editorial boards of the IEEE/ACM Transactions on Networking congestion control,” in Proc. IFAC World Congress, Barcelona, Spain, and IEEE Transactions on Automatic Control. He was the chair of the 2002 2002. Available at http://www.eng.cam.ac.uk/˜gv. IEEE Computer Communications Workshop and will be a program co-chair [34] G. Vinnicombe, “Robust congestion control for the Internet,” of IEEE INFOCOM in 2007. His research interests include communication University of Cambridge Technical Report, 2002. Available at networks, stochastic processes, queueing theory, information theory, and game http://www.eng.cam.ac.uk/˜gv. theory. His email address is: rsrikant@uiuc.edu. [35] J.T. Wen and M. Arcak, “A unifying passivity framework for network ﬂow control,” IEEE Trans. Automatic Control, 49(2):162–174, 2004.