# Lecture 11 2010

Document Sample

```					     EE384x Spring 2010
Lecture

1. "Scaling Internet Routers Using Optics"
[Isaac Keslassy et al. 2003]

Nick McKeown
An Aside: Capacity growth

Backbone link speeds followed Moore’s Law
User demand approx doubles every year

Router capacity limited by memory speed
DRAM random access time approx same

a Routers should have fallen behind
Aside: Backbone router capacity
Followed Moore’s Law
1Tb/s

100Gb/s

10Gb/s
Router capacity per chassis
doubled every 18 months
1Gb/s

1986   1988   1990   1992   1994    1996   1998   2000   2002   2004
Aside: What limits capacity growth
Packet processing
 Packet   buffering, link scheduling, lookup,
classification, stats gathering
 Tunneling, encapsulation, recirculation

System
 Switch   arbiters (schedulers)
This lecture is about scaling capacity while
maintaining 100% throughput
Remember deterministic approaches (e.g.
DSM): How well do they scale?                     4
Recap
   “Maximize throughput” is equivalent to
“remain stable for all non-oversubscribing
traffic matrices”. i.e. l < m for every queue
in the system, for all such traffic matrices.

   Observations:
 Burstiness   of arrivals does not affect
throughput
 When traffic is uniform, solution is trivial

5
If traffic is uniform
Traffic matrix
 l11 K     l1N 
               
 K lij     K ,    lij  l < 1/N,   ij
lN1 K
           lNN 


Each VOQ
                           m
l  R/ N


Intuition
If scheduler ensures m  l then switch is stable
If Traffic Is Uniform
R                    l
R/N R / N        m  R/ N    R
In        R/N
Out
R/N


R R R/ N                R/N
 R/ N          R
R/N                           R
 R/ N
In                        Out
R/N

R/N
R                    R/N                          R
In       R/N              Out

If traffic was uniform, life would be easy
Outline for Today
 Packet mis-sequencing
 An optical switch fabric

8
100% Throughput in a Mesh Fabric
R            ?
R                                   R
In     R
?
Out
R
?

R
?
R                                  R
R   R
In     ?
R                     R
R   Out
?
R

R
?
R               ?
R                                R
In    R
?                          Out
Switch capacity = N2R
Router capacity = NR
9
If Traffic Is Uniform
R                 l
R/N R / N   m  R/ N   R
In        R/N
Out
R/N

R/N
 R/ N
R R
 R/ N   In   R   R/N
Out
R
 R/ N            R/N

R/N
R                  R/N                   R
In       R/N          Out

10
Real Traffic is Not Uniform
R R R // N
RR N
                  R
R/N                             R
 R/ N      In       R/N
Out
R/N

R/N
 R/ N
R
R R   R/ N                                               R
?
R/N
 R/ N
In   R
Out
R/N

R/N

R R // N
R RR N

R/N
R
 R/ N
In   R
R/N                 Out

Can we make traffic “sufficiently uniform” to
make the problem trivial?                   11
R        In
R/N
Out
R       R/N
Out
R
R/N                         R/N

R/N                        R/N

R/N                        R/N
R        In    R/N
Out
R       R/N
Out
R
R/N                        R/N

R/N                        R/N

R/N                        R/N
R        In                Out
R
Out
R
R/N                        R/N

100% throughput for weakly mixing traffic (Valiant, C.-S. Chang)
12
3
2
1   R                                R
In                     Out
R/N        R/N

R/N        R/N

R/N        R/N

R/N        R/N
R                                R
In   R/N        R/N    Out
R/N        R/N

R/N       R/N

R/N
R         R/N
R
In   R/N        R/N
Out

13
R                                  R
In                       Out
R/N          R/N

R/N
1    R/N

R/N          R/N

R/N         R/N
R                                  R
In   R/N          R/N    Out
R/N
2   R/N

R/N         R/N

R/N
R         R/N
R
In   R/N          R/N
Out
3

14
Intuition: 100% Throughput
R                                      R
In   R/N
R/N
R/N
Out
R/N
R/N              R/N

R        R/N                           R
In                           Out
R/N
R/N              R/N
R/N              R/N

R/N              R/N
R                                      R
In                           Out
R/N               R/N

R/N              R/N

a                    b     C
1 1 1
1                      
 Arrivals to second mesh: b  U  a, where U  1 1 1
N                 1 1 1
     
R
 Capacity of second mesh: C  U
N

   Buffers in middle stage: We would like            C  b (i.e. m  l)
1
C b        RU  U a  0           [C.-S. Chang]
N                                           15
Another way of thinking about it
External                    Internal                External
Inputs                      Inputs                  Outputs
1                            1                        1

cyclic shift                cyclic shift

Interesting properties:
• 100% throughput, no arbiter (but 2x switching capacity)
• No part of the system need operate faster than the line rate
16
Outline
 Packet mis-sequencing
 An optical switch fabric

17
Packet Reordering

R                                   R
In                        Out
R/N           R/N
2 1
R/N          R/N

R/N           R/N

R/N          R/N
R                                   R
In   R/N           R/N    Out
R/N          R/N

R/N          R/N

R/N
R         R/N
R
In   R/N           R/N
Out
18
Bounding Delay Difference
Between Middle Ports
 cells
R               1                             R
In                                  Out
R/N             R/N

R/N            R/N

R/N             R/N

R/N            R/N
R                   2                         R
In   R/N             R/N            Out
R/N            R/N

R/N            R/N

R/N
R         R/N
R
In   R/N             R/N
Out
19
0
R                                      R
In                          Out
R/N        R/N

3 2 1         R/N       R/N

R/N       R/N

R/N       R/N
R                                      R
2 1
In   R/N        R/N         Out
R/N       R/N

R/N       R/N

R/N
R          R/N
R
In   R/N       R/N
Out

20
FOFF (Full Ordered Frames First)

R                               R
In                   Out
R/N      R/N

R/N     R/N

R/N      R/N

R/N      R/N
R                               R
2 1
In   R/N      R/N    Out
R/N      R/N

R/N     R/N

R/N
R          R/N
R
In   R/N      R/N
Out

21
FOFF (Full Ordered Frames First)
N
4 3 2 1

2 1

Input Algorithm
   N FIFO queues corresponding to the N output flows
   Spread each flow uniformly: if last packet was sent to
middle port k, send next to k+1.
   Every N time-slots, pick a flow:
- If full frame exists, pick it and spread like UFS
- Else if all frames are partial, pick one in round-robin
order and send it                                            22
Bounding Reordering
N
N
3 2 1   R                                    R
In                         Out
R/N            R/N

R/N           R/N

R/N            R/N

R/N           R/N
R                                    R
In   R/N            R/N    Out
R/N           R/N

R/N           R/N

R/N
R         R/N
R
In   R/N            R/N
Out
23
FOFF
Output
N
4 1 1 1

2 2

3 3 3

Output properties
 N FIFO queues corresponding to the N middle
ports
 Buffer size less than N2 packets
 If there are N2 packets, one of the head-of-line
packets is in order
24
VLB + FOFF Properties

With quite a lot of work, packet order is
maintained

Interestingly, expected packet delay is within a
constant of OQ switch (surprising)

Therefore, VLB with FOFF has 100% throughput

25
Outline
 Packet mis-sequencing
 An optical switch fabric

26
From Two Meshes to One Mesh
One linecard
R                                            R
In                                 Out
R/N                    R/N

R/N                   R/N

R/N                    R/N

R/N                   R/N
R
In   R/N     In             R/N    Out
R

Out
R/N                   R/N

R/N                   R/N

R/N
R         R/N
R
In   R/N                    R/N
Out
27
From Two Meshes to One Mesh

One linecard
R              R
In             First mesh
Out                   Second mesh
R
In
Out
R
In
Out

R
In
Out

28
From Two Meshes to One Mesh

R         2R
In         Combined mesh
Out
2R
In
Out
2R
In
Out

2R
In
Out

29
Many Fabric Options
One linecard                    N channels each at rate 2R/N
C1, C2, …, CN
In
Out            C1

In
Out            C2                                   Options
Any spreading   Space: Full uniform mesh
In                           device       Time: Round-robin crossbar
Out            C3
Wavelength: Static WDM

In
Out            CN

30
AWGR (Arrayed Waveguide Grating Router)
A Passive Optical Component

1 1
l1
Linecard 1 l 1, l 2 … N          l11   Linecard 1

Linecard 2                       l12   Linecard 2
NxN
AWGR

Linecard N                       l1N   Linecard N

   Wavelength i on input port j goes to output
port (i+j-1) mod N
   Can shuffle information from different inputs 31
Static WDM Switching: Packaging

A, A, A, A
A   In
Out   A, B, C, D
B, B, B, B
B   In                             AWGR
Out   A, B, C, D
C, C, C, C        Passive and
In
C         A, B, C, D             Almost Zero
Out
Power
D, D, D, D
D   In
Out   A, B, C, D

N WDM channels,
each at rate 2R/N

32
Review
 Did we achieve our goal of a scalable
system with 100% throughput?
 What happens when linecards fail or are

Opinion: VLB is promising, but requires
more work to make practical

33

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 8 posted: 11/2/2011 language: English pages: 33