# massey_bmac04

Document Sample

```					      Beyond BGP

Dan Massey
Internet Routing
   Challenges Facing Internet Routing
 Internet     Has Grown Dramatically
– Large number of routing entries
– Frequent topological changes
 Fault-Model       Has Changed Dramatically
– More malfunctioning components
– Intentional attacks
   Do we need a fundamentally new
routing architecture?

24 October 04               massey@cs.colostate.edu     2
Toward a New Architecture
   One claim: BGP is nearing the end of its
   The Internet will soon collapse unless we act!!
   Other claim: BGP is the best engineering
solution we are likely to produce
   We need incremental patches to new problems
   Who is right?
   Beyond BGP uses
– Measurements to assess where we are
– Identification of (new?) routing requirements
– Development of changes (incremental or new

24 October 04                  massey@cs.colostate.edu            3
How Did We Get To BGP
   Simple Distance Vector Routing Algorithms
   Used in early Internet routing designs
   Convey only limited information
   Prone to long lasting loops
   Expensive Link State Routing Algorithms
   Learn the Full Network Topology
   Signal every change in every link
   Path Vector Routing (BGP)
   Middle ground that signals some path data
   But does not signal the full topology

24 October 04              massey@cs.colostate.edu       4
RIP and DBF
RIP                Distributed Bellman-
•Exchange distance info. • Keep distance info from all neighbors
Ford(DBF)
•B’s route to D:                     •B’s route to D:
Nexthop=A, Dist=4                           Nexthop=A, dist=4
Alternate Nexthop=C, Dist=4
• Keep shortest path only
RIP and DBF:
D
• 30sec refreshing interval
A
D:1              •Damping timer to space out two
B                       E                   F      triggered updates: 1~5 seconds

C                                    •Poison reverse: B sends
infinity distance to A

24 October 04                  massey@cs.colostate.edu                             5
BGP Background
   Internet: composed of thousands of Autonomous Systems(ASes).
BGP (Border Gateway Protocol): the de facto inter-AS routing protocol


BGP Routers                                          BGP Routers
AS A      R1
R2

AS B
R3                                      AS E
R6
R4

R5
AS C

24 October 04               massey@cs.colostate.edu                      6
How BGP works
    Uses path vector protocol
– similar to distance vector protocol.

   Consider an AS as a node
   route includes entire path(sequence of nodes)
   what if no path available?                            D

B’s route to D:                                               A

B                  E
Route via C = <C E A>
Route via A = <A>                                               C

24 October 04                         massey@cs.colostate.edu               7
Path Vector Routing Changes
   Worms triggered edge instabilty
   Routers crashed due to ARP cache overflow.
   Links were congested by worm traffic.
   BGP Path Exploration Exacerbates Dynamics
D
withdraw
B’s route to D                                       withdraw
A

Route via A=<A>
B                       E
Route via C=<C E A>

C
withdraw
   Obsolete backup path <C E A >is used and convergence is delayed

24 October 04                 massey@cs.colostate.edu                        8
Policies and Policy Withdrawal
D
policy withdraw
B’s route to D                                   A
A

Route via A= <A >
Route via A=
B
B
<A>                                                    E
E
Route via C=<C A>
Route via C=<C E E
A>                                               C
C

But A could stop advertising to B due to a policy change,
path <C E A> is still valid!

   Attach a Failure Withdrawal Community Attribute
   Only apply the approach to failure withdrawal

24 October 04               massey@cs.colostate.edu                 9
BGP Traffic Engineering
    We assumed an AS could be modeled as a node
    with a single best path to the destination
   But a single AS may advertise more than one path.
   BGP Traffic Engineering:
R4 chooses path <C B A>
R5 chooses path <C E A>

Divide one AS into Logical
ASes such that
All routers within a logical AS have
the same best patheach logical
AS can be modeled as a node.

24 October 04                    massey@cs.colostate.edu                           10

Original BGP                               Substantial reduction is
achieved.
Enhanced BGP                                    E.g. 3766 to 1419 in
the 60-AS topology

within 30 seconds, only
allowed.

   It “packs” consecutive
Number of ASes in Network                     changes into one
update.

24 October 04             massey@cs.colostate.edu                                11
Convergence time
Enhanced BGP reduces the
Convergence Time(seconds)



Original BGP                                   convergence time
substantially.
Enhanced BGP
   E.g. 337.0 seconds to
19.5 seconds in the 60-
AS topology

   Elimination of one
convergence time by
30 seconds

Number of ASes in Network

24 October 04                massey@cs.colostate.edu                               12
Improving Path Vector Convergence
   Infocom 02 [4] uses consistency to detect invalid paths.
   Reject path <x1, x2,…, xn, r1,r2…, rm> if
r1’s path is not <r1, r2, …., rm>
   Adjusted to account for policy and implement in BGP
   Infocom 03 [Afek, et al] quickly flushes invalid paths.
   BGP requires updates be separated by a min interval
   Send withdraw (to flush route) if blocked by the interval
   Our recent work [5] attaches a new attribute:
   Identifies the failed link and includes a sequence number.
   Allows any route relying on the failed link to be rejected.

24 October 04                massey@cs.colostate.edu                    13
Analyzing Path Vector Convergence
   Route fail-over has
two stages.
   First, nodes inside the
blue triangle lose
routes and explore
backup paths.
   All short invalid paths
are explored
   Second, an edge (a0)
eventually selects the
valid backup path via
Sk.
   Valid routes begin to
propagate through the
blue triangle.

24 October 04                massey@cs.colostate.edu   14
Generic Convergence Results
Algorithm              Fail-Over Convergence Bounds

SPVP (BGP)           (N-1) (M + ld) + 3 Pmax(|E|-degree(G,0))

SPVP-AS               (N- degree(G,0) ) (M+ld) +
3Pmax(|E| - |E^| + Degree(G^))
SPVP-GF               (N-1) ld + 3Pmax(|E| - degree(G,0))

SPVP-RCN              Distance(G,0) (ld) + (Pmax) Distance(G,0)

Pmax = Node Processing Delay,                           ld = Link Delay

24 October 04                massey@cs.colostate.edu                         15
Simulation Results

24 October 04       massey@cs.colostate.edu   16
   Convergence Discussion Neglects Security
   What if routers send intentionally bad information?
   What is the Simplest Possible Attack?
   Announce someone elses routes
   Example: Suppose Univ. of Colorado announces it
is the origin for 129.82.0.0/16
   In other words, CU announces CSU IP Address Space
   Can this Happen and/or What Would Prevent It?

24 October 04                massey@cs.colostate.edu               17
Multiple Origin AS (MOAS) Cases

   Prefixes originate from Multiple Origin AS (MOAS)
   Lower curve likely due to valid operational needs
   Spikes are errors that disrupt routing to prefix
   Includes loss of routes to top level DNS servers
24 October 04               massey@cs.colostate.edu         18
Infrastructure Faults and Attacks
   BGP and DNS Provide No Authentication
   Faults and attacks can mis-direct traffic.
   One (of many) examples observed from BGP logs.
   Server could have replied with false DNS data.
originates route to
ISPs announced new path                                  192.26.92/24
for 20 minutes to 3 hours

Internet        c.gtld-servers.net

BGP                                                 192.26.92.30
monitor

24 October 04               massey@cs.colostate.edu                         19
18.0.0.0/8    BGP-based Solution Example

18/8, PATH<58>, MOAS{58,59}
AS58

AS52
AS59
18/8, PATH<52>, MOAS{52, 58}
18/8, PATH<4>, MOAS{4,58,59}
Example configuration:
router bgp 59
neighbor 1.2.3.4 remote-as 52
neighbor 1.2.3.4 send-community
neighbor 1.2.3.4 route-map setcommunity out
route-map setcommunity

24 October 04              massey@cs.colostate.edu                                      20
BGP false origin detection
Simulation Results

(a) One Origin AS                     (b) Two Origin AS’s
24 October 04            massey@cs.colostate.edu                 21
A Simple Filter
   Current BGP provides dynamic routes
   Explore the opposite extreme...

   Select a single static route to each server.
   Apply AS path filters to block all other announcements.
– Also filter against more specifics.

   Route changes on a frequency of months, if at all.
   Change in IP address, origin AS, or transit policy.
   Adjust route only after off-line verification

24 October 04                    massey@cs.colostate.edu          22
Why This Works: Theory
   Scale is limited to a small number of routes.
   No exponential growth in top level DNS servers.

   Loss of a server is tolerable, invalid server is not.
   Resolvers detect and time-out unreachable servers.
– Provided surviving servers handle load, cost is some delay.

   Expect predictable properties and stable routes.
   Servers don’t change without non-trivial effort.
   Servers located in highly available locations.

24 October 04               massey@cs.colostate.edu                   23
Why This Works: Data
   Analysis based on BGP updates from RIPE.
   Archive of BGP updates sent by each peer.
   9 ISPs from US, Europe, and Japan.
   February 2001 - April 2002
    Some data collection notes
   Used only peers that exchange full routing tables
– Otherwise some route changes are hidden by policies
   Adjusted data to discount multi-hop effect.
– Multi-hop peering session resets don’t reflect ISP ops.

24 October 04                     massey@cs.colostate.edu                   24
Impact on Reachability
ISP1 (US/Tier 1)

24 October 04        massey@cs.colostate.edu   25
How Static Are The Routes?
   3 changes in route to “A” over 14 months.
   2 (valid) changes in the origin AS
   5/19/01 origin AS changed from 6245 to 11840
   6/4/01 origin AS changed from 11840 to 19836

   1 change in transit AS routing policy
   11/8/01 (*,10913, 10913, 10913,*) -> (*,10913, *)
   Could have built filter to allow this...

24 October 04                  massey@cs.colostate.edu           26
What Routes Are Lost?
   Results from 3/1/01 until 5/19/01 AS change.
   Reduced reachability to “A” from 99.997% to 99.904%

   18 events when trusted route was withdrawn
   2 resulted in no route available (28 secs, 103 secs)
   8 instances of a back-up route lasting over 3 minutes
   Longest lasting back-up advertised for 15 minutes

   Similar results for other time periods and servers.

24 October 04             massey@cs.colostate.edu               27
Example of Filtered Routes
Time                                  Tail of AS Path
12:35:30             * 19836 19836 19836 19836
16:06:32             * 10913 10913 10913 10913 10913 10913 10913 19836
16:06:59             * 1239 10913 19836
16:07:30             * 701 10913 10913 19836
16:08:30             withdrawal
16:15:55             * 19836 19836 19836 19836

1239
10913
*                                                server
19836
No route at 16:08:30 701

    With filter no route at 16:06:32
24 October 04                  massey@cs.colostate.edu                    28
Worst Case In Study
ISP 3 (Europe)

ISP 3 used one main route and a small
number of consistent back-up routes.

24 October 04           massey@cs.colostate.edu         29
Toward a More Balanced Approach
   Required infrequent updates to the filter.
   Especially useful to automate infrequent tasks.
– Natural tendency to forget task or forget how to do
   More paths improves robustness
   Simple filtered allowed only 1 path.
   ISP3’s reachability can be improved if filter
allows two routes…
   Strike a balance between allowing dynamic
changes and restricting to trusted paths.

24 October 04                  massey@cs.colostate.edu                  30
   Slow down the route dynamics and add
validation.
   Apply hysteresis before accepting new paths
   Add options for validating new paths:
– Believe route based purely on hysteresis
– Probabilistic query/response testing against known
data.
– Trigger off-line checking (did origin AS really
change?)

24 October 04                  massey@cs.colostate.edu                 31
Impacts on Reachability
ISP1

gTLD servers

Root servers

24 October 04                  massey@cs.colostate.edu                         32
Impacts on Reachability
ISP3

gTLD servers

Root servers

24 October 04                  massey@cs.colostate.edu                         33
Convergence And Authentication
   BGP Suffers From Both Convergence Problems
and Authentication Problems
   Convergence fixes are good, if no attacks.
   Authentication fixes work for redundant sites
   Can you improve both convergence and
authentication in a realistic environment?
   Do you need to replace BGP?
– If yes, with what?
   Would you pick BGP for your new network?
– If no, what would you do instead?
   Wide Variety of Other Routing Challenges
   Check out CS 580 and BBGP Project if interested

24 October 04                     massey@cs.colostate.edu      34
BGP Measurement and Artifacts
   BGP peers establish TCP session and
send full route table (120K+ routes)
   Updates sent only if routes change.                   Initial Table
   Our results show frequent session                        (120K+ routes)

resets between ISP routers and
the monitoring point.
   Monitoring point sessions cross            Route
multiple systems in the Internet.         Changes
   But very few ISP-ISP session resets.
   Our work in [1] presents rules to
remove session reset artifacts.

24 October 04                massey@cs.colostate.edu                       35

24 October 04   massey@cs.colostate.edu   36

Total
Attack

Routing
Change                              Measureme
s                                   nt Artifacts

24 October 04             massey@cs.colostate.edu                  37
What Our Analysis Shows

37%
37.6%
42%
40.2%     BGP Table Exchange
New Announcements
Withdraws
Implicit Withdraws

8%
8.3%
8%       5%
8.8%

A substantial percentage of the BGP messages during the worm
attack were not about route changes
24 October 04                 massey@cs.colostate.edu                        38
FRTR: Improving Peer Communication
   BGP Updates Are Not (Topology) Event Driven
   Session resets trigger high volume surges
– Govindan shows cascade failures can result.
   Lifetime of Invalid Routes is Unbounded
   Never recover (until reset) if update is somehow lost.
– Despite TCP, we found cases of “lost” withdrawals.
   Attacker can poison a route with one update.
   Soft-state (periodic re-announce) is too costly…

   FRTR Uses Periodic Bloom Filter Digests
   Digests quickly confirm state after session reset.
   Periodic digests bound lifetime of faults (w/ high prob).
   Co-Author Keyur Patel (Cisco) is exploring Cisco
development.

24 October 04                    massey@cs.colostate.edu                 39
FRTR Performance
   For each route at receiver,
check against the digest.
   Bloom filter results in no
false negatives.
   Compare total digests for
missing route detection.
   False positive possible with
known rate.
   Add salts to reduce the
chance of repeated false
positives.
   Overhead is a function of
digest size and frequency.
   Work with Cisco suggests
   Complete Details to
appear in [2] (DSN 2004)

24 October 04                    massey@cs.colostate.edu   40
Packet Delivery during Routing Convergence

    Failures do occur in the Internet
–       20% of intra-ISP links have a MTTF < 1 day [Diot:IMW02]
–       40% of Inter-ISP routes have a MTT-Change < 1 day [Labovitz:FTCS-29]

     Routing convergence after failure takes time
–    IS-IS(Intra-ISP protocol): 5+ seconds [Diot:IMW02]
–    BGP(Inter-ISP protocol): 3+ minutes [Labovitz:Sigcomm00]

     Packets can be delivered during convergence

A                 B               C                   D

E                                 F                  G

24 October 04                                massey@cs.colostate.edu                        41
What Is the Goal of Routing
   How to maximize packet delivery during routing
convergence?

– Previous work focused on: preventing loops, minimizing

– Topological connectivity’s impact?
– Studying: RIP, Distributed Bellman-Ford(DBF), BGP

   This problem becomes more important with
Larger Internet topology [Huston01] --> higher freq. of component failures
Richer connectivity[Huston01] --> potentially helps with more alternate paths
Higher bandwidth --> more packets sent during convergence

24 October 04                        massey@cs.colostate.edu                                42
Simulation conducted
20 pkts/second

    7 by 7 mesh topologies similar those
in [Baran64]

   Simulated node degree range [3 ~ 16]

     Measure Packet loss, loops, path
convergence time, throughput,
and e2e delay.

24 October 04               massey@cs.colostate.edu               43
Packet Losses (I) : Observation

Packet losses of
DBF, BGP’ and
BGP decrease to
Packet Loss

zero at degree 6.
RI
P
Richer connectivity
helps RIP little.
DBF, BGP’ and BGP

Node
Degree
24 October 04               massey@cs.colostate.edu                    44
Packet Loss(II): Lessons Learned
D
       Keeping alternate
paths                        RIP:                  A

B                   E   F

DBF, BGP:                       C

   Connectivity Matters                                           D
no immediate available                             A
alternative due to poor
connectivity and poison
reverse                             B                   E   F

alternative is more likely                         C
with richer connectivity

24 October 04                massey@cs.colostate.edu           45
Is an alternate path valid?

D
     Valid Alternate Paths: not                                       A
       Poison reverse and BGP’s path              B                       E       F
information are not enough!
[Pei:Infocom2002]
C
C2
X
U                W
V
   Richer connectivity -->
 reduces one single link’s impact
 better availability of valid(but may

be suboptimal) path

24 October 04                                massey@cs.colostate.edu                    46
Transient Loops(I): Observation

BGP
•BGP has the most
loops!
Losses due to

•RIP has no loops

•Richer connectivity
loops

DB     BGP’                              reduces the chance
F                                        of looping.

Node
Degree
24 October 04                  massey@cs.colostate.edu                       47
Transient Loops(II): Msg Propagation

Damping timer
slows the msg                                         V   W
U
propagation, causing                                          X
looping                                                               D
Y
A

B                  E           F

Richer connectivity can                                    C
reduce the chance of                         30
looping                                      seconds!
   More details in:
“A Study of Transient Loops in
BGP”

24 October 04                 massey@cs.colostate.edu                   48
Throughput(pkts/second   Instantaneous Throughput

BGP’

BGP
DBF

RIP
RIP

Time

24 October 04                           massey@cs.colostate.edu         49
Packet Delay During Convergence

24 October 04     massey@cs.colostate.edu   50
Forwarding Path Convergence time

BGP: no loss at                     Time till the forwarding path
degree 6 or higher                   from S to D stabilizes.
Time till there is no routing msg.
BGP:13

BGP:70
BGP’:2

BGP’:10
    Shall we still tune
MRAI timer to
minimize
convergence
time(with the risk of
Node                           increasing
24 October 04            massey@cs.colostate.edu                        51
Packet Delivery After a Failure

24 October 04        massey@cs.colostate.edu   52

```
DOCUMENT INFO
Shared By:
Categories:
Stats:
 views: 12 posted: 3/4/2010 language: English pages: 52
How are you planning on using Docstoc?