Docstoc

Purdue - PowerPoint Presentation

Document Sample
Purdue - PowerPoint Presentation Powered By Docstoc
					 The Evolution of the U.S.
Internet Peering Ecosystem

         William B. Norton
Co-Founder and Chief Technical Liaison
            Equinix, Inc.
           Internet Researcher
•   90% with Internet Companies Engineers
•   EQIX: Massive Carrier Neutral Colocation
•   Lots of Blinking lights…
•   Observe: documentation on HW&Protocols
•   Lack of Operations documents
•   Research How does Peering work?


                                  White paper process..
Community Operations Research
• Ground Truth w/dozens of experts
• Write White Paper v0.1
• Walk community through WP for
  comments
• Revise White Paper into new version
• Present White Paper at conferences
• Solicit comments over lunches and dinners

                                White papers so far…
Internet Operations White Papers
1) “Interconnection Strategies for ISPs”
2) “Internet Service Providers and Peering”
3) “A Business Case for Peering”
4) “The Art of Peering: The Peering
   Playbook”
5) “The Peering Simulation Game”
6) “Do ATM-based Internet Exchanges
   Make Sense Anymore?”
     Freely available. Send e-mail to wbn@equinix.com
                       New white Paper: The Evolution of the U.S. Peering Ecosystem, Agenda…
                   Agenda
• Definitions to level set
  – Peering, Transit, ISPs
  – Synch Point here.
• Introduction to notion:“Peering Ecosystem”
  – Internet=Many Internet Regions each with
  – Tier 1 ISPs, Tier 2 ISPs, Content Providers
• Evolution of the U.S. Peering Ecosystem
  – 3 major evolutions
  Def: Internet Service Provider
• Def: Internet Service Provider is an entity
  that sells access to the entire Internet. This
  service is often called “Internet Transit”
• Def: Transit is a business relationship
  whereby one entity sells access to the entire
  Internet.
                                                              I
          Definition of Transit                               N
                                                              T
Def: Transit is the business                                  E
relationship whereby one ISP                                  R
announces (sells)                                             N
reachability to the *entire*                                  E
Internet to a customer.      2) Reachability                  T
                              Announcement
                                                  Upstream
                                                  Upstream
                                                Upstream
                                               Upstream
          1) ISP A buys Transit Service            Transit
                                                   Transit
                                                 Transit
                                                Transit       N
  ISP A
                  3) Traffic Flows                Provider
                                                   Provider
                                                 Provider
                                               Providers      E
•$$$ ? Typically usage-based                                  T
•Pricing ’04: $18-$200/Mbps                                   W
•Volume based on 95th Percentile measure                      O
•Transit is Simple, Convenient:                               R
• Upstream handles the delivery of packets                    K
     to the Internet by some means                            S
Usage: “I’m purchasing transit from Level 3.”
    Cost of Transit Traffic Exchange
                                          Transit

            250
            200
   $/Mbps




            150
                                                               Transit $/Mbps
            100
            50
             0
                  1
                      14
                           27
                                40
                                     53
                                          66
                                               79
                                                    92
                                                         105
                      Traffic Exchanged (Mbps)

Source: 2004 Survey Avg. Range: $20-$400/Mbps 95th Percentile
2001 Pricing Sampling Range: $100-$1200/Mbps
1/15/04 field spot quotes: GX: $65/Mbps, HE: $25/Mbps
     My Financial Models used $388/Mbps
Cogent $20/Mbps UU: $45/Mbps, etc. varied commits 0-1000Mbps
                                                                           Why Peer?
                          Scaling at the Edge
 Streaming Media                                     The Zone: +1M gaming user/mo!
         Broadcast.com: 100,000 concurrent unicast streams
         15 million streaming hrs/mo, 1300 Live events/day
Streaming:
Broadcast
Telephony            B      M         IG                                                  U
Video
Multimedia
              T                                                          $$$              P
Interactive
Gaming
SPOT Events
                          Content Heavy ISP                                               S
             V                                                                            T
                                                                                          R
                                                                         $$$              E
                           Access Heavy ISP                                               A
                                                                                         M
              56k               384k                                                      S
 Access                                                    1.5m
                                  1 T1 consumer streaming 24/7 cost $388/mo in transit
                                                                                         Tier1 Equipment:
        AOL+ DSL: 1,000/day                                                            $13B/’00 -> $42B/’04
                                                                                        (Source: Infonetics)
        Roadrunner Cable Modems: 1M subscribers
                                                                          Negotiate Peering with Upstream ISPs
                                                                          Or focus on peering with like-minded ISPs…
  Why Peer? Motivations for Peering
• Financial: Reduce load on expensive Transit service
  • Traffic src/dest
  • Measure vs Intuit
  • Usage-based Billing
• Engineering: Lower latency              Transit
                                          $$$
                                 ISP A
               Seek transport
                                  x           Transit ISP
               Interconnection
                      $
                                 ISP B
1st Stage of Peering:                    Transit
  Top 10 destination ISP list           $$$
                                                     Top 10, Def: Peering…
Sample Top 10 Destination List
            In te rn e t S e rvic e P ro vid e r A
                                                                                          M ak e W e
  A S N u m b e r M b p s D e s tin a tio n IS P           C o n ta c t
            6 1 7 2 2 4 .3 5 H O M E -N E T -1             [H O M E -N O C -A R IN ]
              701     8 .9 0 A L T E R N E T -A S          [IE 8 -A R IN ]
            1668      8 .1 4 A O L -P R IM E H O S T       [A O L -N O C -A R IN ]
            4766      7 .0 8 A P N IC -A S -B L O C K      [S A 9 0 -A R IN ]
            3320      5 .1 2 R IP E -A S N B L O C K 4     [R IP E -N C C -A R IN ]
              577     4 .2 4 B A C O M                     [E Q -A R IN ]
            6327      3 .9 0 S H A W F IB E R              [IA S -A R IN ]
                 1    3 .8 9 B B N P L A N E T             [C S 1 5 -A R IN ]
            7018      3 .6 6 A T T -IN T E R N E T 4       [J B 3 3 1 0 -A R IN ]
            9318      3 .1 3 A P N IC -A S -3 -B L O C K   [S A 9 0 -A R IN ]
            5769      2 .6 7 V ID E O T R O N              [N A V 1 -A R IN ]
            6830      2 .3 0 H C S N E T -A S N B L K      [M D 2 0 5 -A R IN ]
            9277      2 .2 2 A P N IC -A S -3 -B L O C K   [S A 9 0 -A R IN ]
          10994       2 .0 8 T A M P A 2 -T W C -5         [J D 6 -A R IN ]
            1239      2 .0 5 S p rin tL in k               [S P R IN T -N O C -A R IN ]
                                                                                          Def: Peering
                Definition of Peering
Def: Peering is the business relationship whereby ISPs
reciprocally announce reachability to each others’ transit customers.

                 Peering                 Peering


      WestNet                 USNet                 EastNet


                Routing                            Transit
                Tables
   •Peering is *not* a transitive relationship
   •Peering *does not* provide access to the entire Internet

Usage: “I buy transit from UUNet and Peer with EastNet, WestNet.”
                                                        Cost Peering vs. Transit
                    The Cost of Peering
                                        Peering

         2500
         2000
$/Mbps




         1500
                                                                      Peering
         1000
         500
           0
                1
                    12
                         23
                              34
                                   45
                                         56
                                              67
                                                   78
                                                        89
                                                             100
                         Traffic Exchanged (Mbps)


                                                   1) Monthly Circuit Fee into IX
                                                   2) Monthly IX Port Fees
         When does Peering Make Sense
                 Financially?
                         Peering vs. Transit

         2500
         2000
$/Mbps




         1500                                     Transit $/Mbps
         1000                                     Peering
         500
           0
                1

                     6

                         11

                              16

                                   21

                                        26

                                             31

                    Traffic Exchanged (Mbps)
               Generalization:
   ISP Peering Breakeven Analysis Graphs
 $/Mbps
Exchanged




                  Breakeven Point
              (ISPs Indifferent between
                 Peering and Transit
                  traffic exchange)
      Peering                        Prefer Peering
       Risk
                                                        Cost of Transit
                         Cost of Traffic Exchange in Peering Relationship


              Number of Mbps exchanged
               Synch Point
• You should have
  – a working lexicon at this point
  – A sense of the motivations for Peering &
    Transit
• Any questions so far?
The Internet Peering Ecosystem

       Pre-Crash : circa 2000
         Peering Ecosystem
• System of interconnected (Peering and
  Transit) players
• Each group with homogenous peering
  motivations and observed behaviors
  – Tier 1 ISPs
  – Tier 2 ISPs
  – Content Providers
             Def: Tier 1 ISP
• Definition: A Tier 1 ISP is an ISP that has
  access to the entire Internet Region solely
  through peering relationships. (i.e. doesn’t
  buy transit from anyone to see regional
  routes)
• Motivation: Doesn’t need to peer with
  anyone else, so
• Peering only with other Tier 1 ISPs
            Def: Tier 2 ISP
• Definition: A Tier 2 ISPs is any ISP that has
  to purchase transit to reach destinations in
  an Internet Region
• Motivation: Peer to decrease costs, improve
  performance
• Peering: with Tier 1’s is preferable but hard
  to get, peering with Tier 2’s as appropriate
• Must purchase Transit
       Def: Content Providers
• Def: A Content Provider is an entity that
  produces content for delivery over the
  Internet but does not sell transit.
• Motivation: Best Customer Experience
  (engineered performance)
• Peering: Not typically done, prefer SLAs.
• Transit from ISPs
                           1998-2000

                                   Tier 1 ISPs
                       $
           Transit


                     Tier 2 ISPs

Thickness represents
In-Group Peering
                                          Content
The Evolution of the U.S.
   Peering Ecosystem

      Act II: 2000-2003
   Evolution of the U.S. Peering
            Ecosystem
Four key events leading to drastic disruption in the
   Peering Ecosystem:
1) The 1999/2000 Telecom Collapse
2) The growth of the Used Equipment Market
3) The Upstream Provider for the Cable Companies
   (@Home) went bankrupt
4) Peer-to-Peer file sharing systems (like Kazaa)
   grow in popularity, traffic grows exponentially
   between Access Providers (1.5MB MP3 
   700MB AVI files)
5) Transit Prices Drop, Transport Prices Drop
                                Leads to 3 Major Evolutions
    Evolution #1 – Cable Companies
              are Peering
•  40% of traffic Kazaa
•  3-5 Gbps of transit
                    T ransit P rovider
   traffic-> Gbps ofInterconnections
                    (U pstream (s))
   peering potential !
Significant because                                         P eering
1) Volume
2) Open Peering                          C able C om pany

3) Kazaa Effect
                     (O ptional)
                     B ackbone B ackhaul
U.S. Cable Companies Eyeballs
          Cable Modem Subscribers

 MSO               Country YE02       Change in 02
 Comcast           USA      3,620,300     1,199,100
 Time Warner       USA      2,613,000     1,027,000
 Cox               USA      1,407,900       524,400
 Charter           USA      1,180,000       572,300
 CableVision       USA        770,100       263,500
 Shaw (Big Pipe)   Canada     750,000 ?
 Rogers            Canada     650,000 ?
 Adelphia          USA        610,000       305,900
 Bright House      USA        490,000       159,000
 Mediacom          USA        191,000        76,000
 RCN               USA        160,400        49,800
 Insight           USA        144,800        56,700
 Cable One         USA         78,100        45,200
 Total                     12,665,600     4,278,900
                            Source: The Bridge & Leichtman Research
                            Source: I$P HO$TING ReportMay 2003, Volume VII, Num
                            Article "Dialup Bakeoff - Is it really so bad at EarthLink? O
                            Source: Bill Norton conversations with Rogers and Shaw 2
                                                                                   AOL



                       C ab le C o m p an y



                                                     C able C om p any




                                                                                     C able C om p any




C ab le C o m p an y




                                 C able C om p any
                                                                         C able C om p any
                           1998-2000

                                   Tier 1 ISPs
                       $
           Transit


                     Tier 2 ISPs

Thickness represents
In-Group Peering
                                          Content
2000-2003 Evolution #1 – Cable
      Companies Peering
                     Tier 1 ISPs


                 CableCos
                                      Peering
       Tier 2 ISPs




                            Content
 Evolution #2 – Network Savvy
 Large Scale Content Companies
        get into Peering
• To reduce transit costs
• To improve the end user experience
• They need to move out of bankrupt colo anyway
Significant because
1) Volume of traffic is huge
2) Content Providers have Open Peering
3) Leading players pave the way
                           1998-2000

                                   Tier 1 ISPs
                       $
           Transit


                     Tier 2 ISPs

Thickness represents
In-Group Peering
                                          Content
2000-2003 Evolution #2 – Large
 Scale Network Savvy Content
            Peers
                     Tier 1 ISPs




       Tier 2 ISPs

                                      LSNSC

                            Content
     Other Broadband Players
• Significant, but…
• Not Open Peers so disincentive to peer
• So less volume and less disruptive
Evolution #3 – Cable Companies
   Freely Peer with Content
          Companies
• Content literally on the Cable Company
  Network
• Lowest possible latency from content to
  eyeballs
• Internet Gaming, Broadband Streaming, etc.
                           1998-2000

                                   Tier 1 ISPs
                       $
           Transit


                     Tier 2 ISPs

Thickness represents
In-Group Peering
                                          Content
2000-2003 Evolution #1 – Cable
      Companies Peering
                     Tier 1 ISPs


                 CableCos

       Tier 2 ISPs




                            Content
2000-2003 Evolution #2 – Large
 Scale Network Savvy Content
            Peers
                     Tier 1 ISPs




       Tier 2 ISPs

                                      LSNSC

                            Content
2000-2003 Evolution #3 – Cable
Peers with Large Scale Network
        Savvy Content
                     Tier 1 ISPs


                 CableCos

       Tier 2 ISPs

                                      LSNSC

                            Content
           International Dynamics –
             Separate White Paper
          IIJ           NTT
                 T ier 1 IS P s         D SL
       K D D I C ab leC o s       JT




T ier 2 IS P s




                          C o n ten t



                            JP                       U.S.

   U.S. Tier 1 ISPs are Tier 2 ISPs in Japan Internet Region
   Japan Tier 1 ISPs are Tier 2 ISPs in the U.S. Internet Region
Q&A

				
DOCUMENT INFO