Internet2 QoS Architecture, Implementation, Future by andre12287

VIEWS: 19 PAGES: 52

									Internet2 QoS:
Architecture, Implementation, Future


     Ben Teitelbaum <ben@internet2.edu>
     January 12th, 2001
     UC Irvine
     Talk Overview

          0                                               Project Overview

                                                          Internet2 & Network
                                                          Research
                                                          Internet2 QoS


45                                       15               Abilene QoS

                                                          Future Directions

                                                          Questions


         30    Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   2
        Executive Overview

Elevator Explanation
  • Internet2's mission is to develop and deploy advanced
    network applications and technologies, accelerating the
    creation of tomorrow’s Internet
What are we really?
  • Membership organization of US research universities
  • Parent 501.3c (UCAID) has board of university presidents
  • Project supported by numerous partnerships (government,
    industry, international)
Goals
  • Enable new generation of advanced applications
  • Re-create leading edge R&E network capability
  • Transfer capability to global production internet
                         Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   3
                 Internet Development Spiral



Commercialization                                                Privatization




         Internet1

Research and                                              Internet2
Development                                                  Partnerships
Source: Ivan Moura Campos   Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   4
            Virtual Laboratories


Real-time access to
remote instruments

3-D Brain Mapping
(Pitt, PSC)
Distributed nanoManipulator
(UNC-CH)




                              Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   5
             Tele-immersion



Shared virtual
environments

University of Illinois
at Chicago

Virtual
Temporal                                                                                  Images courtesy
Bone                                                                                       Univ. of Illinois-
                                                                                              Chicago




                         Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   6
Tele-cubicles and the CAVE




                                        Source: University of Illinois-Chicago




          Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   7
                Distributed Computation


                                                                   Large-scale
                                                                   computation

                                                                   University
                                                                   Corporation for
                                                                   Atmospheric
                                                                   Research




Image courtesy of UCAR     Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   8
Internet2 Universities
179 Universities as of June 2000




                                       + over 30 affiliate members
                 Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   9
      Internet2 Corporate Partners

3Com                     Lucent Technologies
Advanced Network &       Marconi
Services                 MCI Worldcom
Alcatel                  Microsoft
Ameritech                Newbridge Networks
AT&T                     Netcom Systems
Cabletron Systems        Nortel Networks
Cisco Systems            Qwest Communications
IBM                      WCI Cable
ITC^Deltacom             + over 70 corporate members
                     Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   10
          International MoU Partners
          32 as of June 2000
AAIREP (Australia)                         INFN-GARR (Italy)
APAN (Asia-Pacific)                        Israel-IUCC (Israel)
APAN-KR (Korea)                            JAIRC (Japan)
ARNES (Slovenia)                           NORDUnet (Nordic countries)
BELNET (Belgium)                           POL-34 (Poland)
CANARIE (Canada)                           RCCN (Portugal)
CESnet (Czech Republic)                    RedIRIS (Spain)
CERNET, CSTNET, NSFC (China)               RESTENA (Luxembourg)
CUDI (Mexico)                              REUNA (Chile)
DANTE (Europe)                             RNP2 (Brazil)
DFN-Verein (Germany)                       SingAREN (Singapore)
Fundacion Internet 2 Argentina             Stichting SURF (Netherlands)
(Argentina)                                SWITCH (Switzerland)
GIP RENATER (France)                       TAnet2 (Taiwan)
GRNET (Greece)                             TERENA (Europe)
HEAnet (Ireland)                           JISC/UKERNA (UK)
HUNGARNET (Hungary)
                             Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   11
Abilene Topology                                             •47 connectors
January, 2001                                              •183 participants
                                        •34 connections to 20 peer networks




                Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   12
                  Developing International Peering


                      STARTAP Abilene   CA*net3           IUCC     GEMNET
                              vBNS      SURFnet           APAN     Renater
                              ESnet     MIRnet            CERN
                              DREN      NORDUnet          SINET                                   NYC
                              NREN      REUNA             Singaren                             (Telehouse
                              NISN                        Tanet                               25 Broadway)
 SEATTLE                                                                                                             DFN
                                                                                                                     INFN
 Abilene                                                                                                             DANTE
 ESnet                                                                                                               CA*net3
 CA*net3
 AARnet
 CERNET                                                                                            NYC
                                                                                               (60 Hudson)            CERN
                                                                                                                      SINET
                                                                                              Abilene      JAnet
 LA                                                                                           ESnet        SURFnet
                                                                                              vBNS         NORDUnet
  Abilene    CUDI
  Singaren
             HARNET
             SINET
                                                                                        Miami
                                                                                                      Argentina
                                                                                                      Brazil
                                                                                                      Chile
                                                                                                      Columbia
                                                                                                      Brazil
Courtesy: Linda Winkler, STAR TAP
                                        Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   13
Abilene Load Snapshot




          Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   14
         New Network Capabilities

Quality of Service
  • http://www.internet2.edu/qos/

Multicast
  • http://www.internet2.edu/multicast/

IPv6
  • http://www.internet2.edu/ipv6/

Distributed Storage: I2-DSI
  • http://dsi.internet2.edu/

Middleware
  • www.internet2.edu/middleware/


                                Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   15
     Internet2 & Network Research...

          0                                                Project Overview

                                                           Internet2 & Network
                                                           Research
                                                           Internet2 QoS


45                                        15               Abilene QoS

                                                           Future Directions

                                                           Questions


          30    Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   16
        Network Research Liaison Council

NRLC is charged with guiding Internet2's
support for networking research
  • Dave Farber, Len Kleinrock, Larry Landweber, Dave
    Meyer (chair), ...
Internet2 aims to support the networking
research community by...
  • Increasing the awareness of Internet2's advanced capabilities
  • Facilitating access to existing and, where required, new network
    facilities and capabilities
  • Collaborating with the researcher to attract research support from
    research agencies and corporations associated withInternet2
  • Building a highly instrumented network infrastructure with open
    access to measurement data
                            Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   17
       NRLC CFPs and Workshops


In Spring 2000, NRLC issued CFP for
research projects that could benefit from:
  • Increased support from Internet2 staff;
  • Increased access to Internet2 network facilities.

Nine projects selected
Project PIs invited to June, 2000 workshop to
discuss needs and benefits of partnership
NRLC is planning second CFP and second
workshop (April 18-19, 2001)
                         Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   18
             Internet2 Network Research
             Partnerships

Bridging the scalability and reliability chasm         Craig Labovitz (Microsoft Research)
Wide area application performance                      Paul Barford (Boston University)
measurement in Internet2
Web100: Facilitating effective and                     Basil Irwin (NCAR)
transparent network use
Access/VCAIRN: Overlay testbed for                     Allison Mankin (USC/ISI)
computer science experimenters in Internet2
Intelligent Internet2 Agents for distributed           Yanquing Zhang (Georgia State)
data mining
Resource negotiation and pricing protocol              Xin Wang (Columbia University)
QoS amplification experiments                          Kihong Park (Purdue University)
Simple active networks and predictable                 Ashvin Goel (Oregon Graduate
streaming applications over diverse                    Institute)
networks
Scalable streaming media delivery                      Mary Vernon (University of
                                                       Wisconsin)




                                       Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   19
     Internet2 QoS...

          0                                                 Project Overview

                                                            Internet2 & Network
                                                            Research
                                                            Internet2 QoS


45                                         15               Abilene QoS

                                                            Future Directions

                                                            Questions


          30     Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   20
     How We Got Here (short version)

• Began chanting:
   “enable advanced applications,…”
• Assessed requirements
• Recommended DiffServ
• Selected “Premium” service to meet
  demands of loss/jitter sensitive apps
• Charted QBone initiative                                                    architecture
• Specified QBone architecture
• Now proceeding to implement it and
  tweak the architecture
                                                                              deployment


                      Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   21
IntServ/RSVP vs DiffServ


   IntServ/RSVP
           •Per-flow service state at every hop
           •Scalability problems
           •Focus on multipoint multicast




         BB                                       BB



   DiffServ
      •Abstract/manage each cloud’s resources (BBs)
      •Packets colored to indicate forwarding “behavior”
      •Focus on aggregates not individual flows
      •Policing at Internet2 QoS:get services
                   edge to Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   22
       DiffServ Overview

Applications contract for specific QoS profiles
  • Policing at network periphery
  • “Color” packets with a few simple, differentiated per-hop
    forwarding behaviors (PHBs)
     – Indicated in packet header
     – Applied to PHB traffic aggregates
  • PHBs + policing rules = range of services
DS domains contract with each other for
aggregate QoS traffic profiles
  • Policing at cloud-cloud boundary
  • Supports simple, bilateral business agreements
Exploits edge/core distinction for scalability
                           Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   23
       Example Service #1: Premium


Assurance: like a leased line
PHB: Expedited Forwarding (RFC 2598)
  • EF in separate queue configured with minimum
    departure rate
  • Example mechanisms: strict priority, MDRR, WFQ
Policing: police to a specified peak rate
and drop out-of-profile packets; effectively
a leaky bucket with depth 1 MTU


                      Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   24
       Example Service #2:
       Controlled Load

Assurance: network looks “lightly-loaded”
for conforming traffic
PHB: Assured Forwarding (RFC 2597)
  • 4 independent AF classes
  • 3 drop preference levels within each class
  • Example mechanisms: WRED, WFQ

Policing: police to specified rate and burst
profile, remarking out-of-profile packets to
have higher drop probability

                        Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   25
      Example Service #3: CoS


Assurance: “better than Joe”
PHB: “drop the lower classes first ” (AF or
class selector PHBs)
Policing: could be based on anything
(e.g. higher priority for the CEO)
A.K.A.“Olympic” classes of BE service
(e.g. Gold, Silver, Bronze)


                   Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   26
          QBone Architecture

A Service: QBone Premium Service
  • Built on Expedited Forwarding (EF) (RFC 2598)
  • Assurance: near-zero loss & low, bounded jitter for
    marked traffic conforming to a specified peak rate
     – a.k.a. “virtual leased line”, “virtual wire”

Reservation Setup Protocol
  • Now: long-lived, manual setup
  • Proposed: SIBBS protocol between QBone domains;
    RSVP end-to-end between hosts
QBone Measurement Architecture
  • Uniform collection of QoS metrics
  • Uniform dissemination interface
                                 Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   27
             QBone Measurement Architecture1/2

Collection                                      Active                                                  MIB-based
  •  metrics, EF and BE...                  Measurements
                                                                             Boundary
                                                                                                         statistics

  • Active metrics (paths)                      AM node
                                                                              Router

      – One-way delay-variation           Intra-Domain                                                Inter-Domain
      – One-way loss                      Premium Path                                                Premium Path
      – Traceroutes
      – e.g. IPPM Surveyors                      PM node                                                 PM node
  • Passive metrics (interfaces)
      – Load                                   Passive                                                 Passive
      – EF reservation load                  Measurements                                            Measurements

      – Discards (suggested)
      – Link bandwidths (suggested)                                         QBone
      – e.g. OCxMon, RTFM, MIBs                                             Domain2
                                                QBone                                                   QBone
                                                Domain1                                                 Domain3
                                   Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   28
        QBone Measurement Architecture2/2

Dissemination
  • Standard URL query syntax:
    label ::= <alphanum> { <alphanum> }
    router ::= label"-ROUTER”
    probe ::= label"-PROBE”
    sniffer ::= label "-SNIFFER”
    host ::= router | probe | sniffer
    path ::= host "/" host
    PHB ::= "BE" | "EF”
    metric ::= "LOSS" | "ONEWAY" | "PING" | "IPDV" | "LOAD" | "TRACEROUTE" |
    "COMMITMENT" | "RESERVATION"
    year ::= digit digit digit digit
    month ::= "01" | "02" | ... | "12”
    day ::= "01" | "02" | ... | "31”
    YYYYMMDD ::= year month day
    aggregation ::= <unsigned_integer>
    prefix ::= “http” | “ftp” | <other>
    query ::= prefix ”://" path "/" PHB "/" metric "/” YYYYMMDD "/" metric "/" aggregation
  • whois server to learn canonical names for QBone
    domains, routers, sniffers, etc
                                   Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   29
          QBone E2E Picture

 Campus
   A

          GigaPoP
             A

 Campus
   B                     Backbone
                                                                                               Campus
                                                                                                 C

                                                                   GigaPoP
Key
                                                                      B
X Kbps of QPS from
hither to………..yon                                                                              Campus
                                                                                                 D
                     Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   30
       SC2000 Interdomain QoS Demo
       November 6-9, 2000



Premium service over two wide-area paths
  • LBNL-ESnet-Abilene-SCinet-Internet2 booth
  • Stanford-CalREN2-Abilene-SCinet-Internet2 booth
Congestion induced at multiple points
CD-quality interactive audio application
shown with/without QoS
ESnet and Abilene QoS capabilities will form
nucleus of QBone
SC2000 Network Challenge Winner: Award
for "Most Captivating and Best Tuned" Demo
                            Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   31
Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   32
     Abilene QoS...

          0                                                Project Overview

                                                           Internet2 & Network
                                                           Research
                                                           Internet2 QoS


45                                        15               Abilene QoS

                                                           Future Directions

                                                           Questions


          30    Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   33
        APS Participation

Goal:
  • Make APS a reference implementation of the QBone
    architecture
Current Participants
  • MAGPI (U. Penn)               • ANL
  • iCAIR                         • UIUC
  • PSC (Penn State)              • DOE Science GRID (peering
  • OARNet (Ohio State)             transit network)
  • NASA EOS
Others in the wings
  • TF-NGN (through DANTE)
  • MIRnet
  • Various other international
                        Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   34
            Initial Engineering Plan (obsolete)


Sweetwater Midland                      Odessa                                       Pecos

Measurement (Surveyor + SNMP + HTTP)
Edge Policing (“Firehose” CAR)
Manual Setup (Whiteboard + CLI)
                 EF Core Forwarding (MDRR)
                                         EF Edge Forwarding (MDRR)
                                        Automated Setup (BB)
                                                                                      Shaping (GTS)




                                  Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   35
       APS Measurements


Status:
  • Collecting EF/BE loads and CAR conform/exceed stats
  • Not currently monitoring IPDV, but Abilene Surveyor
    nodes now OC-3 connected and operational
  • Ohio-ITEC hosting APS measurements and QBone-
    wide whois server

Near Future:
  • IPDV along edge-to-edge QBone paths
  • Abilene Surveyor timing improvements
     – Better NTP
     – New CDMA timing sources (can't get GPS in Qwest POPs)
  • Collection of AS-level traffic matrices
                         Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   36
        Committed Access Rate (CAR)


Classifies traffic based on certain matching
criteria (interface, DSCP, or ACL) and meters
it to a leaky bucket traffic profile
Depending on metering result, different
actions applied (drop, transmit, set DSCP,…)
Syntax:
rate-limit {input | output} [access-group [rate-limit]
 acl-index] bps burst-normal burst-max conform-action
 action exceed-action action


                        Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   37
        CAR Experience


For the most part, CAR is exactly what the
DiffServ doctor ordered
However, there are some limitations…
  • PPS performance cost
  • Quirky constraints on token bucket depths
  • Not easy to do "virtual trunk" style classification




                          Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   38
       CAR Limitation 1: Performance


Problem:
  • On E0 GSR edge cards, lack of ASIC support for CAR
    results in non-trivial pps hit

Solution:
  • Really out of our hands; must wait for E3 edge cards,
    which will have CAR in hardware
  • Load on access interfaces is still light, so performance
    hit not a big issue for current participants




                         Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   39
     CAR Limitation 2:
     Policing Granularity

Problem:
  • Cisco CAR Doc:
     – “burst-normal Normal burst size in bytes. The minimum
       value is bps divided by 2000”
  • i.e. burst-normal  max (mtu, bps/2000)
  • But, QPS demands token bucket depth of 1 MTU!
  • Implication: bps  3Mbps (for Ethernet MTU)

Solution:
  • Again, out of our hands
  • Have raised concern with Cisco and are hoping E3
    cards will address this limitation too
                       Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   40
       CAR Limitation 3: Classification


Problem: we want...
                                                                                           ("virtual
                                                                                           trunk")

But this...                                   …is what we have
                                              ("firehose")

Solution: with additional rope,CAR can also
classify by qos-group (Cisco proprietary)
  • Packets assigned to QoS groups through QoS Policy
    Propagation via BGP (QPPB)

                      Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   41
      Abilene Architecture Limitation:
      “Porous” Edge Problem

DoQoS problem with current architecture
MDRR (EF forwarding) on interior interfaces
easily subverted by unpoliced connectors




                  Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   42
       How to “Crisp” the Edge?


Problem: EF requires that all connectors be
policed
Solution:
  • Short term: Stochastically detect illegal EF traffic with
    NetFlow and/or OCxMon passive monitoring; gigaPoP
    would be asked to police
  • Longer term: Wait for E3 edge cards, deploy them
    aggressively, and police everywhere
  • Road not taken:Re-write all non-participant traffic with
    DSCP 000000 (using PIRC hack); need to pass some
    DSCP values (reason why coming up…)

                         Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   43
            Current Engineering Plan


Sweetwater Midland                      Odessa                                       Pecos

Measurement (Surveyor + SNMP + HTTP + whois + traffic matrices)
Edge Policing (CAR + QPPB)
Manual Setup (Whiteboard + CLI)
                 EF Core Forwarding (MDRR)
                                         EF Edge Forwarding (MDRR)
                                        Automated Setup (SIBBS? + DSTE?)
                                                                                      Shaping (GTS)




                                  Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   44
     Future Directions...

           0                                                Project Overview

                                                            Internet2 & Network
                                                            Research
                                                            Internet2 QoS


45                                         15               Abilene QoS

                                                            Future Directions

                                                            Questions


          30     Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   45
       QBone Premium Service Outlook

Good News:
  • DiffServ functionality in most modern routers
  • Many hosts support QoS signaling
  • Lots of isolated testbed trials
  • Some partial backbone implementations
Bad News:
  • Deploying QPS "requires upgrading the world"
  • Low demand for QPS (app/net chicken/egg dynamic)
  • Vendor implementations don’t always live up to hype
  • Requires a lot of elbow grease, which is in short supply
    in most campus IT organizations
Bottom line:
  • Progress can be made towards end-to-end real-time
    services (QPS), but it is going to be slow
                        Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   46
       Looking forward...


QBone Premium
  • Revise/complete architecture (joint work with TF-NGN)
  • Make measured progress on deployment
  • Focus on ad hoc QoS solutions where it needed, but
    deploy in a way consistent with QPS architecture

Big measurement push
  • QBone measurement architecture
  • Broader Internet2 measurement push
  • E2E Performance Initiative
Reap lower-hanging fruits of DiffServ
  • Internet2 Scavenger Service (I2SS)
                       Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   47
       Internet2 Scavenger Service


Basic idea
  • A lower priority class of best-effort
  • Voluntary marking hints to network that degraded
    service is OK (think of a "nice" for the network)

Intended uses
  • Non-time-critical traffic (e.g. server-to-server NNTP,
    anonymous FTP, network backups)
  • Bulk data transfers using TCP
  • Non-mission applications (e.g. Napster, games, etc)
  • New kinds of distributed applications that attempt to use
    idle network capacity
                        Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   48
       I2SS Service Specification

Rigorously defining the E2E I2SS is difficult!
  • Hoping to define relative to best effort
I2SS traffic indicated by DSCP 001000
  • Modification of class selector PHB
  • Note that the I2SS codepoint has global significance
I2SS domain requirements
  • Traffic leaving must be marked I2SS, if it entered so
Router requirements (all SHOULDs)
  • Forward I2SS independently giving it a lower probability
    of timely forwarding OR forward in same manner as BE
  • Offer I2SS a very small minimum departure rate
  • Offer I2SS all un-used bandwidth
                         Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   49
     Any Questions?

          0                                               Project Overview

                                                          Internet2 & Network
                                                          Research
                                                          Internet2 QoS


45                                       15               Abilene QoS

                                                          Future Directions

                                                          Questions


         30    Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   50
       For more information...


Internet2 Home:
  • http://www.internet2.edu/
Internet2 QoS Working Group Home:
  • http://www.internet2.edu/wg/qos/

QBone Home:
  • http://qbone.internet2.edu/

Abilene Premium Service Home:
  • http://www.internet2.edu/abilene/qos/



                         Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   51
Internet2 QoS: Architecture, Implementation, Future—UC Irvine (January 12th, 2001)   52

								
To top