20080122 bobyshev demar crawford by HC12083104287

VIEWS: 1 PAGES: 17

									Fermilab




              Lambda Station
                 Project
                  http://www.lambdastation.org


           Andrey Bobyshev; Phil DeMar; Matt Crawford
             ESCC/Internet2 Winter 2008 Joint Techs
                    January 22; Honolulu, HI
Will


           replace

                       recycled
 as the most commonly purloined
 presentation prop?
Outline of the talk

   Goals and major directions of the project

   Software architecture, API, SOAP/XML, some details

   How Lambda Station works - use cases

   Status and the near-term plans
The goal of the project...

   Target: last-mile problem between local computing
    facilities and alternate WAN paths
   Strategy: develop a network path selection service to
    interface production storage & computing facilities with
    wide-area network paths.
       alternate network paths for high impact data movement
       selective forwarding on a per flow basis
       graceful cutover and fallback on path changes
       on-demand from applications (authentication & authorization)
       current implementation based on policy-based routing &
        including the support of DSCP marking
               Software architecture
                                          Storage & application space
Management


                                                                                     Remote Lambda Station
                                                                           SOAP            SOAP/Clarens
SOAP/Clarens
                                                                                          LSInterface
               LS-Management &
                                                 LSInterface
               Reporting Interface
                                                                        Service-based Architecture:
   local
                 LSDIScovery                                             CLARENS is framework for service-based
 definitions                                     mySQL:requests
                   Service                                              architecture, mutual authentication of
                                                 , history,security
  online                                                                requests
 updates
                LSRESource                                               LSController – synchronizes work of all

                Scheduler                          LSController         services, has control functions
                                                                         LSInterface - unified interface for


                                                                        intercommunication between LS and
                          NETWORK CONFIGURATOR                          applications, and LS-to-LS
Vendor specific                                                          LSDIScovery service – detects new
                       CISCO              Force10           WAN
   modules                                                              lambdastations, and PBR clients at remote
                                                                        and local sites
     Data Exchange                                                       LSRESource Scheduler – estimates


  Control & Management               Si
                                          Si
                                                                        bandwidth allocation, monitor real-time
                                                                        usage of resources
                                                                         Network Configurator – dynamic


                                                                        reconfiguring of LAN and WAN
For more details...

   Check references at www.lambdastation.org
       Papers published at CHEP04, CHEP06, Gridnet2006,
        CHEP07
       Demonstrations at SC05, I2 FMM 2007; SC07



     Instead, we would like to walk you
    through a few the real world use cases
                                                  LambdaStation
   Internet2/ ESNet
   Dynamic Circuits

                                   WAN                             UNL Tier2

           OSCARS                                          LambdaStation

                                     UltraLight




                                                        Caltech Tier2
LambdaStation
@ Fermilab



                                                                  Normal traffic flow
                                                                  High Impact traffic
                                                                  Control messages
                US CMS Tier 1 Facility                            Network configuring
Use case 1: Alternate traffic forwarding on-
demand of Data Movement Applications
(dCache/SRM)
US CMS Tier1 Center                                                                                      Tier2 Facility
                                              High Impact Data Network
                                                                                                                   Site Lambda Station
Site Lambda Station




                                                                                                                           dCache/SRM
dCache/SRM



                                                                                                  Network
                                                                                                  infrastructure
                      Network
                      infrastructure
                                                    Routed R&E Networks
Fermilab                                                                                                                    Caltech

                       Default network path                                              Control plane
                       An alternate, high-impact path


  Two sites can exchange traffic via several networks. Each site has a Lambda Station to steer selected flows into an alternate path
  on-demand of applications. Site networks can be dynamically reconfigured by Lambda Stations or statically pre-configured. If
  supported by site network infrastructure or/and by applications QoS ToS or DSCP can be used to match selected flows.
 Sequence Diagram - Use Case 1
 (simplified)
    SRM@FNAL                   LS@FNAL               NET@FNAL       NET@CALTECH                   LS@CALTECH          SRM@CALTECH
                                                         Data Movement

                Request BW             Reservation
                    Local ID
                                                             Request BW/path

                                                               Remote ID
                                                                ACK

                                    Configuring Network                            Configuring Network
                   Active
                                            OK
                                                                                           OK
 High BW
                                                                Status?
  path

                  Ticket expired                                    OK

                                     Re-conf to default                              Re-conf to default


                                                                  Status?
                                                                  Completed



All time through the life cycle of reservation each site checks periodically the corresponding reservation at remote site. If
an error is detected, local reservation will be cancelled.
Use case 2: Setup alternate traffic path between
FNAL and UNL via Dynamic Circuit based on
flow analysis.
 Flow analysis   Lambda Station       IDC/ESNet        IDC/Internet2
                    Server
                                                                                              Lambda Station Server




             Flow
                                     ESnet OSCARs     Internet2 DCS
             data




                                             Routed R&E Network              Network
                                                                             infrastructure
                    Network
                    infrastructure
                                                                                                  University
                                                                                                 of Nebraska
  Fermilab

                                                                       Control plane
                      Default network path
                                                                       Circuit call setup & teardown
                      A path across DCN                                LAN reconfiguration to use circuit
                                                                       Inter-Lambda Station coordination
                                                                       FtWatch request/response
 Sequence diagram - use case 2 (simplified)
ftwatch           LS@FNAL        net@FNAL               net@UNL                           UNL LS
                                                                            IDC/I2
      openSvcTicket
                                     OpenSvcTicket & call IDC

                                         Remote ID
                                                                               Reserve/Activate


                                                                                     OK
                                         Status ?
                                            OK
                          netconf                                 netconf

                            OK                                     OK
    Path is established




                          netconf                                 netconf



                                                                                     teardown
Use case 3: Sinking traffic via DCN or High
         Impact Networks

                                        IDC-A                        IDC-B

                                                Dynamic Circuits Networks
                                                  Domain A        Domain B

 Site Lambda Station
                                                High Impact Data Network


                                                       General Internet


                                                                                     Network
                                                                                     infrastructure
                       Network
                       infrastructure

 Site A                                                                                                    Site B

                                                                             Control plane
                         Default network path                                Circuit call setup & teardown
                         Forward path across DCN                             LAN reconfiguration to use circuit
                                                                             Application-initiated request/response
                         Reverse path across general Internet
Summary – Current Status

   Functional system deployed at FNAL, UNL, & Caltech
   Application LS awareness - dCache/SRM
   Primitive protocol to request a DC via ESNet/Internet2
   Successful demonstration SC05, I2 FMM, SC07
   (Sort of…) Production use with US CMS Tier1, Tier2s
    at UNL and Caltech
   Flow analysis to invoke a Lambda Station ticket based
    on observed traffic
Plans

   Improve code to make production quality
   Support multi-platform clients
   Enhance protocol for interaction with OSCARs/ESnet &
    DCN/I2
   Improve distribution package
   Develop management/configuring tools
   Interface with monitoring system
   Documentation
Large-scale data recover via DCN


   One week after I2 FMM
    demo, UNL lost their Tier-2
    data cache

   50TB Data recovered by
    transfer FNAL Tier-1:
       Largely via Internet2/ESNet
        Dynamic Circuit
       Completed in 32 hours
  Progress Chart of Lambda
Station Request in SC07 Demo
The END

								
To top