OSPF Validation

Document Sample
OSPF Validation Powered By Docstoc
					OSPF Validation

       Dirk Jacob

  Conformance Testing


  OSPF Test Suites

  Implementation for SSFNet


                              Dirk Jacob, Page 2

  Conformance Testing


  OSPF Test Suites

  Implementation for SSFNet


                              Dirk Jacob, Page 3
SSFNet - Overview

 Simulation of very large networks

 Focus on scalability
    modeling scalability
    computing scalability

 Layered Architecture

                                      Dirk Jacob, Page 4
Scalable Simulation
  Simulator Kernel – not dedicated to simulation of networks

  „Models“ are built on top of the SSF API
     consists of only five core classes:
      Entity, Event, inChannel, outChannel and process
     API hides all simulator internals
     C++ and Java bindings
     models are standard programs which extend the standard SSF classes

  Different SSF implementations available

                                                                     Dirk Jacob, Page 5
SSF Network Models

 Set of SSF Java models, which simulate the networking world

 distributed under the GPL

 consists of several sub-packages
    SSF.OS
     Framework for protocol modeling
    SSF.Net
     models for hardware components
    SSF.Util.Random
     generation of multiple independent random number streams
    SSF.Util.Streams
     efficient multi-point monitoring infrastructure

                                                                Dirk Jacob, Page 6

 Framework for modeling of protocols

 three classes:
    ProtocolSession,
    ProtocolMessage,
    ProtocolGraph

 Packets are modeled by ProtocolMessages

                                            Dirk Jacob, Page 7
SSFNet Protocols

  SSFNet comes with a variety of protocols

     IP
     UDP
     TCP
     Sockets
     HTTP

     BGP
     „Static“ OSPF
     OSPFv2

                                              Dirk Jacob, Page 8

 Models for the simulation of networking components

    Networks
    Hosts
    Routers
    Links
    Network Interfaces (NIC)
    additional Classes like packet queues, routing tables, etc.

                                                                   Dirk Jacob, Page 9
Modeling Topologies

  Topologies are modeled using DML
     consists of nested key/value pairs
     supports attribute substitution
     supports inheritance
     component configurations can be kept in a dictionary for reuse

  Example
  dict [                                  Net [
    myrouter [                              router[ id   1
      interface [ id 0                        _extends   .dict.myrouter ]
        _extends .dict.if100Mbit]           router[ id   2
      graph [                                 _extends   .dict.myrouter ]
        ProtocolSession[name ip
          use SSF.OS.IP]                      link[ attach 1(0) attach 2(0) ]
        ProtocolSession[name ospf         ]
          use SSF.OS.OSPF.sOSPF]
                                                                       Dirk Jacob, Page 10
Role of Testing

  Protocols must be implemented correctly

  Conformance tests to prove conformance with standards

  Regression tests to verify that things still work after changes

  Testing is extremely important!

                                                          Dirk Jacob, Page 11

  Conformance Testing


  OSPF Test Suites

  Implementation for SSFNet


                              Dirk Jacob, Page 12
Categories of Testing

  Conformance testing

  Regression testing

  Performance testing

  Stress testing

  Roustness tests

  Here: focus on conformance testing

                                        Dirk Jacob, Page 13
Conformance Testing

 Verify that a protocol implementation performs as required
  in the standard

 Black box testing: observe outputs, when IUT is feeded with
  various inputs

 Exhaustive testing: apply every possible input sequence
    not possible with complex protocols

 Test only important input sequences

 Main problem: identify important input sequences

                                                      Dirk Jacob, Page 14
OSI Testing Methodology

 Framework for conformance testing of OSI protocols

 Standardized in ISO 9646

 based on the OSI Reference Model

 Standardized test procedure

                                                   Dirk Jacob, Page 15
Test Architectures

  Local Method   Distributed Method

                                  Dirk Jacob, Page 16
Test Architectures (2)

  Coordinated Method   Remote Method

                                   Dirk Jacob, Page 17

 standardized notation for the description of test cases

 event trees, which describe the external behavior of a protocol

 two forms: TTCN/gr and TTCN/mp

 four parts                    LT!SYN_Packet
    overview part                  START wait_timer
    declarations part                  LT?ACK_Packet
    constraints part                       LT!FIN_Packet
    dynamic part                       TIMEOUT wait_timer
                                     Example for dynamic part

                                                            Dirk Jacob, Page 18
Test Case Selection

  not specified in OSI framework

  but: central role for coverage and correctness of test suite

  efforts to use formal techniques
     provable correctess
     provable coverage

                                                          Dirk Jacob, Page 19
Test Selection based on
  several methods for test case selection based on FSMs
      transition tours
      distinguishing sequences
      characterizing sequences (W-Method)
      unique I/O sequences

  apply an input sequence which must produce
   a specific output sequence
  protocol must be specified as an FSM
  restrictions
      strongly connected
      fully specified

  not applicable to most complex protocols            Dirk Jacob, Page 20
Formal Description
  data portion of protocols often are not specified as FSM

  FDTs are formal languages for specification
     SDL
     MSC
     LOTOS
     Estelle

  specifications can be represented as labelled transition systems
     similar to FSMs
     different methods/algorithms for derivation of test cases from LTS

                                                                       Dirk Jacob, Page 21
Informal Testing

  many protocols are specified using natural language

  translating them into a FDT may cause errors
     test cases may not reflect the standard

  many formal methods cannot be applied to complex protocols

  often informal methods are used
     reading the standard and deriving tests directly
     no provable correctness and coverage
     but: „high degree of certainty“

                                                         Dirk Jacob, Page 22

  Conformance Testing


  OSPF Test Suites

  Implementation for SSFNet


                              Dirk Jacob, Page 23
OSPF Overview

 Link-state intra-domain routing protocol
    OSPF Router knows the whole topology of the network

 Specified in RFC 2328

 Routing Information is exchanged with directly attached neighors
    flooding procedure assures distribution over the whole network

 Hierarchical Concept
    support for large networks
    reduce CPU and memory requirements

 Support for multiple least cost routes to the same destination

 Uses Dijkstra‘s SPF algorithm to calculate routing tables
                                                                      Dirk Jacob, Page 24
OSPF Overview (2)

 Routers identify their neighbors
 Some neighbors form adjacencies in order to exchange
  routing information
 Synchronization through the exchange of link state databases
 Detect link failures through periodic sending/receiving of
  Hello Packets
 Changes in the topology are flooded as Link State Updates
 All information is re-flooded periodically to keep databases

 OSPF distinguishes different network types:
 broadcast, nonboadcast-multiaccess (NBMA), point-to-point and point-to-multipoint
                                                                    Dirk Jacob, Page 25
Hierarchical Routing

  each AS can be divided into Areas
  each area has ist own Link State Database
  detailed information of area topology only known inside the area
  information about area is summarized into other areas
  special backbone area (area 0)
  all areas must be connected to the
  virtual links to connect areas
   to the backbone which have no
   physical connection
  Stub areas to reduce database
                                                       Dirk Jacob, Page 26
Link State Database

  Contains information about area topology
  Describes a graph
     Routers and networks are vertices
     Links between vertices are edges
     Links are asociated with a cost/metric
  Network vertices are represented by a designated router

                                                       Dirk Jacob, Page 27
Functional Areas

  4 mostly independent functional areas

     detection and maintenance of neighbors

     building adjacencies

     the flooding procedure

     routing table calculation

                                               Dirk Jacob, Page 28
Neighbor Discovery and
  Send Hello Packets periodically
     Include information about neighbors

  Assure bi-directional communication

  Assure that neighbors agree on certain parameters

  Elect Designated Router on broadcast and NBMA networks
     Represent the network in
      the Link State Database
     reduce the number of

                                                       Dirk Jacob, Page 29
Building Adjacencies

  neighbors with bi-directional communication become adjacent
     dependent of network type

  Synchronization of Link State Databases
     Negotiate, who is the master in this process
     Master controls the exchange process
     Send summary of own Database in Database Description Packets
     Request unknown or more recent LSAs from the neighbor
     Send requested LSAs to the neighbor

                                                                     Dirk Jacob, Page 30
Reliable Flooding

  Each router must keep synchronized with every other router in
   the same area

  Udates are flooded to all adjacent neighbors

  Every Update must be acknowledged
     Explicitly through an Ack packet
     Implicitly with an Update of the same LSA flooded back

  Retransmit Update if there is no such acknowledgement

  Possibility of delayed Acks to minimize number of packets

                                                               Dirk Jacob, Page 31
Link State Advertisements

  different types of LSAs
     Router LSAs – represent a router with all its associated links
     Network LSAs – represent broadcast and NBMA networks
     Network Summary LSAs – represent routes to destinations in other areas
     ASBR Summary LSAs – represent routes to AS boundary routers
     AS-external LSAs – represent routes to AS external destinations

  LSAs have an age associated with them
     aging in databases and during the flooding procedure
     premature aging to remove LSAs from databases
     don‘t use LSAs that are older than 1 hour for routing calculation
     refresh own LSAs that are older than ½ hour

  use of sequence numbers to determine, which instance of an
   LSA is the most recent one
                                                                          Dirk Jacob, Page 32
Routing Table Calculation

  Dijkstra‘s Shortest Path First algorithm

  Calculate SPF tree for the area using the SPF algorithm
     Consider only routers and network nodes from the LSDB

  Add stub networks to the tree

  Add inter-area routes to the tree

  Examine summary LSAs to find better routes for destinations
   in areas connected through virtual links

  Add AS external routes to the tree

                                                              Dirk Jacob, Page 33
Testing OSPF

 take advantage from the functional structure
    four smaller testing problems instead of a big one

 each functional area can be tested independently

 take care of dependencies
    functional hierarchy rather than completely independent
    test „lower level“ functionality first

                                                               Dirk Jacob, Page 34

  Conformance Testing


  OSPF Test Suites

  Implementation for SSFNet


                              Dirk Jacob, Page 35
OSPF Test Suites

  most of the formal methods require protocol specification
   with FSMs

  OSPF only partially specified by FSMs
     many parts like LSA origination not covered
     most parts are specified using natural language

  OSPF consists of multiple concurrent processes

  Formal methods require formal specifications
     not provided for OSPF, must be derived manually from standard

  Formal methods difficult to apply to OSPF
                                                                      Dirk Jacob, Page 36
C-TTCN Based Test Suite

 approach to testing OSPF with formal methods

 modeling test cases with C-TTCN
    extension to TTCN which allows modeling of concurrent behaviors

 OSPF specification was translated into a FDT called CEBE
    based on LTS, which are similar to FSMs

 resulting test suite had over 4000 test cases

 after elimination of unimportant test cases: 543 test cases

 Problem: how to prove that formal specification covers the
  complete OSPF specification?
                                                                  Dirk Jacob, Page 37
IOL Test Suite

  IOL is a test lab, whose services are used by many vendors

  Test suites are derived by „carefully reading the standards“

  good coverage and correctness
     used by many vendors
     refined over the time

  covers the functional areas of OSPF presented before
     Hello Protocol tests
     Fooding and Adjacency tests
     Link state advertisement tests
     Route calculation tests
     Additionally: Configuration and Formatting tests

  78 test cases with several test steps each             Dirk Jacob, Page 38
Commecial Test Products

    software solution for conformance and performance testing
    used by many well-known vendors
    provides automated testing
    OSPF test suite consists of over 300 test cases from 12 groups
    no formal methods are used

 QARobot / RouterTester
    hardware solution together with protocol and test suite software
    conformance and stress testing
    OSPF test sute has 70 test cases from 12 groups
    no formal methods are used

                                                                        Dirk Jacob, Page 39
Testing OSPF in Practice

  in practice: mixture of several different test products and/or
   lab testing

  Example: ZebOS Advanced Routing Suite
     basic testing with ANVL
     additionally performance testing using other test tools
     after that: lab testing at the IOL and other labs

  Testing against multiple test suites increases probability of
   finding any offences against the specification

                                                                Dirk Jacob, Page 40

  Conformance Testing


  OSPF Test Suites

  Implementation for SSFNet


                              Dirk Jacob, Page 41
OSPF Test Suite for SSFNet

  OSI methods can be easily transferred into SSFNet
  Test Procedure needs not to be modified
  Extended test architecture to support distributed test behavior

  Test cases can be modeled using appropriate DML scenarios
  some test cases need implementation of specialized testers
                                                         Dirk Jacob, Page 42

 Goal: implement a complete OSPF for SSFNet

 Must correctly implement the standard

 until now:
    Hello Protocol, Database Exchange, Flooding and Routing Table Calculation
    only Router LSAs
    restricted to point-to-point networks
    no vitual links or authentication

 work still in progress

                                                                   Dirk Jacob, Page 43
Test Selection

  depends on several criteria
     correctness
     coverage
     relevance and acceptance in practice
     ease of implementation
     applicability early during development
     availability

  formal methods are difficult to apply and not very common

  dedicated test equipment not applicable easily

  software solution like ANVL also not applicable

  IOL test suite seems the best choice
                                                      Dirk Jacob, Page 44
Design of the Test Suite

  many tests can be implemented using standard OSPF behavior

  some require specialized testers with non-standard behavior
      topology changes such as link failures
      simple errors like malformed packets
      complex tests which are aware of the state

  test behaviors are implemented in SSF.OS.OSPFv2.test

  five subsidiary test suites (see IOL test suite)

  additionally: tests for packet forwarding and equal-cost multipath

                                                        Dirk Jacob, Page 45
Design of the Test Suite (2)

  each test is performed in several steps
     simulation of the scenario
     evaluation of the logs
     assignment of a verdict

  evaluation is done automatically by PERL scripts


  test suite passes only when all tests pass

  common components for simulation are kept in a test suite
                                                      Dirk Jacob, Page 46

  contains classes to implement test behavior
     Configurator
     Reset
     UnreliableIP
     IPwithErrorInjection
     PacketGenerator
     OSPFMonitor, OSPFDumpPro

  main task: manipulate packets coming from or sent to OSPF

                                                    Dirk Jacob, Page 47
SSF.OS.OSPFv2.test (2)

 class hierarchy to be able to use different classes of testers

                                                          Dirk Jacob, Page 48
Implementation of Test
  model test setup in DML

  replace broadcast networks by a set of point-to-point links

  features specific to broadcast networks, virtual links, etc.
   not yet tested

  use tester classes to model tester behavior whenever necessary
                                                           Dirk Jacob, Page 49
Example: old_lsa_rcpt

 • a router must discard an LSA
   that is older than the copy in ist
   own database                                            Tester

 • routers synchronize their databases

 • then the tester sends a new LSA with sequence no. 0x70000001

 • after that, it sends another LSA with sequence no. 0x 8FFFFFFE

 • IUT must send back the newer instance and discard
   the second LSA
                                                       Dirk Jacob, Page 50
Additional Tests

  operation of the SPF algorithm not tested by IOL test suite

  packet forwarding and support of equal-cost multiplath not
   tested in IOL test suite

  provide additional test scenarios for these features

                                                          Dirk Jacob, Page 51
Test Results

  most tests PASSED

  some of the tests FAILED because of features missing from
   SSFnet (esp. multicast support)

  many tests INCONCLUSIVE, because features are not yet

  tests were very valuable during development
     helped identifying problems early
     helped make code more stable
     helped to find even „standard bugs“ like NullPointerExceptions

                                                              Dirk Jacob, Page 52

  Conformance Testing


  OSPF Test Suites

  Implementation for SSFNet


                              Dirk Jacob, Page 53

 goals:
    find out how to test protocols
    provide SSFNet with a test suite for OSPF

 formal methods seem not suitale for practical testing
    informal methods sufficient when used carefully
    formal methods could have grater impact if test suites were provided with
     the protocol standards

 testing is extremely important and useful
    to prove cnformance to the standard (at least to some degree)
    to find bugs in the implementation
    to understand, what a protocol really should do

                                                                      Dirk Jacob, Page 54

  OSPF validation in SSFNet:
     http://ssfnet.d-jacob.net

  SSFNet Website:
     http://www.ssfnet.org

  Test equipment and test suites:
      http://www.agilent.com
      http://www.ixia.com
      http://www.iol.unh.edu

  ... or send me a mail:
       dirk@d-jacob.net
                                     Dirk Jacob, Page 55

       Dirk Jacob, Page 56

Shared By: