BB definitions by nw17D34

VIEWS: 0 PAGES: 31

									Definitions for FSM based
          testing
        Chapter 8.1
What is Black-Box Testing?
• When test inputs are generated using
  functional specifications, we say that we
  are doing black-box testing or functional
  testing.

• Functional testing tests how well a
  program meets the functionality
  requirements.
       The Methodology

• The derivation of test inputs is based on
  functional specifications.
• Clues are obtained from the
  specifications.
• Clues lead to test requirements.
• Test requirements lead to test
  specifications.
• Test specifications are then used to
  actually execute the program under test.
                  Test Methodology
               Specifications
                   Clues        Expected behavior

                                                    Program output
          Test requirements                         is correct
                                      Oracle          or
         Test specifications
                                                    Program has
                                                    failed; make a
Until specs.
                Test driver                         note and proceed
                                Actual behavior
Exhausted.                                          with testing or
                 Program                            get into the debug
                                                    mode.
         Specification-Based Testing

• We are given a specification, and
  coverage criterion
  – Test generation
     • Generate a test suite from the specification
       satisfying the criterion
     • A test suite is a set of pairs consisting of an input
       and its expected output
  – Test execution
     • Apply an input to the program and observe the
       output
     • Determine if the observed output is correct
        – Compare the observed output to the expected output
      Observability in Testing
• Black-box testing
  – Only the inputs and outputs of the
    implementation are observable
  – We cannot access the internals of the
    implementation
• White-box testing
  – The inputs, outputs, and internals are
    observable
• Grey-box testing
         Black-Box Testing
       < specification >                 < implementation >
            a/0


             s1
                                     a
                                     b                         1
 b/1                   b/0



s2                         s3
            b/1

a/1                        a/0            We can infer that
                                     state before the test is is or
                                 thethe state before the test s1 s2 s2
                                      state after the test is is or
                                  thethe state after the test s2 s2 s3
        Finite State Machines
• A FSM is a tuple (S, I, O, T)
  – S is a finite set of states
                                                    a/0
  – I is a finite set of input events
  – O is a finite set of output events              s1

  – T is a finite set of transitions. b / 1               b/0
     • A transition is a tuple (s, i, o, s’)
        –   s is the source state
                                               s2          s3
        –   i is the input event                    b/1
        –   o is the output event
        –   s’ is the target state         a/1            a/0
       Finite State Machines
• Two states are equivalent if
  – for every input sequence, the output
    sequences from the two states are the same
                                         a/1
                              s1   a/0

  s1 and s2 are equivalent?               s3

                              s2   a/0
     Finite State Machines (3)
• A FSM (S, I, O, T) is minimized (or
  reduced) if
  – no two states are equivalent
• There are standard algorithms for
  minimizing FSMs in which equivalent
  states are collapsed into one state
  – I assume that every student in the class is
    familiar with the algorithms
        Finite State Machines (4)
• A FSM (S, I, O, T) is completely-specified if
   – for every state s and input i, there is some output o and state s’
     such that (s, i, o, s’) is a transition in T.



                      What if a occurs at s1?
                                   s1

                        b/1                b/0



                        s2                   s3
                                  b/1

                      a/1                   a/0
     Finite State Machines (5)
• Completing incompletely-specified FSMs
  – If there is no transition for a state s and input i,
     • “Implicitly defined” transitions
        – Add a loop transition with the null output (s, i, null, s)
        – Add a loop transition with an error output (s, i, error, s)
     • “Undefined by default” transitions
        – Add a set of transitions {(s, i, o, s’) | o  O, s  S}
     • “Forbidden” transitions
        – The input i is cannot be introduced at state s
        – Cannot be made completely-specified
    Finite State Machines (6)
• Completing incompletely-specified FSMs

               Forbidden transition
          Undefined by default transitions
           Implicitly defined transitions:
                              or 0
                     a / null a / a / error    a/0
              a/0
             We cannot apply a at s1
               a/1                            a/1
                              s1

                b/1                       b/0
                             a/1


                s2                            s3
                             b/1

               a/1                            a/0
     Finite State Machines (7)
• A FSM (S, I, O, T) is deterministic if
  – for every state s and input i, there is at most
    one output o and one state s’ such that (s, i, o,
                           a/0
    s’) is a transition in T.
  What if a occurs at s1?
                            s1

                   b/1            b/0
                            a/1


                   s2              s3
                            b/1

                  a/1             a/0
     Finite State Machines (8)
• A FSM (S, I, O, T) is initialized if
  – there is a state s_init in S designated as the
    initial state
       Finite State Machines
• We will assume that FSMs are
  – Minimized
  – Completely-specified
  – Deterministic
  – Initialized (only when necessary)
       Finite State Machines
• Homing sequences
  – We know everything about the
    implementation
  – But we do not know in which state the
    implementation is
  – A sequence of input events is a homing
    sequence if
    • After applying the sequence of input events, we
      can determine the final state by observing the
      sequence of output events
          Finite State Machines
• Homing sequences

          < specification >              < implementation >
               a/0


                s1
                                    ab
    b/1                   b/0



   s2                         s3
               b/1

   a/1                        a/0
   Finite State Machines (12)
• Synchronizing sequences
  – We know everything about the
    implementation
  – But we do not know in which state the
    implementation is
  – A sequence of input events is a
    synchronizing sequence if
    • The sequence of input events always takes the
      implementation to the same final state regardless
      of the state the implementation is in
          Finite State Machines
• Synchronizing sequences

           < specification >              < implementation >
                a/0


                 s1
                                    aba
     b/1                   b/0
               a/0


    s2                         s3
                b/1

    a/1
        Finite State Machines
• State identification
  – We know everything about the
    implementation
  – But we do not know in which state the
    implementation is
  – A sequence of input events is a
    distinguishing sequence if
     • After applying the sequence of input events, we
       can determine the source state by observing the
       sequence of output events
           Finite State Machines
• State identification: what is the current
  state?
            < specification >              < implementation >
                 a/0


                  s1
                                      ab
      b/1                   b/0



     s2                         s3
                 b/1

     a/1                        a/0
        Finite State Machines
• State verification
  – We know everything about the
    implementation
  – But we do not know in which state the
    implementation is
  – A sequence of input events is a UIO
    sequence if
     • After applying the sequence of input events, we
       can determine the source state is a specific state s
       by observing the sequence of output events
        Finite State Machines
• State verification: the current state is s2?

             < specification >             < implementation >
                  a/0


                   s1
                                       a
       b/1                   b/0



      s2                         s3
                  b/1

     a/1                         a/0
       Finite State Machines
• Conformance testing
  – We are given the specification of an
    implementation
  – But we know nothing about the
    implementation
  – Determine whether the implementation is
    equivalent to the specification
          Testing
   Hypothesis/Assumptions
                          Conformance
                            relation
An abstract model of                    An abstract model of
   a specification                       an implementation




             Hypothesis                              Hypothesis
            Assumptions                             Assumptions


                          Conformance
                            relation
    A specification                      An implementation
                  Fault Models (2)
• Output faults
          < specification >         < implementation >

                a/0                        a/0


                 s1                         s1

     b/1                 b/0        b/1            b/0



     s2                       s3   s2                s3
                b/1                        b/1

    a/1                   a/0      a/0              a/0
                  Fault Models (3)
• Transfer faults
          < specification >               < implementation >

                a/0


                 s1                               s1
                                   a/0
     b/1                 b/0              b/1            b/0



     s2                       s3         s2                s3
                b/1                              b/1

    a/1                   a/0            a/1              a/0
                  Fault Model (4)
• Transfer faults with additional states
          < specification >         < implementation >

                a/0                         a/0
                                                          s4


                 s1                         s1

      b/1                b/0        b/1            b/0



     s2                       s3   s2                s3
                b/1                        b/1

     a/1                  a/0      a/1              a/0
               Fault Models (5)
• Fault models for software
  –   Sequencing faults
  –   Arithmetic and manipulative faults
  –   Calling wrong functions
  –   Wrong specification of data type
  –   Wrong values
  –   Wrong number of variables
  –   Wrong operators
  –   ……
• Fault models for EFSMs
  – Fault models for FSMs and software
• Fault models of CEFSMs
  – Fault models for EFSMs and communications
         FSM-based Testing
• Output faults
  – TT-method (transition tours)
• Output faults + transfer faults
  – DS-method (distinguishing sequences)
  – W-method (characterizing sets)
  – UIO-method (UIO sequences)
  – UIOv-method
  – Wp-method
  – Harmonized Wp-method
  – Etc.

								
To top