VIEWS: 0 PAGES: 31 POSTED ON: 10/4/2012 Public Domain
Definitions for FSM based testing Chapter 8.1 What is Black-Box Testing? • When test inputs are generated using functional specifications, we say that we are doing black-box testing or functional testing. • Functional testing tests how well a program meets the functionality requirements. The Methodology • The derivation of test inputs is based on functional specifications. • Clues are obtained from the specifications. • Clues lead to test requirements. • Test requirements lead to test specifications. • Test specifications are then used to actually execute the program under test. Test Methodology Specifications Clues Expected behavior Program output Test requirements is correct Oracle or Test specifications Program has failed; make a Until specs. Test driver note and proceed Actual behavior Exhausted. with testing or Program get into the debug mode. Specification-Based Testing • We are given a specification, and coverage criterion – Test generation • Generate a test suite from the specification satisfying the criterion • A test suite is a set of pairs consisting of an input and its expected output – Test execution • Apply an input to the program and observe the output • Determine if the observed output is correct – Compare the observed output to the expected output Observability in Testing • Black-box testing – Only the inputs and outputs of the implementation are observable – We cannot access the internals of the implementation • White-box testing – The inputs, outputs, and internals are observable • Grey-box testing Black-Box Testing < specification > < implementation > a/0 s1 a b 1 b/1 b/0 s2 s3 b/1 a/1 a/0 We can infer that state before the test is is or thethe state before the test s1 s2 s2 state after the test is is or thethe state after the test s2 s2 s3 Finite State Machines • A FSM is a tuple (S, I, O, T) – S is a finite set of states a/0 – I is a finite set of input events – O is a finite set of output events s1 – T is a finite set of transitions. b / 1 b/0 • A transition is a tuple (s, i, o, s’) – s is the source state s2 s3 – i is the input event b/1 – o is the output event – s’ is the target state a/1 a/0 Finite State Machines • Two states are equivalent if – for every input sequence, the output sequences from the two states are the same a/1 s1 a/0 s1 and s2 are equivalent? s3 s2 a/0 Finite State Machines (3) • A FSM (S, I, O, T) is minimized (or reduced) if – no two states are equivalent • There are standard algorithms for minimizing FSMs in which equivalent states are collapsed into one state – I assume that every student in the class is familiar with the algorithms Finite State Machines (4) • A FSM (S, I, O, T) is completely-specified if – for every state s and input i, there is some output o and state s’ such that (s, i, o, s’) is a transition in T. What if a occurs at s1? s1 b/1 b/0 s2 s3 b/1 a/1 a/0 Finite State Machines (5) • Completing incompletely-specified FSMs – If there is no transition for a state s and input i, • “Implicitly defined” transitions – Add a loop transition with the null output (s, i, null, s) – Add a loop transition with an error output (s, i, error, s) • “Undefined by default” transitions – Add a set of transitions {(s, i, o, s’) | o O, s S} • “Forbidden” transitions – The input i is cannot be introduced at state s – Cannot be made completely-specified Finite State Machines (6) • Completing incompletely-specified FSMs Forbidden transition Undefined by default transitions Implicitly defined transitions: or 0 a / null a / a / error a/0 a/0 We cannot apply a at s1 a/1 a/1 s1 b/1 b/0 a/1 s2 s3 b/1 a/1 a/0 Finite State Machines (7) • A FSM (S, I, O, T) is deterministic if – for every state s and input i, there is at most one output o and one state s’ such that (s, i, o, a/0 s’) is a transition in T. What if a occurs at s1? s1 b/1 b/0 a/1 s2 s3 b/1 a/1 a/0 Finite State Machines (8) • A FSM (S, I, O, T) is initialized if – there is a state s_init in S designated as the initial state Finite State Machines • We will assume that FSMs are – Minimized – Completely-specified – Deterministic – Initialized (only when necessary) Finite State Machines • Homing sequences – We know everything about the implementation – But we do not know in which state the implementation is – A sequence of input events is a homing sequence if • After applying the sequence of input events, we can determine the final state by observing the sequence of output events Finite State Machines • Homing sequences < specification > < implementation > a/0 s1 ab b/1 b/0 s2 s3 b/1 a/1 a/0 Finite State Machines (12) • Synchronizing sequences – We know everything about the implementation – But we do not know in which state the implementation is – A sequence of input events is a synchronizing sequence if • The sequence of input events always takes the implementation to the same final state regardless of the state the implementation is in Finite State Machines • Synchronizing sequences < specification > < implementation > a/0 s1 aba b/1 b/0 a/0 s2 s3 b/1 a/1 Finite State Machines • State identification – We know everything about the implementation – But we do not know in which state the implementation is – A sequence of input events is a distinguishing sequence if • After applying the sequence of input events, we can determine the source state by observing the sequence of output events Finite State Machines • State identification: what is the current state? < specification > < implementation > a/0 s1 ab b/1 b/0 s2 s3 b/1 a/1 a/0 Finite State Machines • State verification – We know everything about the implementation – But we do not know in which state the implementation is – A sequence of input events is a UIO sequence if • After applying the sequence of input events, we can determine the source state is a specific state s by observing the sequence of output events Finite State Machines • State verification: the current state is s2? < specification > < implementation > a/0 s1 a b/1 b/0 s2 s3 b/1 a/1 a/0 Finite State Machines • Conformance testing – We are given the specification of an implementation – But we know nothing about the implementation – Determine whether the implementation is equivalent to the specification Testing Hypothesis/Assumptions Conformance relation An abstract model of An abstract model of a specification an implementation Hypothesis Hypothesis Assumptions Assumptions Conformance relation A specification An implementation Fault Models (2) • Output faults < specification > < implementation > a/0 a/0 s1 s1 b/1 b/0 b/1 b/0 s2 s3 s2 s3 b/1 b/1 a/1 a/0 a/0 a/0 Fault Models (3) • Transfer faults < specification > < implementation > a/0 s1 s1 a/0 b/1 b/0 b/1 b/0 s2 s3 s2 s3 b/1 b/1 a/1 a/0 a/1 a/0 Fault Model (4) • Transfer faults with additional states < specification > < implementation > a/0 a/0 s4 s1 s1 b/1 b/0 b/1 b/0 s2 s3 s2 s3 b/1 b/1 a/1 a/0 a/1 a/0 Fault Models (5) • Fault models for software – Sequencing faults – Arithmetic and manipulative faults – Calling wrong functions – Wrong specification of data type – Wrong values – Wrong number of variables – Wrong operators – …… • Fault models for EFSMs – Fault models for FSMs and software • Fault models of CEFSMs – Fault models for EFSMs and communications FSM-based Testing • Output faults – TT-method (transition tours) • Output faults + transfer faults – DS-method (distinguishing sequences) – W-method (characterizing sets) – UIO-method (UIO sequences) – UIOv-method – Wp-method – Harmonized Wp-method – Etc.