Docstoc

Object-Oriented Software Testing

Document Sample
Object-Oriented Software Testing Powered By Docstoc
					Object-Oriented Software
        Testing

           ICS 224A
             Joachim Feise
          jfeise@ics.uci.edu
  A Brief Meditation on the Essential
     Nature of Software Testing
Thus spoke the master: “Any program, no matter how small, contains bugs.”
The novice did not believe the master’s words. “What if the program were so small that it
performed a single function?” he asked.

“Such a program would have no meaning,” said the master, “but if such a one existed, the
operating system would fail eventually, producing a bug.”
But the novice was not satisfied. “What if the operating system did not fail?” he asked.

“There is no operating system that does not fail,” said the master, “but if such a one existed,
the hardware would fail eventually, producing a bug.”
The novice still was not satisfied. “What if the hardware did not fail?” he asked.

The master gave a great sigh. “There is no hardware that does not fail,” he said, “but if such
a one existed, the user would want the program to do something different, and this too is a
bug.
A program without bugs would be an absurdity, a nonesuch. If there were a program without
any bugs then the world would cease to exist.”
                                                   Geoffrey James, The Zen of Programming
    June 4, 1997                       OO Software Testing                                 2
   Object-Oriented Testing Papers
• Object-Oriented Integration Testing
      – Jorgensen and Erickson
      – CACM Sept. 1994
• Object-Oriented Software Testing - A
  Hierarchical Approach
      – Shel Siegel
      – J. Wiley and Sons, 1996

June 4, 1997           OO Software Testing   3
                 Structure vs. Behavior
• most popular notations in software
  development portray software structure
               • examples: E/R models, data flow diagrams, structure
                 charts, etc.
      – used to show relationships between components,
        interfaces, control and dataflow, etc.
      – not very useful for testing
• SW testing is fundamentally concerned with
  behavior, i.e. what it does
June 4, 1997                   OO Software Testing                 4
          Traditional Testing Notions
• Traditional software is
      –   written in an imperative language
      –   described by functional decomposition
      –   developed in a waterfall life cycle
      –   separated into three levels of testing
               • unit level, integration level, system level
• these points do often not apply directly to
  object-oriented software
June 4, 1997                     OO Software Testing           5
           Imperative vs. Declarative
• Imperative languages lend themselves to a
  description as a directed graph
• declarative languages suppress sequentiality
• the event-driven nature of OO systems
  forces a “declarative spirit” on testing
      – not evident at unit level since most object-
        oriented languages are imperative
      – but pronounced at integration and system levels
June 4, 1997           OO Software Testing            6
                   Unit Testing Overview
• Units can be defined in several ways
               •   a single, cohesive function
               •   a function which, when coded fits on one page
               •   the smallest separately compilable segment of code
               •   the amount of code that can be written in 4 to 40 hours
               •   a task in a work breakdown structure
               •   code that is assigned to one person
               •   code that one person designs, codes and test in a three-
                   month period
June 4, 1997                      OO Software Testing                   7
     Unit Testing Overview (cont.)
• Many organizations have not chosen their
  definition of a software unit
• Units are always tested “by itself”
      – stubs and drivers are used to emulate inputs
        and outputs
      – goal: verification that, taken by itself, the unit
        functions correctly


June 4, 1997             OO Software Testing                 8
      Integration Testing Overview
• Once the units are separately tested, they
  are integrated together
               • gradual replacement of stubs and drivers by
                 separately tested units
               • pairwise integration, i.e. each unit is integrated with
                 its adjacent units
               • bottom-up integration guided by functional
                 decomposition tree
               • “big bang” integration where all units are thrown
                 together at once
June 4, 1997                     OO Software Testing                       9
           System Testing Overview
• Conducted exclusively in terms of inputs
  and outputs that are visible at the port
  boundary of a system
• Tester only has access to those port events
  that are visible to the customer/user
• Can occur on the target platform only


June 4, 1997       OO Software Testing          10
 System Testing Overview (cont.)
• Examples of system testing activities
               • sequences of machine instructions
               • sequences of source instructions
               • scenarios of normal usage
               • system-level test cases
               • stimulus/response pairs
               • behavior that results from sequences of system-level
                 inputs
               • sequences of transitions in a state machine
                 description of the system
June 4, 1997                    OO Software Testing                11
           Object-Oriented Software
    – within an object, individual methods are imperative
    – declarative aspect because of event-driven nature
    – dynamic binding creates indefiniteness that
      resembles declarative programs
    – it is impossible to ever know the full set of
      “adjacent” objects with which a given object may
      be composed
          • two object may be correct, but errors might result when
            they are composed
June 4, 1997                OO Software Testing                 12
       Composing Errors Illustrated




June 4, 1997     OO Software Testing   13
               Levels of OO Testing
• Unit testing
      – method testing
• Integration testing
      – Message quiescence
      – Event quiescence
• System testing
      – Thread testing
      – Thread interaction testing
June 4, 1997             OO Software Testing   14
                    Integration Testing
• Method executions are linked by messages
• Quiescence provides natural endpoints
• Message quiescence
      – Method/Message path (MM-Path)
               • sequence of message executions linked by messages
               • starts with a method
               • ends when method is reached that does not issue any
                 messages of its own
                  – endpoint is called message quiescence
June 4, 1997                     OO Software Testing              15
           Integration Testing (cont.)
• Atomic System Function (ASF)
               • input port event,
               • followed by a set of MM-Paths,
               • terminated by output port event
      – ASFs constitute the point at which integration
        and system testing meet
      – elemental function visible at the system level


June 4, 1997                    OO Software Testing      16
   MM-Paths and ASFs Illustrated




June 4, 1997   OO Software Testing   17
     ATM System Class Hierarchy




June 4, 1997   OO Software Testing   18
                Example: ATM System
• Examples of
      – MM-Path: Digit entry
      – ASF: PIN Entry
      – Thread (sequence of ASFs):
               • simple transaction: PIN entry, transaction type
                 selection, account detail presentation, operation
                 execution, result reporting
      – Thread sequence:
               • a session consisting of multiple simple transactions
June 4, 1997                    OO Software Testing                     19
  Digit Entry MM-Path Illustrated




June 4, 1997   OO Software Testing   20
           PIN Entry ASF Illustrated




June 4, 1997        OO Software Testing   21
                  Observations
• Many-to-many relationships
  dominate (see E/R diagram)
      – an object may be involved in
        many threads
      – threads entail many objects
      – an object may be involved
        with many ASFs
      – ASFs may entail many
        objects
June 4, 1997           OO Software Testing   22
               Observations (cont.)
• The five levels of OO testing result in
  distinct, useful testing goals
• Bottom-up testing order
• Seamless juncture between OO integration
  testing and system testing
• Constructs are directly usable as basis for
  test coverage metrics
• Works well with composition
June 4, 1997         OO Software Testing        23
          The Hierarchical Approach
• Everything is a system
• Defines and applies testing standards for
  several levels of software components
      –   objects
      –   classes
      –   foundation components
      –   systems

June 4, 1997            OO Software Testing   24
Hierarchical Approach Illustrated




June 4, 1997   OO Software Testing   25
               Safe Components
• Components that meet the testing standards
  for that kind of components
• only safe components can be integrated with
  other safe components to produce the next-
  level component
• Safe is a relative state
      – depends on risk management practices, attitude
        toward risk, what is enforced, the application, etc.
June 4, 1997            OO Software Testing             26
               Foundation Components
• Can be one complete class hierarchy
• or some cluster of classes
      – that performs a core function,
      – that represents a logical or physical
        architectural component, etc.
• After testing to a safe level, can be
  integrated with other foundation
  components
June 4, 1997            OO Software Testing     27
Integration Testing of Foundation
           Components
• Needs to address the interconnections
  between the foundation components and
  any new composite functionality ONLY
• Eliminates the need to test all of the
  combinations of states during integration
  testing
• Improves productivity

June 4, 1997      OO Software Testing         28
        Integration Testing Problem
                 Illustrated




June 4, 1997      OO Software Testing   29
                 Test Suites
•   Conditional Test Suite
•   Hierarchical Incremental Test Suite
•   Integration Test Suite
•   System Test Suite
•   Regression Test Suite



June 4, 1997        OO Software Testing   30
               Conditional Test Suite
• Four types of test scripts
      –   Assertions: abort if condition fails
      –   Exceptions: raise exception if condition succeeds
      –   Concurrent test operations: monitor condition
      –   Message polling: evaluate condition on demand
• Conditions
      – class invariant, precondition, postcondition, loop
        invariant
June 4, 1997              OO Software Testing           31
             Pre/Postconditions Illustrated
class STACK2 [T] export                                                  do
      push, pop, top, empty, nb_elements, full                                  nb_elements := nb_elements – 1
feature                                                                  ensure
      implementation: ARRAY [T]                                                 not full;
      max_size: INTEGER;                                                        nb_elements = old nb_elements – 1
      nb_elements: INTEGER;                                              end; -- pop

Create (n: INTEGER) is                                            top: T is
             -- Allocate stack for a maximum of n elements                      -- Top element
             -- (or for no elements if n < 0)                            require
      do                                                                        not empty – i.e. nb_elements > 0
             if n > 0 then max_size := n end;                            do
             implementation.Create (1, max_size)                                Result := implementation.entry (nb_elements)
      end; -- Create                                                     end; -- top

empty: BOOLEAN is                                                 push (x: T) is
             -- is stack empty?                                                -- Add x on top
      do                                                                require
             Result := (nb_elements = 0)                                       not full
      end; -- empty                                                                   -- i.e. nb_elements < array_size in this
                                                                                      -- representation
full: BOOLEAN is                                                        do
             -- Is stack full?                                                 nb_elements := nb_elements + 1;
        do                                                                     implementation.enter (nb_elements, x)
             Result := (nb_elements = max_size)                         ensure
        end; -- empty                                                          not empty;
                                                                               top = x;
pop is                                                                         nb_elements = old nb_elements + 1
               -- Remove top element                                    end; -- push
         require                                                  end – class STACK2
               not empty – i.e. nb_elements > 0

  June 4, 1997                                       OO Software Testing                                                    32
      Hierarchical Incremental Test
                  Suite
• Structural models
      – control-flow model
               • models class as series of control-flow graphs
                 deriving from the method implementations
      – data-flow model
               • models class as series of control-flow graphs with
                 data annotations showing the flow of data values
                 through the method implementations


June 4, 1997                    OO Software Testing                   33
    Hierarchical Incremental Test
            Suite (cont.)
• Functional models
   – state-transition model, aka OMT dynamic model
       • models states and transitions for a class in the context
         of its position in the inheritance hierarchy
   – transaction-flow model
       • models object life cycle as transaction from
         construction to destruction
   – exception model
            • specifies what methods raise what exceptions under
June 4, 1997 what conditions Software Testing
                             OO                                 34
               Integration Test Suite
• Functional Test Suite
      – tests vertical integration of a cluster of
        components, i.e. the way the cluster implements
        specific functions from the requirements
• Abstraction Test Suite
      – tests horizontal integration of a cluster of
        components, i.e. the way the cluster abstracts a
        service contract into an API

June 4, 1997            OO Software Testing            35
    Integration Test Suite (cont.)
• Possible test scripts
    – class-to-class test script
        • interaction between classes based on message
          relationship
        • protocol-interface, synchronization, exception
          handling tests
    – UI test script
             • tests UI components of cluster and the components
               on which they depend
June 4, 1997
             • true-time, character-recognition, widget-playback 36
                               OO Software Testing
               tests
               System Test Suite
• System Acceptance Test Suite
      – derived from the set of use cases covering the
        system requirements
• User Options Test Suite
      – based on probable usage patterns of the features
        and functions delivered with the system
      – designed to simulate the way the end users will
        combine the features and functions
June 4, 1997            OO Software Testing              37
               Regression Test Suite
• Series of tests run against a previous version
  of the CUT
• Almost always automated
      – performing manual tests is usually not cost
        effective




June 4, 1997            OO Software Testing           38
                             Epilog

    “Testing proves the presence, not the absence, of
    bugs.”
          E.W. Dijkstra

    “Absence of evidence is not evidence of absence.”
          Source unknown
                  From: E.V. Berard, Issues in the Testing of OO Software


    Slides available at http://www.ics.uci.edu/~jfeise/ICS224A
June 4, 1997                 OO Software Testing                       39