Docstoc

Object Oriented Testing arif edu

Document Sample
Object Oriented Testing arif edu Powered By Docstoc
					Object Oriented Testing

             Ali Kamandi
       (kamandi@ce.sharif.edu)
    Sharif University of Technology
                            Object Oriented Testing
l   Find the greatest possible number of errors with a
    manageable amount of effort applied over a realistic time
    span.
l   The nature of OO programs changes both testing strategy
    and testing tactics.
l   Do we need less efforts for testing because of greater reuse of
    design patterns?
l   Answer is no! Binder argues that more testing is needed to
    obtain high reliability in OO systems, since each reuse is a
    new context of usage.




                                                                      2
     Testing Object-Oriented Applications:
                      Why is it Different?


l   No sequential procedural executions
l   No functional decomposition
l   No structure charts to design integration testing
l   Iterative O-O development and its impact on testing and
    integration strategies




                                                              3
                                         Objectives
l   To cover the strategies and tools associated with
    object oriented testing
     Analysis and Design Testing
     Unit/Class Tests
     Integration Tests
     Validation Tests
     System Tests
l   To discuss test plans and execution for projects


                                                        4
                              Object-Oriented Testing
l   When should testing begin?
l   Analysis and Design:
       Testing begins by evaluating the OOA and OOD models
       How do we test OOA models (requirements and use cases)?
       How do we test OOD models (class and sequence diagrams)?
       Structured walk-throughs, prototypes
       Formal reviews of correctness, completeness and consistency
l   Programming:
     How does OO make testing different from procedural programming?
     Concept of a ‘unit’ broadens due to class encapsulation
     Integration focuses on classes and their execution across a ‘thread’
      or in the context of a use case scenario
     Validation may still use conventional black box methods
                                                                             5
             Criteria for Completion of Testing
l When are we done testing? (Are we there yet?)
l How to answer this question is still a research question
1. One view: testing is never done… the burden simply shifts
   from the developer to the customer
2. Or: testing is done when you run out of time or money
3. Or use a statistical model:
       Assume that errors decay logarithmically with testing time
       Measure the number of errors in a unit period
       Fit these measurements to a logarithmic curve
       Can then say: “with our experimentally valid statistical model we
        have done sufficient testing to say that with 95% confidence the
        probability of 1000 CPU hours of failure free operation is at least
        0.995”
                                                                              6
                                                 Strategic Issues
l   Issues to address for a successful software testing strategy:
     Specify product requirements long before testing commences
      For example: portability, maintainability, usability
      Do so in a manner that is unambiguous and quantifiable
     Understand the users of the software, with use cases
     Develop a testing plan that emphasizes “rapid cycle testing”
      Get quick feedback from a series of small incremental tests
     Build robust software that is designed to test itself
      Use assertions, exception handling and automated testing tools (Junit).
     Conduct formal technical reviews to assess test strategy & test cases
      “Who watches the watchers?”




                                                                                7
                       Testing OOA and OOD Models (1)


l   The review of OO analysis and design models is especially useful because the
    same semantic constructs (e.g., classes, attributes, operations, messages) appear
    at the analysis, design, and code level.
l    Therefore, a problem in the definition of class attributes that is uncovered
    during analysis will circumvent side effects that might occur if the problem
    were not discovered until design or code (or even the next iteration of analysis).
l    By fixing the number of attributes of a class during the first iteration of OOA,
    the following problems may be avoided:
     Creation of unnecessary subclasses.
     Incorrect class relationships.
     Improper behavior of the system or its classes.
l    If the error is not uncovered during analysis and propagated further more
    efforts needed during design or coding stages.




                                                                                         8
               Testing OOA and OOD Models (2)
l   Analysis and design models cannot be tested in the conventional sense, because
    they cannot be executed.
l   Formal technical review can be used to examine the correctness and consistency
    of both analysis and design models.
l   Correctness:
     Syntax: Each model is reviewed to ensure that proper modeling conventions
       have been maintained.
     Semantic: Must be judged based on the model’s conformance to the real world
       problem domain by domain experts.
l   Consistency:
     May be judged by considering the relationship among entities in the model.
     Each class and its connections to other classes should be examined.
     The Class-responsibility-collaboration model and object-relationship diagram
       can be used.

                                                                                     9
                                    Testing Models
l   Criteria
    Correctness
    Completeness
    Consistency
l   Early informal models are tested informally
l   The criteria should be interpretive in the context of
    iterative incremental approach




                                                        10
                      Model Testing Approaches
l   Testing by comparison
    compares each model to its predecessor or to previous
      forms of the model
l   Testing by inspection
    uses checklists to make sure that the model meets certain
      criteria
l   Testing by verification
    follows certain steps to assure completeness and
      consistency of one part of the model with another


                                                                11
             Examples of Analysis and Design
                         Models to be Tested

l   CRC cards
l   Class specifications
l   Use cases
l   State-Transition Models
    State transition diagrams for classes, clusters, and
      subsystems
l   Object network
    Message sequence between methods in classes
l   Transaction-Flow Models
                                                           12
                                Testing the Class Model
1. Revisit the Use Cases, CRC cards and UML class model.
   Check that all collaborations are properly represented. Inspect
   the description of each CRC index card to determine if a
   delegated responsibility is part of the collaborator’s definition
     Example: in a point of sale system.
     A read credit card responsibility of a credit sale class is accomplished if
      satisfied by a credit card collaborator
2. Invert connections to ensure that each collaborator asked for a
   service is receiving requests from a reasonable source
     Example: a credit card being asked for a purchase amount
l   Have you tested your analysis and design?
     If not, who will do it?
3. These steps are applied iteratively to each class and through
   each evolution of the OOA model.                             13
                   Testing OO Code
Class/Unit tests        Integration
                           tests




System                     Validation
 tests                       tests

                                        14
             The Structured Testing Pyramid
Requirement Definition                           Systems Testing




      Preliminary Design                    Integration Testing




             Detailed Design            Unit Testing




                               Coding

                                                                   15
                              The OO Testing Pyramid

Use Case Analysis                                       Use Case Testing

      Test OOA

                                                  Cluster (Integration) Testing
     Class Hierarchy Design


              Test OOD                                Class Testing


              Method Design                     Method Testing


                    Test OOD


                                Method Coding
                                                                       16
                                                         Unit Test
l   What is a unit?
    A single, cohesive function?
    A function whose code fits on one page?
    The amount of code that can be written in 4 to 40 hours?
    Code that is assigned to one person?
l   We can no longer test a single operation in isolation but
    rather as part of a class.
l   In object-oriented programs, a unit is a method within a
    class.
l   Smallest testable unit is the encapsulated class


                                                                     17
             Class Testing Process

           How to test?
              class
              to be
             tested

                                 results
software
engineer            test cases   Why a loop?



                                           18
         Methods for Generating Test Cases
                          For Unit Testing
l   Statement coverage
l   Graph based
    Branch coverage
    Condition coverage
    Path coverage


l   All unit testing methods are also applicable to
    testing methods within a class.


                                                      19
                                        Class Test Case Design
   Berard [BER93,94] proposes the following approach:
1. Identify each test case uniquely
   - Associate test case explicitly with the class and/or method to be tested
2. State the purpose of the test
3. Each test case should contain:
      a. list of specified states for the object that is to be tested
      b. A list of messages and operations that will be exercised as a consequence of
           the test
      c. A list of exceptions that may occur as the object is tested
      d. A list of external conditions for setup (i.e., changes in the environment
           external to the software that must exist in order to properly conduct the
           test)
      e. Supplementary information that will aid in understanding or implementing
           the test
    Automated unit testing tools facilitate these requirements
                                                                                        20
                            Challenges of Class Testing
l   OO class is the target for test case design.
l   Encapsulation:
      Difficult to obtain a snapshot of a class without building extra methods
       which display the classes’ state
l   Inheritance and polymorphism:
      Each new context of use (subclass) requires re-testing because a method
       may be implemented differently (polymorphism).
      Other unaltered methods within the subclass may use the redefined method
       and need to be tested
l   How is class testing different from conventional testing?
     - Conventional testing focuses on input-process-output, whereas class testing
        focuses on each method, then designing sequences of methods to exercise
        states of a class
l   White box tests:
      Basis path, condition, data flow and loop tests can all apply to individual
       methods, but don’t test interactions between methods
                                                                                     21
                     Notes on OO Testing 1.
l   Applicability of conventional test case design:
     Although white-box testing is useful, the efforts might be redirected to
       tests at a class level.
     Use cases can provide useful input in the design of black-box and state-
       based tests.
l   Fault-based testing: (Tests have high likelihood of uncovering faults)
     Its planning begins with the analysis model.
     If real faults in an OO system are perceived to be implausible then this
        approach is really no better than any random testing technique.
     If the analysis and design models can provide insight into what is likely to
         go wrong, then fault-based testing can find significant numbers of
         errors with relatively low efforts.


                                                                                22
                     Notes on OO Testing 2.
l   Impact of OO programming on Testing:
     When an operation is invoked, it may be hard to tell exactly what code
      gets exercised.
     It can be hard to determine the exact type or class of a parameter.
     OO operations are smaller, more time needed for integration. So
      integration faults become more plausible.
      In 3 large scale studies: 50% of methods are less than 2 C++ statements or 4
      Smalltalk statements (Law of Demeter connection)
l   Scenario-based test design:
    Concentrates on what the user does, not what the product does.
    Capturing the tasks via use-cases that the user has to perform.
l   Testing surface structure and deep structure
    Surface structure: externally observable structure (end-user).
    Deep structure: internal technical details of an OO program.

                                                                                 23
                  OOT Methods: Class Level 1
l   Random testing
    identify operations applicable to a class
    define constraints on their use
    identify a minimum test sequence
         l   an operation sequence that defines the minimum life history of the class
             (object)
    generate a variety of random (but valid) test sequences
         l   exercise other (more complex) class instance life histories
l   Example:
    Class: Account
    Operations: open, setup, deposit, withdraw, balance, summarize, creditlimit, close.
    1.   Open – setup – deposit – withdraw – close
    2.   Open – setup – deposit –[deposit | withdraw | balance | summarize] * –
         withdraw – close. Generate random test sequences using this template
                                                                                   24
                 OOT Methods: Class Level 2
l   Partition Testing
     reduces the number of test cases required to test a class in much the same way as
        equivalence partitioning for conventional software
     state-based partitioning
         l categorize and test operations based on their ability to change the state of a
           class (e.g.: deposit, withdraw)
     attribute-based partitioning
         l categorize and test operations based on the attributes that they use ( e.g.:
           creditlimit attribute)
     category-based partitioning
         l categorize and test operations based on the generic function each performs.
           (e.g.: (Init OP: open , setup) (Comp. OP: deposit, withdraw) (queries: balance,
           summarize, creditlimit) (Termination OP: close))



                                                                                    25
     What Methods to Test Within Classes
l   New methods: defined in the class under test and
    not inherited or overloaded by methods in a
    superclass: Complete testing
l   Inherited methods: defined in a superclass of the
    class under test:    Retest only if the methods
    interacts with new or redefined method.
l   Redefined methods: defined in a superclass of but
    redefined in the class under test : complete Retest
    reusing tests from the superclass.
                                                          26
                    Class Testing Techniques
In addition to testing methods within a class
(either glass box or black box), the following three
techniques can be used to perform functional
testing for the class:
State-Transition testing
Transaction-Flow testing
Exception testing




                                                       27
                         State-Transition Testing


l   A state-transition model describes the different
    states and transitions of a class in the context of its
    position in the inheritance hierarchy.
                           (OMT dynamic model)
l   The state of an object is the combination of all the
    attribute values and objects that the object
    contains.
l   An object may transition from a state to state as a
    result of an event, which may yield an action.
                                                              28
                                                            Example



Prospect              Receive application              Member

                      Establish-membership


                                                                5-years anniversary
              Receive cancellation



           Retired-member                             Life-member
                               Receive cancellation




                                                                                29
                        State-Transition Testing
l   Create test cases corresponding to each transition
    path that represent a full object life cycle
l   Make sure each transition is exercised at least
    once.




                                                         30
                                     [2] Integration Testing
l     OO does not have a hierarchical control structure so
      conventional top-down and bottom-up integration tests
      have little meaning
l     Integrating operations one at a time into a class is often impossible
      because of the direct and indirect interactions of the components that
      make up the class.
l     Integration applied three different incremental strategies:
        Thread-based testing: integrates classes required to respond to one
         input or event
        Use-based testing: integrates classes required by one use case
        Cluster testing: integrates classes required to demonstrate one
         collaboration

    - What integration testing
         strategies will you use?                                              31
                Cluster (Integration) Testing:
                          Why is it Different?
l   Event-Driven execution
l   No functional decomposition
l   The sequence of execution is not known
l   Integration testing must be driven by how object
    will behave dynamically
l   Object composition introduces a new dimension of
    integration testing



                                                       32
                   Random Integration Testing
l   Multiple Class Random Testing
    1. For each client class, use the list of class methods to
       generate a series of random test sequences.
       Methods will send messages to other server classes.
    2. For each message that is generated, determine the
       collaborating class and the corresponding method in the
       server object.
    3. For each method in the server object (that has been
       invoked by messages sent from the client object),
       determine the messages that it transmits
    4. For each of the messages, determine the next level of
       methods that are invoked and incorporate these into the
       test sequence
                                                                 33
                 Cluster (Integration) Testing
l   A cluster is a collection of classes (possibly from
    different systems) cooperating with each other via
    messaging.
l   A cluster specification should include methods from
    each class that will be accessed
l   Cluster testing focuses on the interaction among the
    instances of the classes in the cluster
l   It assumes that each class has been tested individually
l   Cluster testing is considered a second level of
    integration testing
                                                          34
                 Methods for Forming Clusters
l   Function-based clustering
    Based on requirements and use cases
    Difficult to perform if requirements were not available
      during the design phase
l   Subject-based clustering
    Based on subject areas that need to test separately
l   Project Schedule-based clustering
l   Contract-based clustering


                                                              35
                Types of Errors Found During
                           Integration Testing
l   Messaging errors:
    Failure to meet a requirement, i.e., no method to send or
      receive a message
    Incompatible method and message in sender and receiver
    Incorrect event timing between object actions
    Incorrect instantiation or destruction of objects




                                                                36
                Types of Errors Found During
                     Integration Testing (cont.)
l   User interface errors:
    A given sequence of user actions does not have the
      expected effect on the component.
    The timing of events received from the user results in
     incorrect functioning of the component




                                                             37
             Techniques for Object-Oriented
                         Integration Testing



l   Message Quiescence
l   Event Quiescence




                                               38
                            Message Quiescence
l   A Method/Message path (MM-Path) is a sequence
    of method executions linked by messages.
l   An MM-Path starts with a method and ends when
    it reaches a method that does not issue a message
    of its own, i.e., reaches a message Quiescence.




                                                        39
                                Event Quiescence


l   An input port event followed by a set of MM-
    Paths, and terminated by an output event is called
    Atomic System Function (ASF).




                                                         40
INPUT PORT EVENT
    A                      OUTPUT PORT EVENT
             Class 1               A                   Class 3
        meth1                                               meth2
                       1
                                                  meth1
        meth2          meth3
 B
                                                                 B
ASF INPUT PORT                                              ASF OUTPUT PORT
EVENT                                                       EVENT
                   2                   Class 2
                                 meth1         meth3
                                          3
        MM-Path
        Message                      meth2


                                                                      41
                                       ATM PIN Entry
l   Customer enters card (event)
l   Screen requesting PIN entry is displayed
l   An interleaved sequence of digit key toughs with audible
    and visual feedback
l   Possibility of cancellation by customer
l   System disposition (valid PIN or card retained)




                                                               42
  CARDSLOT                          BANK




         SCREEN
                                  SECURITY


                  SpecialKeypad


Keypad                             NumKeypad

                                               43
                              ASF ends here                             ASF Starts here
                                                                       Customer inserts card
                               Message is displayed

              Screen                                             CardSlot
                                                          memberCard
         showMessage

                                                                 ValidateCard




Key pushers
                                                      Security
                            NumKeypad
                                                      checkPin
                       getKeyEvents

                                                                                    Bank
                        parseKeyEvent
                                                                                 pinForPan




                                                                                               44
                                    [3] Validation Testing
l   Are we building the right product?
l   Validation succeeds when software functions in a manner
    that can be reasonably expected by the customer.
l   Focus on user-visible actions and user-recognizable outputs
l   Details of class connections disappear at this level
l   Apply:
     Use-case scenarios from the software requirements spec
     Black-box testing to create a deficiency list
     Acceptance tests through alpha (at developer’s site) and beta (at
      customer’s site) testing with actual customers
l   How will you validate your software product?
                                                                          45
                                          [4] System Testing
l   Software may be part of a larger system. This often leads
    to “finger pointing” by other system dev teams
l   Finger pointing defence:
    1.   Design error-handling paths that test external information
    2.   Conduct a series of tests that simulate bad data
    3.   Record the results of tests to use as evidence
l   Types of System Testing:
        Recovery testing: how well and quickly does the system recover
         from faults
        Security testing: verify that protection mechanisms built into the
         system will protect from unauthorized access (hackers,
         disgruntled employees, fraudsters)
        Stress testing: place abnormal load on the system
        Performance testing: investigate the run-time performance within
         the context of an integrated system                                46
                                     System Testing
l   All rules and methods of traditional systems
    testing are also applicable applicable to object-
    oriented.
l   A thread can be “re-defined” as a sequence of
    method executions linked by messages in the object
    network.
l   Use cases provide a more precise mechanism for
    writing test cases.



                                                        47
                                                        Example
    Use Case # 1 of the Telephone Banking System:
        Task:             Making a balance transfer
        User Class:       Current customer
        Frequency:        one per week per customer


User Action                          System Response
User dials the number                System plays greeting and ask for
                                     account number
User enters invalid account number System informs user and ask for
                                   account number again
User enter a valid account number    System asks for PIN#


                                                                         48
                           Automated Testing
l   Junit at Junit.org
l   CPPUnit on SourceForge.net,
l   NUnit for .Net
l   Rational Robot




                                               49
                             OO Testing Methodology
l   Jorgensen and Erickson propose 5 levels:

        A Method                               Unit testing

        Message Quiescence                     Integration


        Event Quiescence                       Integration

        Thread testing                         System

        Thread interaction                     System




                                                              50
                        Stages of Test [RUP]
l   Developer Testing
l   Unit Test
l   Integration Test
l   System Test
l   Acceptance Test




                                               51
                              Test Planning [RUP]
l   What iteration you are you in?
l   What stage of test (unit test, integration test,
    system test) you are performing?
l   Types of test (functional, stress, volume,
    performance, usability, distribution, and so on).
l   Evaluation criteria used (code-based test coverage,
    requirements-based test coverage, number of
    defects, mean time between failure, and so on.)
l   Testing techniques used (manual and automated)

                                                          52
Completion Criteria [RUP]




                            53
Manual/Automated Testing [RUP]




                                 54
The END


          55

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:1
posted:10/1/2012
language:English
pages:55