Basics Of Software Testing

Document Sample
Basics Of Software Testing Powered By Docstoc
					SOFTWARE TESTING AND
 QUALITY MANAGEMENT
              Software Testing

 What       is it ?
 –   Source code MUST be tested to find (and correct )
     as many errors as possible before it has been
     delivered finally.
 –   Design the TESTING TECHNIQUES which gives
     high probability of finding errors.
 –   Software Testing Techniques - gives the
     guidance for TEST to
       a) Exercise the Internal Logic
       b) Exercise the input-output parameters
                 Software Testing

   Who Does it?
    –   Normally a software engineer or testing specialist



Why       it is important ?
    –High amount of “COSTS” are involved.. Upon failure
    –Quality
                   Software testing

   OBJECTIVES :
    –   The process of executing the software to find out the
        ERRORS.
    –   A good Test CASE … gives high probability finding an error ,
        which is yet to be known.
    –   A successful TEST … finds the error which is yet to be known.
   Objectives SUMMARY --- To design tests that
    systematically discover the DIFFERENT CLASSES of
    Errors with minimum amount of TIME and EFFORT.
   Testing CAN NOT show the ABSENCE of Errors …
    shows only the errors / defects PRESENT.
Software Testing

   Steps ?
    –   Two Different steps –
          a) White Box … Test Case design to test internal
           program logic AND
          b) Black Box … design to test the software
           requirements.
    –   Our aim is to find the maximum number of
        errors with minimum MAN HOURS
        EFFORTS and TIME.
    Basic Principles of Software Testing
   All tests should be traceable to customer requirement.
   Test should be planned long before testing begins.
   Apply Pareto Principle ( 80% of all uncovered errors
    during testing is likely due to 20% of all program
    components … Isolate such components .
   Testing should begin “in small” and progress toward
    testing “in the large” – focus on components first ..while
    testing in progress… focus on clusters of such
    components.
   Exhaustive testing is not possible – It is impossible to
    execute every combinations of path.
   Testing should be conducted by the THIRD party.
Attributes of GOOD Test

   High probability of finding an Error
          Tester should DIGEST the software … develop the picture
           of HOW IT BEHAVES ?
   Should not be redundant
          Every test should be different (purpose) from others
   BEST from others
          Group of tests , it must be able to find highest number of
           errors
   Should be neither not too simple nor too
    complex
Testing METHODS

   Our Objective ;
    –   to find the maximum number of errors with minimum MAN
        HOURS EFFORTS and TIME.
   Methods provide a mechanism for ensuring the
    completeness and provide the highest
    probability of errors.
   Provides the systematic approach to testing
   Two Methods ;
    –   a) Black Box & b) While Box Method
Testing Methods
   BLACK BOX :
    –   Used to test software functions …
    –   I/p is properly accepted …?
        O/p is produced ?
    –   Checking of FUNDAMENTAL PARTS OF
        THE SOFTWARE … not focuses on
        internal logic structure of the software.
    –   Tests conducted at the software interface
Methods

   White Box :
    –   Focuses on PROCEDURAL logic of the software.
    –   Logical Paths are tested ( by specific set of
        conditions & Loops )
    –   The status of the program is checked at various
        points.
   BUT………..!!!!
            Even for small programs the number of logical paths ARE
             VERY LARGE … Exhaustive TESTING
   So,important logical paths are selected & Exercised
White Box Testing (Structural Testing)

   Sometimes called GLASS-BOX testing
   Uses the control structure of the procedural
    design (Test the software‟s structural logic)
   Butt….not ensuring that user requirement has
    satisfied or not ( disadvantage)
   Why to do this test ?
          Logical Errors
          Logical paths
White Box Testing (Structural Testing)

   We can derive with this method
          Guarantee that all independent paths within a module have
           been exercised AT LEAST ONCE
          Exercise all logical decisions on their TWO values
          Execute ALL LOOPS
          Exercise the internal data structures to ensure their validity.
   Here.. Logical paths are tested using
    conditions / loops … The status of the program
    is checked at various points with the actual
    status.
Basis Path Testing

   White box technique.
   Proposed by Tom McCabe
   It derives the logical complexity measure and
    using this gives the set of execution paths
    –   Guarantees to execute every statement in the
        program AT LEAST ONCE , during the testing .
   Types ;
            Flow Graph Notation
            Cyclomatic complexity
            Graph Matrices
Let’s understand FLOW CHART

   Pictorial shapes representing a procedural
    detail
          A Box … indicate a processing step
          A diamond … logical condition
          An Arrow … flow


                                                T



                                            Do - While
Sequence          If - Else
Flow graph notation


                     1


                     2

             6       3    4

     7           8
                          5
11       9           10
Flow Graph notation

Flow Graph
                             1              Nodes

                 Edges   2,3

                    6    R2           4,5   R1
                                                 R4
             7     R3    8

                    9                        Regions (R1..R4)
                                 10
    11
Flow Graph notation

   Region may contain the area outside the graph
   If compound conditions ( more Boolean
    operators) are present … this method becomes
    COMPLICATED … Why ? Becoz each
    conditions results separate node
Cyclomatic Complexity

   Measures the logical complexity
   Concept - defines the number of INDEPENDENT
    PATHS in a program which gives us the upper limit of
    the number of tests that MUST be conducted …
    (Independent Path – any path giving at least one NEW
    set of statements … at least move along one edge not
    covered so far)
   assures that … EXECUTION OF ALL STATEMENTS
    AT LEST ONCE
Cyclomatic Complexity

   Independent Paths

Path ONE :: 1-2-3-4-5-10-1-11
                     Path TWO :: 1-2-3-6-8-9-10-1-11
Path THREE: 1-2-3-6-7-9-10-1-11
                     Path FOUR : 1-11

    Each Path shows NEW EDGE … should not include
    the combination of already specified paths
Cyclomatic Complexity

   How many Independent Paths ?
    –   V(G) = E – N + 2
            E – The number of flow graph EDGE
            N – flow graph nodes
            V(G) = 11 – 9 + 2 = 4
   V(G) provides the UPPER LIMIT of number of
    independent paths tests … upper bound of
    number of tests
Graph Matrix

   A square matrix whose size (number of rows
    and columns) = number of nodes
   Each row & column corresponds to a NODE
    and matrix entries are based on EDGES
   Each node is identified by number … edge by
    letters
   A letter entry is made in to matrix
Graph Matrix
                                  Connected to node
                                  1 2 3          5
             1
                 a       Node 1          a
             3
     e
              b      d               d       b
 5           4
         f                   4       c           f
               c
     g       2                       g e
                             5
Graph Matrix

   Link weight
    –   1 when connection exists
    –   0 connection doesn‟t exist
                        1 2 3        5 Connections
                Node 1         1       1-1 =0


                           1    1      2-1 = 1   3+1 = 4
                     4      1        1 2-1 = 1
                            1 1        2-1 = 1
                     5
Control Structure testing

   To expand the test area and improve quality
   Condition testing
    –   Test case design method that exercises the logical
        conditions contained in a program
    –   Expression 1 < relational operator > Expression 2
        (Relational expressions)
    –   <,>=,<,<=, non equality
    –   Allows compound expressions … Boolean
        expressions
Condition testing

   Focuses on testing each condition of a
    program
   Purpose is not only to detect errors in the
    conditions but also the errors in the program
   Types of errors
          Boolean operator error
          Boolean variable error
          Boolean parenthesis error
          Relational operator error
          Arithmetic expression error
Condition Testing

   Branch testing
    –   T & F branches of every condition … requires to be
        tested at least once
   Domain Testing
    –   Three or four tests to be derived for a relational
        expression
    –   Exp1<rel-op>Exp2 … Exp1 is >,<,= Exp2
Loop Testing

   Technique use to test the validity of loops
   Simple loops
                                 Suggested Approaches
                                 •Only One pass thru loop
                                 •Two passes
                                 •‘m’ passes … m<n (n is
                                 the max. allowable pass)
                                 •‘n-1,n,n+1 times
Loop testing

   Nested loops


                   Suggested Approaches
                   •Start with the innermost ..
                   Set all outer to minimum
Black Box testing

   Called Behavioral testing
   Focuses on the FUNCTIONAL requirements
   Concept – derive all the sets of input conditions that
    will exercise all functional requirements
   Checking of FUNDAMENTAL PARTS OF
    THE SOFTWARE … not focuses on
    internal logic structure of the software.
Black Box testing

   Helps to find ;
    –   Incorrect / Missing functions
    –   Interface errors
    –   Improper data structure
   Example – A certain function key produce a
    specific result when pressed … functional test
    says validate this by pressing the key and
    observe the result.
Functional – Black Box testing

   Advantages
    –   Simulates actual system usage
    –   Makes no structure assumptions
   Disadvantages
    –   Will not test logical error
    –   Possibility of redundant test
Functional – Black Box testing

   These tests are all VALIDATION tests
    inspecting system performance like;
    –   Unit testing – verifies the system functions
    –   Integrated testing – Execute TASKS design for
        integrated applications / database
    –   System testing – simulates the entire system
        software
    –   User acceptance – May be your staff or customers
        verify these functions once they start interacting with
        the system
Equivalence partitioning

   Divides the Input domain of a program into
    “classes of data” for which test cases can be
    derived.
   Ideal test – uncovers the class of errors
   Equivalence partitioning defines the test case
    that uncovers the class of errors … reduce the
    number of tests
Equivalence partitioning

   Test is designed on an evaluation of equivalence class
    for an input condition
   An equivalence class represents a set of VALID or
    INVALID states for input conditions . These classes are
    defined as per the guidelines as;
    –   I/p condition specifies
           range – one valid two invalid equivalence classes
           Value – „as above‟
           A member of set - one valid one invalid
           Boolean – „as above‟
Equivalence partitioning

   As per the guidelines , test for each input
    domain data item can be developed and
    executed
   Tests are conducted such that largest number
    of classes exercised at least once
Boundary Value analysis

   Errors are becoz at the boundaries of the input
    domain ( rather in the center)
   This test case – rather than selecting any
    element of an equivalence class, concentrate
    on EDGES of the class.
   Test case is just not for input domain
    conditions , derives the output domain as well
BVA

   Guidelines are for an input condition ;
    –   Range between a & b
            Test case be designed with values a & b and just below a
             & b.
    –   Number of values
            Values just above & below maximum & minimum values
    –   Data structure has prescribed boundary ( an array)
            Exercise at its boundary
   Above guidelines are also for output conditions
Comparison testing

   Crucial applications requires reliability of the
    software ( for example , avionics )
   Separate independent version of the software
    is developed with the same specifications (
    example - flight simulator)
   Each test is carried out at independent version
   They are executed in parallel with real time to
    ensure consistency
Comparison testing

   These independent versions form the
    comparison testing technique , also called the
    back-to-back testing

   Not fool proof – depend on the independent
    version
Orthogonal Array testing

   If 3 I/p have 3 values … 27 possible tests … 3
    with 4 values … 81 tests
   This test enables us good test with fewer tests
    than the exhaustive tests
   Applied where input domain is relatively small
    but creating exhaustive testing cases
   Useful for finding error because of the faulty
    logic
Orthogonal Array testing

   Conventional approach says … “one input item
    at a time “ … useful only when only single
    parameter value is faulty ( Called Single mode
    faults) … it can not detect logic faults that
    causes becoz of two or more parameters
    simultaneously … Means ability to detect
    errors are LIMITED.
   For example … 1111, 2111,
    3111,1211,1311,1121,1131,1112,1113
Orthogonal Array testing

   When this test occurs … an L9orthogonal array
    of test cases is created
   It has balancing property … uniformly
    distribution of test coverage over the entire
    input domain
   Single mode faults
          Detect and isolate … first three tests
   Double mode faults
          Detects …. Problem due to two parameters together…
Orthogonal Array testing

      1   1    1   1
      1   2    2   2
      1   3    3   3
      2   1    2   3
      2   2    3   1
      2   3    1   2
      3   1    3   2
      3   2    1   3
      3   3    2   1
Verification & Validation
   Verification
    –   provides objective evidence that the design outputs of a
        particular phase of the software development life cycle meet all
        of the specified requirements for that phase.
    –   looks for consistency, completeness, and correctness of the
        software and its supporting documentation, as it is being
        developed
    –   Software testing is one of many verification activities intended
        to confirm that software development output meets its input
        requirements. Other verification activities include various static
        and dynamic analyses, code and document inspections,
        walkthroughs, and other techniques.
Verification & Validation

   Validation
    –   confirmation by examination and provision of
        objective evidence that software specifications
        conform to user needs and intended uses, and
        that the particular requirements implemented
        through software can be consistently fulfilled.
    –   activities may occur both during, as well as at the
        end of the software development life cycle to ensure
        that all requirements have been fulfilled
Validation

  –   Since software is usually part of a larger hardware
      system, the validation of software typically includes
      evidence that all software requirements have been
      implemented correctly and completely and are
      traceable to system requirements
  –   In large measure, software validation is a matter of
      developing a "level of confidence" that the device
      meets all requirements and user expectations
Verification & Validation

   Software verification and validation are difficult
    because a developer cannot test forever, and it
    is hard to know how much evidence is enough.
   Conclusion
    –   Software is validated is highly dependent upon
        comprehensive software testing, inspections,
        analyses, and other verification tasks performed at
        each stage of the software development life cycle
Debugging

   Bug – a problem / fault in computer… software
    bugs & hardware bugs
   Debugging is the process of analyzing &
    finding bugs when software not behaves as
    expected
   Activity that supports testing but can not
    replace testing
   No testing guarantees to discover ALL BUGS
Software testing strategies

   Test methods into a well planned series of
    steps
   What is it ?
    –   Helps to answer many questions like;
            Should u test the entire program ? Or small part of it ?
            When shall we include the customer?
   Why ?
            To conduct test with minimum time and efforts to uncover
             maximum errors
   It provides the milestones to the development
    team.
Software testing strategies

   Proposed strategies
    –   Testing begins at the component level and works
        outward as and when integration is happening
    –   Different techniques at different times
    –   Should be conducted by the developer AND
        independent test group
    –   Should cover debugging
    –   Capable of doing low level testing as well
Unit testing

   Unit testing is the testing of individual components
    (units) of the software.
   Unit testing is usually conducted as part of a
    combined code and unit test phase of the software
    lifecycle.
   When developing a strategy for unit testing, there
    are three basic organizational approaches that can
    be taken. These are top down, bottom up and
    isolation.
General Concepts

 A test driver is software which executes
  software in order to test it, providing a
  framework for setting input parameters,
  executing the unit, and reading the output
  parameters.
 A stub is an imitation of a unit, used in place
  of the real unit to facilitate testing.
Top Down Testing

   Individual units are tested by using them from the units
    which call them, but in isolation from the units called.
   The unit at the top of a hierarchy is tested first, with all
    called units replaced by stubs.
   Testing continues by replacing the stubs with the actual
    called units, with lower level units being stubbed.
   This process is repeated until the lowest level units have
    been tested.
   Top down testing requires test stubs, but not test drivers.
Top Down Testing

                 A       Tested


    Tested           C    Tested          D Under test
B


      E          Stub       F      Stub      G     Stub



             H                     I                J
Top Down Testing

   Figure shows the test stubs and tested units needed
    to test unit D, assuming that units A, B and C have
    already been tested in a top down approach.
   Step (1)
    –   Test unit A, using stubs for units B, C and D.
   Step (2)
    –   Test unit B, by calling it from tested unit A, using stubs
        for units C and D.
Top Down Testing

   Step (3)
    –   Test unit C, by calling it from tested unit A, using tested units B and
        a stub for unit D.
   Step (4)
    –   Test unit D, by calling it from tested unit A, using tested unit B and
        C, and stubs for units E, F and G.
   Step (5)
    –   Test unit E, by calling it from tested unit D, which is called from
        tested unit A, using tested units B and C, and stubs for units F, G, H,
        I and J.
Top Down Testing
   Step (6)
    –   Test unit F, by calling it from tested unit D, which is called from
        tested unit A, using tested units B, C and E, and stubs for units G, H,
        I and J.
   Step (7)
    –   Test unit G, by calling it from tested unit D, which is called from
        tested unit A, using tested units B, C, E and F, and stubs for units H,
        I and J.
   Step (8)
    –   Test unit H, by calling it from tested unit E, which is called from
        tested unit D, which is called from tested unit A, using tested units B,
        C, E, F and G, and stubs for units I and J.
Top Down Testing

   Step (9)
    –   Test unit I, by calling it from tested unit E, which is
        called from tested unit D, which is called from tested unit
        A, using tested units B, C, E, F, G and H, and a stub for
        units J.
   Step (10)
    –   Test unit J, by calling it from tested unit E, which is
        called from tested unit D, which is called from tested unit
        A, using tested units B, C, E, F, G, H and I.
Top Down Testing

   Advantages
    –   Top down unit testing provides an early integration of
        units before the software integration phase. In fact, top
        down unit testing is really a combined unit test and
        software integration strategy.
    –   The detailed design of units is top down … tests in the
        sequence units are designed…so development time can
        be shortened by overlapping unit testing with the detailed
        design and code phases of the software lifecycle.
    –   Redundant functionality in lower level units will be
        identified by top down unit testing, because there will be
        no route to test it.
Top Down Testing

   Disadvantages
    –   controlled by stubs, with test cases often spread across
        many stubs. With each unit tested, testing becomes more
        complicated, and consequently more expensive to
        develop and maintain.
    –   As testing progresses down the unit hierarchy, it also
        becomes more difficult to achieve the good structural
        coverage which is essential for high integrity and safety
        critical applications,
Top Down Testing
   Disadvantages
     – Changes to a unit often impact the testing of sibling units
       and units below it in the hierarchy. For example, consider
       a change to unit D. Obviously, the unit test for unit D
       would have to change and be repeated. In addition, unit
       tests for units E, F, G, H, I and J, which use the tested
       unit D, would also have to be repeated. These tests may
       also have to change themselves, as a consequence of the
       change to unit D, even though units E, F, G, H, I and J
       had not actually changed. This leads to a high cost
     – requires structural knowledge
Top Down Testing

   Overall
    –   A top down strategy will cost more than an isolation
        based strategy, due to complexity of testing units below
        the top of the unit hierarchy, and the high impact of
        changes. The top down organizational approach is not a
        good choice for unit testing. However, a top down
        approach to the integration of units, where the units have
        already been tested in isolation, can be viable.
Bottom up Testing

   Units are tested in isolation from the units which call them,
    but using the actual units called as part of the test.
   The lowest level units are tested first, then used to facilitate
    the testing of higher level units.
   Other units are then tested, using previously tested called
    units. The process is repeated until the unit at the top of the
    hierarchy has been tested.
   Bottom up testing requires test drivers, but does not require
    test stubs.
Bottom up Testing

           A       Driver


               C                     D Under test
B


    E    Tested       F     Tested      G     Tested


        Tested              Tested           Tested
         H                    I                J
Bottom up Testing
   Figure shows the test driver and tested units needed to test
    unit D, assuming that units E, F, G, H, I and J have already
    been tested in a bottom up approach.
   Step (1) (Note that the sequence of tests within this step is
    unimportant, all tests within step 1 could be executed in
    parallel.)
    –   Test unit H, using a driver to call it in place of unit E;
    –   Test unit I, using a driver to call it in place of unit E;
    –   Test unit J, using a driver to call it in place of unit E;
    –   Test unit F, using a driver to call it in place of unit D;
    –   Test unit G, using a driver to call it in place of unit D;
    –   Test unit B, using a driver to call it in place of unit A;
    –   Test unit C, using a driver to call it in place of unit A.
Bottom up Testing

   Step (2)
    –   Test unit E, using a driver to call it in place of unit D and
        tested units H, I and J.
   Step (3)
    –   Test unit D, using a driver to call it in place of unit A and
        tested units E, F, G, H, I and J.
   Step (4)
    –   Test unit A, using tested units B, C, D, E, F, G, H, I and
        J.
Advantages

   All test cases are controlled solely by the test driver,
    with no stubs required.
   requiring no structural design information
   Bottom up unit testing provides an early integration
    of low level functionality, with higher level
    functionality being added in layers as unit testing
    progresses up the unit hierarchy. This makes bottom
    up unit testing readily compatible with the testing of
    objects.
Disadvantages
   As testing progresses up the unit hierarchy, bottom up unit
    testing becomes more complicated, and consequently more
    expensive to develop and maintain.
   Changes to a unit often impact the testing of units above it in
    the hierarchy. For example, consider a change to unit H.
    Obviously, the unit test for unit H would have to change and
    be repeated. In addition, unit tests for units A, D and E,
    which use the tested unit H, would also have to be repeated.
    These tests may also have to change themselves, as a
    consequence of the change to unit H, even though units A, D
    and E had not actually changed. This leads to a high cost
    associated with retesting when changes are made, and a high
    maintenance and overall lifecycle cost.
Disadvantages

   The first units to be tested are the last units to be designed,
    so unit testing cannot overlap with the detailed design phase
    of the software lifecycle.
   The sequence in which units can be tested is constrained by
    the hierarchy of units, with higher units having to wait for
    lower units to be tested, forcing a 'long and thin' unit test
    phase. The first units to be tested are the last units to be
    designed, so unit testing cannot overlap with the detailed
    design phase of the software lifecycle.
Overall
   The bottom up organisational approach can be a reasonable
    choice for unit testing, particularly when objects and reuse
    are considered. However, the bottom up approach is biased
    towards functional testing, rather than structural testing. This
    can present difficulties in achieving the high levels of
    structural coverage essential for high integrity and safety
    critical applications, and which are required by many
    standards. The bottom up approach to unit testing conflicts
    with the tight timescales required of many software
    developments. Overall, a bottom up strategy will cost more
    than an isolation based strategy, due to complexity of testing
    units above the bottom level in the unit hierarchy and the
    high impact of changes.
Isolation Testing

   Isolation testing tests each unit in isolation from
    the units which call it and the units it calls.
   Units can be tested in any sequence, because no unit
    test requires any other unit to have been tested. Each
    unit test requires a test driver and all called units are
    replaced by stubs.
Isolation Testing

            A       Driver


                C                   D Under test
B


    E       Stub       F     Stub      G     Stub



        H                    I                J
Isolation Testing

   Step (1)
    –   (Note that there is only one step to the test plan. The
        sequence of tests is unimportant, all tests could be
        executed in parallel.)
    –   Test unit A, using a driver to start the test and stubs in
        place of units B, C and D;
    –   Test unit B, using a driver to call it in place of unit A;
    –   Test unit C, using a driver to call it in place of unit A;
    –   Test unit D, using a driver to call it in place of unit A and
        stubs in place of units E, F and G,
Isolation Testing

   Test unit E, using a driver to call it in place of unit D and
    stubs in place of units H, I and J;
   Test unit F, using a driver to call it in place of unit D;
   Test unit G, using a driver to call it in place of unit D;
   Test unit H, using a driver to call it in place of unit E;
   Test unit I, using a driver to call it in place of unit E;
   Test unit J, using a driver to call it in place of unit E.
Advantages
   Isolation testing is the easiest way to achieve good structural
    coverage, and the difficulty of achieving good structural
    coverage does not vary with the position of a unit in the unit
    hierarchy.
   Because only one unit is being tested at a time, the test
    drivers tend to be simpler than for bottom up testing, while
    the stubs tend to be simpler than for top down testing.
   the removal of interdependency between unit tests, is that
    changes to a unit only require changes to the unit test for that
    unit, with no impact on other unit tests. This results in a
    lower cost than the bottom up or top down organisational
    approaches, especially when changes are made.
Advantages
   An isolation approach provides a distinct separation of unit
    testing from integration testing, allowing developers to focus
    on unit testing during the unit test phase of the software
    lifecycle, and on integration testing during the integration
    phase of the software lifecycle. Isolation testing is the only
    pure approach to unit testing, both top down testing and
    bottom up testing result in a hybrid of the unit test and
    integration phases.
   Unlike the top down and bottom up approaches, the isolation
    approach to unit testing is not affected by a unit being
    referenced from more than one other unit.
Disadvantages
   The main disadvantage of an isolation approach to unit
    testing is that it does not provide any early integration of
    units. Integration has to wait for the integration phase of the
    software lifecycle. (Is this really a disadvantage?).
   An isolation approach to unit testing requires structural
    design information and the use of both stubs and drivers.
    This can lead to higher costs than bottom up testing for units
    near the bottom of the unit hierarchy. However, this will be
    compensated by simplified testing for units higher in the unit
    hierarchy, together with lower costs each time a unit is
    changed.
Overall
   An isolation approach to unit testing is the best overall
    choice. When supplemented with an appropriate integration
    strategy, it enables shorter development timescales and
    provides the lowest cost, both during development and for
    the overall lifecycle.
   Following unit testing in isolation, tested units can be
    integrated in a top down or bottom up sequence, or any
    convenient groupings and combinations of groupings.
    However, a bottom up integration is the most compatible
    strategy with current trends in object oriented and object
    biased designs.
Overall

   An isolation approach to unit testing is the best way
    of achieving the high levels of structural coverage
    essential for high integrity and safety critical
    applications, and which are required by many
    standards. With all the difficult work of achieving
    good structural coverage achieved by unit testing,
    integration testing can concentrate on overall
    functionality and the interactions between units.
System testing

   Testing the system as a whole to validate that
    it meets its specifications and the objectives of
    the users
   Intended to test the system as a whole rather
    than individual system components
   Finds errors which were undiscovered during
    component testing
Stress testing

   Exercises the system beyond its maximum
    design load
   For abnormal situations
   In a manner that demands abnormal resources
    like quantity , frequency or volume
   As the load increases increase in the network
    traffic … at some point no useful work can be
    done
Security Testing

   Attempts to verify that protection mechanisms
    built into system will in fact protect it from
    improper work
   System and its data are protected from
    accidental damage
   Testing is carried out by requesting for
    penetration in to the system thru any loopholes
Performance testing

   Designed to test the run – time performance of the
    software
   Occurs throughout all steps in the testing process
   At the run time the performance of the individual
    module may be assessed as white box tests are
    conducted
   Until and Unless all units are integrated TOTAL
    performance test is not possible.
   Generally carried out along with the Stress testing
Software faults and failures
   What does it mean that the software has failed?
     – The software does not do what the requirements
       describe
   Reasons to the failures
     – Requirements incomplete, inconsistent, impossible to
       implement
     – Wrong interpretation of the requirements
     – The system design may contain a fault
       (misunderstanding
     – requirements, wrong architecture, incomplete design)
Software faults and failures

   Program design may contain a fault - wrong
    algorithm
   The program code may be wrong - improper,
    incomplete, inconsistent implementation
   Documentation can be wrong - can describe
    incorrectly the behavior of the system
Some other reasons to the failures
   Integration problems
    –   The components integrated into a systems are not compatible
        (wrong components, wrong component versions)
    –   The performance of the entire system does not
        correspond to the requirements (expectations)
    –   The system requires too much resources
    –   Not failures but still unsuccessful
            Bad Graphical User Interface
            Complicated process
            Missed points with requirements (users actually wanted
             something else)
software failure
• System failure occurs when the delivered service no longer complies
  with the specifications
• This definition applies to both hardware and software system failures.
• Faults or bugs in a hardware or a software component cause errors.
• An error is defined as that part of the system which is liable to lead to
  subsequent failure, and an error affecting the service is an indication that
  a failure occurs or has occurred.
• If the system comprises of multiple components, errors can lead to a
  component failure.
• As various components in the system interact, failure of one component
  might introduce one or more faults in another
Software failure

            Fault behavior
   Cost to repair the faults

$$$ to fix the
problems




                                                 Origin of problems

           Specification/   Design   Code   Integration
Fault classification - Types
Debugging

   Bug – a problem / fault in computer… software
    bugs & hardware bugs
   Debugging is the process of analyzing &
    finding bugs when software not behaves as
    expected
   Activity that supports testing but can not
    replace testing
   No testing guarantees to discover ALL BUGS
Debugging
   The purpose of debugging is to locate and fix the offending code
    responsible for a symptom violating a known specification.
    Debugging typically happens during three activities in software
    development, and the level of granularity of the analysis required
    for locating the defect differs in these three. The first is during the
    coding process, when the programmer translates the design into
    an executable code. During this process the errors made by the
    programmer in writing the code can lead to defects that need to be
    quickly detected and fixed before the code goes to the next stages
    of development. Most often, the developer also performs unit
    testing to expose any defects at the module or component level
Debugging
   The second place for debugging is during the later stages of
    testing, involving multiple components or a complete system,
    when unexpected behavior such as wrong return codes or
    abnormal program termination (“abends”) may be found. A certain
    amount of debugging of the test execution is necessary to
    conclude that the program under test is the cause of the
    unexpected behavior and not the result of a bad test case due to
    incorrect specification, inappropriate data, or changes in functional
    specification between different versions of the system. Once the
    defect is confirmed, debugging of the program follows and the
    misbehaving component and the required fix are determined. The
    third place for debugging is in production or deployment, when the
    software under test faces real operational conditions.
Debugging, testing, and verification
mapped to the software life cycle
 Debugging process

                  Execution
    Test                              End Results
                                   (Comparison with expected result)
   Cases


                                    Debugging
Corrections       Identification
                  Of
                                                 Suspected Causes
                  Causes
     Additional tests
Debugging
   The debugging process attempts to match symptom
    with cause
   Characteristics of BUGS
    –   Symptom and Cause may be geographically remote
    –   Symptom may be caused by Human error tahat is not easily
        traceable
    –   It may be difficult to reproduce accurately the same set of input
        conditions
   During Debugging , we‟ll have errors ranging from
    MILD to HEAVY
Approaches

   Debugging objective – to find and correct the
    case of software error
   Three approaches
    –   Brute force
            The most common – least efficient , “Let computer find the
             fault “ – strategy , memory dumps , run time traces etc,
             leads to wastage of time & money
Approaches

   Backtracking
    –   Common approach , used for small software, Where
        ever you encounter some symptom ,the source
        code has been traced MANUALLY backward , until
        the cause is found
    –   But , larger are the number of software lines , the
        number of potentially backward paths may rises
        unmanageably
Approaches

   Cause elimination
    –   A list of all possible causes is developed and tests
        are conducted to eliminate each of them
    –   Data related to error are isolated
    –   Depending on “Cause Hypothesis” data are used to
        prove or disprove

				
DOCUMENT INFO
Description: Basics Of Software Testing