Software Engineering Software Testing Strategies by pengtt


									                     Chapter 18
             Software Testing Strategies

These slides are based in part on those provided by the textbook publisher, the text book author, and Dr. Ralph
D. Meeker. They are copyright by various people including Roger S. Pressman & Associates, Inc., McGraw Hill
Companies, Inc., Dr. Ralph D. Meeker and Dr. John Kevin Doyle.

May 28, 2002                                                                                                      1
    • Testing begins at the module level and works
      outward toward the integration of the entire
      computer-based system.
    • Different testing techniques are appropriate at
      different points in time.
    • Testing is conducted by the developer of the
      software and (for large projects) an independent
      test group.
    • Testing and debugging are different activities,
      but debugging must be accommodated in any
      testing strategy.
May 28, 2002                                             2
               Verification and Validation
    • Verification – are we building the product
       – refers to the set of activities that ensure that
         software correctly implements a specific
    • Validation – are we building the correct
       – refers to the set of activities that ensure that
         the software that has been built is traceable
         to customer requirements.

May 28, 2002                                                3
                      Testing Strategy
          unit test                 integration

        system                           validation
         test                               test

May 28, 2002                                          4
                          Unit Testing

                               to be

               engineer            test cases

May 28, 2002                                              5
                   Unit Testing
                to be
                           local data structures
                           boundary conditions
                           independent paths
                           error handling paths

                        test cases

May 28, 2002                                       6
               Unit Testing Environment
                                     local data structures
               module                boundary conditions
                                     independent paths
                                     error handling paths
         stub     stub

                                  test cases

May 28, 2002                                                 7
         Integration Testing Strategies
    • The big bang approach – slap everything
      together, and hope it works. If not, debug it.
    • An incremental construction strategy – add
      modules one at a time, or in small groups, and
      debug problems incrementally.

May 28, 2002                                           8
               Top Down Integration

                                            top module is
                                            tested with stubs.

                           B        F       G

                               stubs are replaced one
                               at a time, “depth first”.
                               as new modules are integrated,
                               some subset of tests is re-run.
               D       E

May 28, 2002                                                     9
               Bottom-Up Integration


                           B       F       G

                               drivers are replaced one
                               at a time, “depth first”.
                               worker modules are grouped
                               into builds and integrated.
               D       E


May 28, 2002                                                 10
                   Sandwich Testing

                                   A        top modules are
                                           tested with stubs.

                           B       F      G

                               worker modules are grouped
                               into builds and integrated.
               D       E


May 28, 2002                                                    11
                Regression Testing
• Each time a new module is added as part of integration
  testing, or a bug is fixed in the software, the software
• These changes may cause problems with functions that
  previously worked.
• Regression testing is the re-execution of some subset of
  tests that have already been conducted to ensure that
  changes have not generated unintended side effects.
• The regression test suite should be designed to include
  only those tests that address one or more classes of
  errors in each of the major program functions.
 May 28, 2002                                                12
               High-Order Testing
     • Validation Testing
        • Verifies conformance with requirements
        • Answers the question “Did we build the
          correct product?”
     • Alpha and Beta Testing
        • Testing by the customer of a near-final
          version of the product.
     • System Testing
        • Testing of the entire software system,
          focused on end-to-end qualities.
     • Other Specialized Testing

May 28, 2002                                        13
                 Validation Testing
      • Validation succeeds when the software under
        test functions in a manner that can be
        reasonably be expected by the customer.
      • Validation is achieved through a series of
        black-box tests that demonstrate conformity
        with requirements.
      • The test plan should outline the classes of tests
        to be conducted and define specific test cases
        that will be used in an attempt to uncover
        errors in conformity with requirements.

May 28, 2002                                                14
               Alpha and Beta Testing

               Alpha Test
                    software       customer tests
                 developer site    customer site

               Beta Test              software
               developer reviews   customer tests
                 developer site     customer site

May 28, 2002                                        15
                   System Testing
      • Recovery testing – forces the software to fail
        in a variety of ways and verifies that recovery
        is properly performed.
      • Security testing – attempts to verify that
        protection mechanisms built into a system will
        in fact protect it from improper access.
      • Stress testing – executes a system in a
        manner that demands resources in abnormal
        quantity, frequency, or volume.

May 28, 2002                                              16
                   System Testing

      • Performance testing – tests run-time
        performance of software within the context of
        an integrated system.
      • End-to-end testing – test user scenarios, from
        end to end of the system.
      • Stability testing – test system over time (e.g.,
        72 hours) with normal load.

May 28, 2002                                               17
               A Diagnostic Process

May 28, 2002                          18
               The Debugging Process
           test cases

                  new test               results
       regression cases
          tests      suspected

May 28, 2002                                       19
                     Debugging Effort

               time required to
               correct the error
                 and conduct
               regression tests

                                              time required to
                                                diagnose the
                                                symptom and
                                            determine the cause
                            (always exactly 25% )

May 28, 2002                                                      20
               Symptoms and Causes
                        • Symptom and cause may be
                          geographically separated.
                        • Symptom may disappear when
                          another problem is fixed.
                        • Cause may be due to a combination
                          of non-errors.
                        • Cause may be due to a system or
                          compiler error.
                  cause • Cause may be due to assumptions
                          that everyone believes.
                        • Symptom may be intermittent.

May 28, 2002                                                  21
               Consequences of Bugs


     Damage                            extreme

                    mild annoying
                                          Bug Type
         Bug Categories: function-related bugs, system-related
         bugs, data bugs, coding bugs, design bugs,
         documentation bugs, standards violations, etc.
May 28, 2002                                                     22
               Debugging Techniques

          • Brute force – print out values of “all” the
            variables, at “every” step, look through
            the mass of data, and find the error.
          • Backtracking – begin with symptom of
            error. Trace backwards from there (by
            hand) to error.

May 28, 2002                                              23
               Debugging Techniques
     • Induction – moving from particular to general:
          • Run error-detecting test with lots of different
          • Based on all the data generated, hypothesize
            an error cause.
          • Run another test to validate the hypothesis.
          • If hypothesis is not validated, generate a new
            hypothesis and repeat.

May 28, 2002                                                  24
                Debugging Techniques
          • Deduction – moving from general to
               • Consider all possible error causes.
               • Generate tests to eliminate each (we
                 hope/expect all but one will succeed).
               • We may then be able to use further
                 tests to further refine the error cause.

May 28, 2002                                                25
                Debugging Hints

     • First, think about the symptom you are seeing
       – don’t run off half-cocked.
     • Use tools (e.g., dynamic debuggers) to gain
       more insight, and to make the steps
     • If at an impasse, get help from someone else.
     • Be absolutely sure to conduct regression tests
       when you do fix the bug.

May 28, 2002                                            26

To top