Software+Testing

Document Sample
Software+Testing Powered By Docstoc
					Software Testing




           Testing Strategies
Software Testing Strategies
●   First software quality assurance tools applied to
    control the software product's quality before its
    shipment
●   November 1994 survey by Perry
       24 % of project development budget was for testing
       32 % of project management budget was for testing
       27 % of project time was schedule for testing
       (actually they allocate 45 % of their schedule time for
        testing)
Outlines
●   2.1 Introduction
       Testing Objectives
●   2.2 Software Testing Strategies
●   2.3 Software Test Classifications
●   2.4 White Box Testing
●   2.5 Black Box Testing
Objective
●   Explain testing objectives
●   Discuss the difference between the various testing
    strategies
●   Describe the concept of black box and white box
    testing
Outlines
●   2.1 Introduction
       Testing Objectives
●   2.2 Software Testing Strategies
●   2.3 Software Test Classifications
●   2.4 White Box Testing
●   2.5 Black Box Testing
Definition Revisited
●   Myers
       Testing is the process of executing a program with
        intention of finding errors
●   Paul Jorgensen
       Testing is obviously concerned with errors, faults,
        failures and incidents. A test is the act of exercising
        software with test cases with an objective of Finding
        failure and Demonstrate correct execution
●   ISO
       Technical operation that consist of the determination of
        one or more characteristics of a given product, process
        or service according to a specified procedure
What is the objectives of Software
             Testing?
Direct Objectives
●   To identify and reveal as many errors as possible
    in the tested software
●   To bring the tested software, after correction of the
    identified errors and retesting to an acceptable
    level of quality
●   To perform the required tests efficiency and
    effectively, within budgetary and scheduling
    limitation
Indirect Objective
●   To compile a record of software errors for use in
    error prevention (by corrective and preventive
    actions)
Outlines
●   2.1 Introduction
       Testing Objectives
●   2.2 Software Testing Strategies
●   2.3 Software Test Classifications
●   2.4 White Box Testing
●   2.5 Black Box Testing
Stages of Testing
●   Module or unit testing.
●   Integration testing,
●   Function testing.
●   Performance testing.
●   Acceptance testing.
●   Installation testing.
Testing Strategies
●   We began by 'testing-in-the-small' and move
    toward 'testing -in-the-large'
●   For conventional software
       The module (component) is our initial focus
       Integration of module follows
●   For OO software
       OO class that include attributes and operations and
        implies communication and collaboration
Unit Testing
• Program reviews.
• Formal verification.
• Testing the program itself.
  – black box and white box testing.
Black Box or White Box?
• Maximum # of logic paths - determine if white
  box testing is possible.
• Nature of input data.
• Amount of computation involved.
• Complexity of algorithms.
Unit Testing Details
• Interfaces tested for proper information flow.
• Local data are examined to ensure that integrity is
  maintained.
• Boundary conditions are tested.
• Basis path testing should be used.
• All error handling paths should be tested.
• Drivers and/or stubs need to be developed to test
  incomplete software.
Generating Test Data
• Ideally want to test every permutation of valid and
  invalid inputs
• Equivalence partitioning it often required to
  reduce to infinite test case sets
  – Every possible input belongs to one of the equivalence
    classes.
  – No input belongs to more than one class.
  – Each point is representative of class.
Integration Testing Strategies
Test methodologies may vary but two basic testing
  strategies applied:
● Test the software in its entirety

       “Big Bang Testing”
●   Test the software in modules - Unit Tests,
    Integration Tests, Systems Tests
       “Incremental testing”
●   Two strategies for incremental testing
       Bottom - up testing (test harness).
       Top - down testing (stubs).
●   Sandwich Testing
Top-Down Integration Testing
• Main program used as a test driver and stubs are substitutes for
  components directly subordinate to it.
• Subordinate stubs are replaced one at a time with real components
  (following the depth-first or breadth-first approach).
• Tests are conducted as each component is integrated.
• On completion of each set of tests and other stub is replaced with a
  real component.
• Regression testing may be used to ensure that new errors not
  introduced.
Top Down Integration
                               A
                                        top module is tested with
                                        stubs


                       B        F       G

                           stubs are replaced one at
                           a time, "depth first"
               C

                           as new modules are integrated,
                           some subset of tests is re-run
           D       E


●   Stub is a piece of code emulating a called function
Top down – Advantages &
Disadvantages
●   Top-down integration supports fault isolation;
●   Major design flaws show up early. Modules of a
    product can be divided into two groups:
       The logic modules and operational modules.
       By coding and testing the logic modules before the
        operational modules, top-down integration will exploit
        any major design faults early in the development
        process
●   The main disadvantage of top-down integration is
    that potentially reusable modules may not be
    adequately tested.
Bottom-Up Integration Testing
• Low level components are combined in clusters that
  perform a specific software function.
• A driver (control program) is written to coordinate test
  case input and output.
• The cluster is tested.
• Drivers are removed and clusters are combined moving
  upward in the program structure.
Bottom-Up Integration
                                A




                         B       F       G


                             drivers are replaced one at a
                             time, "depth first"
                 C

                             worker modules are grouped into
                             builds and integrated
             D       E



             cluster
 ●   a driver is a piece of code emulating a calling
     function
Bottom-up – Advantages &
Disadvantages
●   The operational modules are thoroughly tested
    when using a bottom-up strategy. It also provides
    fault-isolation, as does top-down integration.
●   Major design faults will be left undetected until
    late in the development, since the logic module are
    integrated last.
●   This may result in large costs in redesigning and
    recoding substantial portions of the project.
Sandwich Testing
●   Combine the two so as to capitalize upon their
    strengths and minimize their weaknesses
●   neither top-down nor bottom-up
    implementation/integration is suitable for all the
    modules, the solution is to partition them
Sandwich Testing
                        A
                                 Top modules are
                                 tested with stubs


                 B       F       G




         C

                     Worker modules are grouped into
                     builds and integrated
     D       E



    cluster
Outlines
●   2.1 Introduction
       Testing Objectives
●   2.2 Software Testing Strategies
●   2.3 Software Test Classifications
●   2.4 White Box Testing
●   2.5 Black Box Testing
Software Test Classification
●   Software test may be classified
       according to the testing concept or
       to the requirement classification
According to Testing Concept
●   What concept to test?
       Output
         ●   Output is used to archive an acceptable level of quality
       Structural of the software
         ●   Internal structural and calculations involved is included for
             satisfactory testing
●   Two Classes have been developed
       Black box testing
         ●   Identify bug according to software malfunctioning
         ●   Functionality testing
       White box testing
         ●   Examine internal calculation paths in order to identify bug
         ●   Structural testing
According to Requirement
●   The test id carried out to ensure full coverage of
    the respective requirement
       Operation
         ●   Correctness, Reliability, Efficiency, Integrity, Usability
       Revision
         ●   Maintainability, flexibility, testability
       Transition
         ●   Portability, reusabilily, interoperability
●   From the requirement we can define the test
    classification
●   White box and black box can be used implemented
    respectively
  Test Classification according to
  Requirement
Correctness        1.1 Output correctness tests
                   1.2 Documentation tests
                   1.3 Availability tests
                   1.4 Data processing & calculations correctness tests
                   1.5 Software qualification tests

Reliability        2. Reliability tests
Efficiency         3. Stress test (load tests and durability tests)
Integrity          4. Software system security tests
Usability          5.1 Training usability tests
                   5.2 Operational usability tests

Maintainability     6. Maintainability tests
Flexibility        7. Flexibility tests
Testability        8. Testability tests
Portability        9. Portability tests
Re usability       10. Re usability tests

Interoperability   11.1 Software interoperability tests
                   11.2 Equipment interoperability test
IEEE definitions
Black box testing
Testing that ignores the internal mechanism of the
    system or component and focuses solely on the
    outputs in response to selected inputs and execution
    conditions
Testing conducted to evaluate the compliance of a
    system or component with specified functional
    requirements
White box testing
    Testing that takes into account the internal
    mechanism of a system or component
Outlines
●   2.1 Introduction
       Testing Objectives
●   2.2 Software Testing Strategies
●   2.3 Software Test Classifications
●   2.4 White Box Testing
●   2.5 Black Box Testing
White Box Testing
●   White Box Testing requires verification of every
    program statement and comment
●   White box testing enables performance of
       Data processing and calculation correctness tests
         ●   Every computation operation must be examined
       Software qualification tests,
         ●   Software code (including comments)
       Maintainability test
         ●   Failure causes detection, support adaption, software
             improvement
       Reusability test
         ●   Reusable for future software packages
Data processing and calculation
correctness tests
●   Checking the operation by each test cases
       Different path
       Which path is going to be tested?
●   Two approaches
       Path coverage
         ●   Path coverage of a test is measured by the percentage of all
             possible program paths included in planned testing.
       Line coverage
         ●   Line coverage of a test is measured by the percentage of
             program code lines included in planned testing.
Path coverage
●   Different path is created by condition statements
       IF-THEN-ELSE
       DO-WHILE
       DO-UNTIL
●   Consider a simple module with 10 conditional
    statement allowing two options each
       Creates 210 different path = 1024 path
       Examples:
         ●   Taximeter
     Taxi meter
1.   Minimal fare: $2. This fare covers the distance traveled up to
     1000 yards and waiting time (stopping for traffic lights or
     traffic jams, etc.) of up to 3 minutes.

 2. For every additional 250 yards or part of it: 25 cents.

 3. For every additional 2 minutes of stopping or waiting or part
    thereof: 20 cents.

 4. One suitcase: 0 change; each additional suitcase: $1.

 5. Night supplement: 25%, effective for journeys between 21.00
    and 06.00.

6. Regular clients are entitled to a 10% discount and are not
   charged the night supplement.
Flow Chart
                               1
                    Charge the minimal fare

         D > 1000                                 D ≤ 1000
                              2
     3                    Distance                             4


         WT > 3                                WT ≤ 3
                             5
    6                   Waiting time                            7


         S >1                 8                    S≤1
    9                  No.of suitcases                         10


            Yes              11                     No
    12                  Regular client?                        13

                                                    14
                                              Night journey?
                             15      Yes                       No

                                                                16
                            17
                      Print receipt.
Examples
●   24 different paths may be indicated
●   In order to archive full path coverage of the
    software module, we have to prepare at least 24
    test cases

●   We need to find a minimum number of paths to
    cover all line of code
       Line coverage
Flow Graph

                  1

                  2
                                  4
        3              R1
                  5
                                  7
        6              R2
                  8
   R6   9              R3         10
                  11
        12                        13
             R4              14
                  15
                        R5        16
                  17
The Minimum number of paths

                                        Minimun 3
                   1                    path is
                                        required
                   2
                                   4
         3              R1
                   5
                       R2          7
         6
                   8
    R6   9              R3         10
                   11
         12                        13
              R4              14
                   15
                         R5        16
                   17
Advantages of White box Testing
Advantages:
   * Direct determination of software correctness as expressed in
     the processing paths, including algorithms.
   * Allows performance of line coverage follow up.
   * Ascertains quality of coding work and its adherence to coding
     standards.
Disadvantages :
   * The vast resources utilized, much above those required for
     black box testing of the same software package.
   * The inability to test software performance in terms of
     availability (response time), reliability, load durability, etc.
Outlines
●   2.1 Introduction
       Testing Objectives
●   2.2 Software Testing Strategies
●   2.3 Software Test Classifications
●   2.4 White Box Testing
●   2.5 Black Box Testing
  Test Classification according to
  Requirement
Correctness    1.1 Output correctness tests
                  1.2 Documentation tests
                  1.3 Availability tests
                  1.4 Data processing & calculations correctness tests
                  1.5 Software qualification tests

Reliability         2. Reliability tests
Efficiency          3. Stress test (load tests and durability tests)
Integrity           4. Software system security tests
Usability           5.1 Training usability tests
                   5.2 Operational usability tests
Maintainability     6. Maintainability tests

Flexibility          7. Flexibility tests
Testability          8. Testability tests
Portability          9. Portability tests
Re usability 10. Re usability tests
      Interoperability      11.1 Software interoperability tests
                  11.2 Equipment interoperability test
Black Box Testing
●   Apart from correctness tests(those 2),
    maintainability and re usability, most of other
    testing classes are unique to black box testing
       Explain the importance of black box testing
●   However, due to the special characteristic of each
    testing strategy, black box testing cannot
    automatically substitute for white box testing
Black-Box Testing
●   Black-box testing is testing from a functional or
    behavioral perspective to ensure a program meets
    its specification
●   Testing usually conducted without knowledge of
    software implementation - system treated as a
    “black box”
●   Black-box test design techniques include:
       equivalence partitioning,
       boundary value analysis,
       cause-effect graphing,
       random testing
How much testing is adequate?

●   Completely validating IEEE 754 floating-point
    division requires 264 test-cases!
        float divide(float x, float y)

●   From practical and economic perspectives,
    exhaustive testing is usually not possible
       Which software pieces should we test?
       Which test cases should we choose?
Equivalence class partitioning (EC)
●   A black box method aimed at increasing the
    efficiency of testing and, at the same time,
    improving coverage of potential error conditions.
Boundary Value Analysis
●   Based on experience / heuristics
        Testing boundary conditions of equivalence classes is more
         effective
●   Choose input boundary values as equivalence classes
    representatives
●   Choose inputs that invoke output boundary values
●   Examples:
        (0, 10] ⇒ validate using 0, 1, 2, 9, 10, 11
        Read up to 5 elements ⇒ validate reading 0, 1, 4, 5, 6 elements
BVA as an equivalence partitioning
extension
●   Choose one (or more) arbitrary value(s) in each
    equivalence class
●   Choose valid values exactly on lower and upper
    boundaries of equivalence class
●   Choose invalid values immediately below and
    above each boundary (if applicable)
Equivalence Class Partitioning (EC)
●   An equivalence class (EC) is a set of input variable
    values that produce the same output results or that
    are processed identically.
●   EC boundaries are defined by a single numeric or
    alphabetic value, a group of numeric or alphabetic
    values, a range of values, and so on.
●   An EC that contains only valid states is defined as
    a "valid EC," whereas an EC that contains only
    invalid states is defined as the "invalid EC."
●   In cases where a program's input is provided by
    several variables, valid and invalid ECs should be
    defined for each variable.
Equivalence Class Partitioning (EC)
According to the equivalence class partitioning
  method:
● Each valid EC and each invalid EC are included in

  at least one test case.
● Definition of test cases is done separately for the

  valid and invalid ECs.
Equivalence Class Partitioning (EC)
 ●   In defining a test case for the valid ECs, we try to
     cover as many as possible “new” ECs in that same
     test case.
 ●   In defining invalid ECs, we must assign one test
     case to each “new” invalid EC, as a test case that
     includes more than one invalid EC may not allow
     the tester to distinguish between the program’s
     separate reactions to each of the invalid ECs.
 ●   Test cases are added as long as there are
     uncovered ECs.
Example – Ticket Price
●   Ticket price depends on four variables
       Day (weekday, weekend)
       Visitor's status (OT = one time, M = members)
       Entry Hour (6 – 19, 19.01 – 24)
       Visitor's age (up to 16, 16.01 – 60, 60.01 – 120)
Entrance Ticket Price table

   Day             on., ue., ed., hu,
                  M T W T Fri.                     Sat., Sun.
   Visitor’s      Ot    Ot    Mem Mem Ot          Ot    Mem Mem
   status
   E try
    n            6.00- 19.01- 6.00- 19.01- 6.00- 19.01- 6.00- 19.01
    o r
   h u           19.00 24.00 19.00 24.00 19.00 24.00 19.00      -
                                                              24.00


    g
   A e: 0.00-     $5    $6 $2.50     $3 $7.50     $9 $3.50 $4
   16.00

    g
   A e 16.01-    $10    $17    $5    $6    $15   $18      $7    $8
   60.00

    g
   A e: 60.01-    $8    $8     $4    $4    $12   $12 $5.50 $5.50
   120.00
Test cases – The Ticket price
  T est         T est Day Visitor’s Entry Visitor’s T case result
                                                     est
  case          case of   status hour age
  type           no. week
  V      C
   alid E s      1    Mn
                       o .    Ot     7.55    8.4             $5
                 2    Sat.   Mem    20.44   42.7             $8
                 3    Sat.   Mem    22.44   65.0           $5.50
                 4    Sat.   Mem     6.00    0.0           $3.50
                 5    Sat.   Mem    19.00   16.0           $3.50
                 6    Sat.   Mem    19.01   16.01            $8
                 7    Sat.   Mem    19.01   60.0             $8
                 8    Sat.   Mem    24.00   60.01          $5.50
                 9    Sat.   Mem    24.00   120.0          $5.50
  Inv      C
     alid E s    10    o
                      M x.    Ot     7.55    8.4         Invalid day
                 11    o .
                      Mn      88     7.55    8.4    Invalid visito statu
                                                                  r’s   s
                 12    o .
                      Mn      Ot     4.40    8.4      Inv       try ou
                                                         alid en h r
                 13    o .
                      Mn      Ot     8@      8.4      Inv       try ou
                                                         alid en h r
                 14   Mn
                       o .    Ot     7.55    T
                                            T R      Invalid visito ag
                                                                   r’s e
                 15    o .
                      Mn      Ot     7.55   150.1    Invalid visito ag
                                                                   r’s e
Advantage of Black Box Testing
Advantages:
● Allows us to carry out the majority of testing

  classes, most of which can be implemented solely
  by black box tests, i.e. load tests and availability
  tests.
● For testing classes that can be carried out by both

  white and black box tests, black box testing
  requires fewer resources.
Disadvantages of Black Box
Testing
Disadvantages:
● Possibility that coincidental aggregation of several

  errors will produce the correct response for a test
  case, and prevent error detection.
● Absence of control of line coverage. There is no

  easy way to specify the parameters of the test
  cases required to improve coverage.
● Impossibility of testing the quality of coding and

  its strict adherence to the coding standards.

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:74
posted:5/7/2010
language:English
pages:57