Ch7.ppt - Technical Resources by xumiaomaio

VIEWS: 4 PAGES: 43

									  Software Engineering




                Dr Zumao Weng
School of Computing and Intelligent Systems, UUM
                    2-11-2010
Software Verification
  and Validation
                  Objectives
 Be  able to differentiate between different approaches
  to verification
 Understand the testing process
 Be able to produce test cases using black-box and
  white box testing
 Be able to apply static verification
 Know about and how to conduct Integration and
  System Tests
 Be able to understand regression testing using junit
 Be able to identify the differences between software
  validation and verification
              What is this?

A failure?

 An   error?

A fault?
.


Need    to specify the desired
    behavior first!
          What is this?

Error!
Testing?
                      Definitions
        – any deviation in the observed behaviour
 Failure
 from the specified behaviour
  A  system failure occurs when the delivered service no
    longer complies with the specifications, the latter being an
    agreed description of the system's expected function
    and/or service.
      – the system is in a state where further
 Error
 processing will lead to failure
  A  software error resulting from bad code in some program
    involved in producing the erroneous result. An error is the
    part of the system state which may lead to a failure.
                           Definitions
 Fault – the cause of an error
    A fault is the cause of an error. A software fault, also known as a
     "crash" or "abend," is when the program directs the computer to go
     outside of its restricted memory boundary.
 Testing– the systematic attempt to find faults in a
  planned way in the implemented software
      Software testing is an investigation conducted to provide stakeholders
       with information about the quality of the product or service under test.
       Software testing also provides an objective, independent view of the
       software to allow the business to appreciate and understand the risks
       at implementation of the software. Test techniques include, but are not
       limited to, the process of executing a program or application with the
       intent of finding software bugs.
     Dynamic and static verification
 Dynamic verification Concerned with exercising and
 observing product behaviour (testing)
   includes   executing the code
 Staticverification Concerned with analysis of
 the static system representation to discover
 problems
   does   not include execution
                   Testing Activities
                               Requirements Specification

Subsystem
            Unit   Tested        System
  Code
            Test   Subsystem
                                 Design
                                Document                                 User
Subsystem   Unit                                                        Manual
  Code      Test
                                Integration                 System
               Tested              Test                      Test
               Subsystem                       Integrated              Functioning
                                              Subsystems                 System


                      Tested
Subsystem   Unit      Subsystem
  Code      Test                                                     developer
            Testing Activities continued
                 Client’s
              Understanding              User
             of Requirements          Environment


Validated                 Accepted
 System                    System
            Acceptance               Installation
               Test ()                Test ()

                                                    Usable
                                                    System


            User’s understanding
                                      System in              User/client
                                        Use
                    Types of Testing
 Unit   Testing:
   Individualsubsystem
   Carried out by developers
   Goal: Confirm that subsystems is correctly coded and
    carries out the intended functionality
 Integration    Testing:
   Groups  of subsystems (collection of classes) and
    eventually the entire system
   Carried out by developers
   Goal: Test the interface among the subsystem
           Types of Testing cont.
 System   Testing:
   The  entire system
   Carried out by developers
   Goal: Determine if the system meets the requirements
    (functional and global)
 Acceptance    Testing:
   Evaluates  the system delivered by developers
   Carried out by the client. May involve executing typical
    transactions on site on a trial basis
   Goal: Demonstrate that the system meets customer
    requirements and is ready to use
                                Unit testing
   objective - to discover defects in programs
   Successful test - a test which causes a program to behave in an
    anomalous way
   Tests show the presence not the absence of defects
   Only exhaustive testing can show a program is free from defects.
    However, exhaustive testing is impossible
   Test data = Inputs which have been devised to test system
   Test cases = Inputs to test the system and the predicted outputs from
    these inputs if the system operates according to its specification.


              test cases           test data                test results        test report

    Design test             Prepare            Run program           Compare results
      cases                Test Data           with test data         to test cases
              Unit Testing Techniques
 Informal:
      Incremental coding
 Static   Analysis:
      Hand execution: Reading the source code
      Walk-Through (informal presentation to others)
      Code Inspection (formal presentation to others)
      Automated Tools checking for
         syntactic and semantic errors

         departure from coding standards

 Dynamic     Analysis:
      Black-box testing (Test the input/output behavior)
      White-box testing (Test the internal logic of the subsystem or object)
      Data-structure based testing (Data types determine test cases)
                        Black-box testing
   Approach to testing where the program is considered as a ‗black-box‘
   The program test cases are based on the system specification
   Inputs from test data may reveal anomalous outputs i.e. defects
   Test planning can begin early in the software process
   Main problem - selection of inputs
       equivalence partitioning (a method for deriving test cases. In this method,
        classes of input conditions called equivalence classes are identified such that each
        member of the class causes the same kind of processing and output to occur. In
        this method, the tester identifies various equivalence classes for partitioning. A
        class is a set of input conditions that are is likely to be handled the same way by the
        system. If the system were to handle one case in the class erroneously, it would
        handle all cases erroneously. )


                        Test cases                              Test results
    Requirements
                                             Program
               Equivalence partitioning
 Partition     system inputs and outputs into ‗equivalence sets‘
      If an input should be between 4 and 10. choose
           1 number < 4, 1 number between 4-10 and 1 number >10
      If input is a 5-digit integer between 10,000 and 99,999,
       equivalence partitions are
           <10,000, 10,000-99,999 and > 10, 000
 Choose       test cases at the boundary of these sets
                                           3                                  11
                                               4            7            10



                           Les s th an 4           B etw een 4 an d 10          M o re t han 10

                     Nu m b er of i np ut v alu es

                                    9 99 9                                   1 00 00 0
                                       1 00 00           5 00 00         9 99 99



                 Les s th an 1 00 00           B etw een 10 00 0 an d 99 99 9      M o re t han 99 99 9

               Inp ut valu es
                       White box testing
 Sometimes   called structural testing or glass box testing
 Derivation of test cases according to program structure.
  Knowledge of the program used to identify additional test
  cases
 Objective is to exercise all program statements (not all path
  combinations)                                Test
                                               cases
 Four types of white-box testing
      Statement Testing – test single statements
      Loop Testing – test loop for                                   Code Code Code Code Code Code Code Code Code Code Co
                                                                                          de Code Code
                                                    Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code
                                                                Code Code Code Code Code Code Code Code Code Code Code Code Code

                                                                               Code Code Code Code Code Code Code
                                                                                    Code Code Code Code Code
                                                                                                                                                Test results
                                                     Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Co
                                                                de Code Code Code Code Code Code Code Code Code Code Code Code Co




            Execute 0, 1 and n times
                                                                                        e Code Code Code Co




        
                                                                         de Code Code Code Code Code Code Code Code Code
                                                    Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code
                                                                                         Code Code Code
                                                    Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code
                                                    Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code
                                                    Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code Code
                                                                               Code Code Code Code Code Code Code




      Path Testing – execute every path
      Branch Testing – every outcome from every condition is tested
                              Path Testing
 Program                flow graphs
   Describes  the program control flow
   Used as a basis for computing the cyclomatic
    complexity
   Complexity = Number of edges - Number of nodes + 2




     i f- t h e n - e ls e        l o o p - w h il e   c a se - o f
    Example – Find the Mean method
public class Calculator{
  public double findMean(String filename) throws IOException{
      double meanvalue = 0.0;
      double inputValue;
      BufferedReader infile = null;
      FileReader fr = null;
      String line = null;                                      1
      int numbNumbers = 0;
      double sumOfNumbers = 0.0;
      // read in the numbers from the file
      infile = new BufferedReader( new FileReader( filename ));
      while ( (line=infile.readLine()) != null ) {       2
      // readLine returns null when EOF is reached
                  inputValue = Double.parseDouble(line);
                  if (inputValue > 0.0){                     3
                     sumOfNumbers = sumOfNumbers + inputValue;
                     numbNumbers++;                              4
                  }
      }                 5
   6 //calculate and return mean
      if (numbNumbers > 0)            7
            return sumOfNumbers/ numbNumbers;   8
      else
            return 0;     9
}
    1
}
    0
Example – Find the Mean method cont.
                    1    Initialising variables

 (a)                          while               CC = 12-10+2 = 4
                    2
              (b)                                 •    1-2-3-4-6-7-8-10
                    3    if                       •    1-2-3-5-6-7-8-10
             (c)                    (d)           •    1-2-3-4-6-2-7-9-10
                                                  •    1-2-7-8-10
       4                        5 <=0
                                                  Data
   end while        6                             (a) Empty file
                                                  (b) At least 1 number in file
                                                  (c) Number > 0
       (e)          7    if                       (d) Number <= 0
                                     (f)
                                                  (e) At least 1 valid number
       8                        9                 (f) No valid numbers

                    10        exit
                       Binary search flow graph
class BinSearch {
public static void search( int key, int[] elemArray, Result r)       1
{     int bottom = 0;
      int top = elemArray.length - 1;
      int mid;                                1
                                                                     2
      r.found =false; r.index = -1;

2     while (bottom <= top)
      {       mid = (top + bottom) /2;                               3
          3   if (elemArray [mid] == key
              {            r.index = mid;
                           r.found = true;                       8           4
                           return;
              }
              else
            4 { if(elemArray[mid] < key)                         9       5           6
                           bottom = mid + 1;       5
                 else
                           top = mid -1;       6
                                                                                 7
              }      8
      }//while 7
} //search
}//BinSearch              9                                                      CC = 4
                        Independent paths
 CC  = The number of tests to test all control statements equals
  = number of conditions in a program + 1
 Independent Paths
      1, 2, 3, 8, 9
      1, 2, 3, 4, 6, 7, 2
      1, 2, 3, 4, 5, 7, 2
      1, 2, 3, 4, 6, 7, 2, 8, 9
 Testcases should be derived so that all of these paths are
  executed
 A dynamic program analyser may be used to check that paths
  have been executed
     Comparison of White & Black-box
 White-box     Testing:
              infinite number of paths have to be tested
   Potentially
   White-box testing often tests what is done, instead of what
    should be done
   Cannot detect missing use cases

 Black-box     Testing:
   Potential combinatorial explosion of test cases (valid &
    invalid data)
   Often not clear whether the selected test cases uncover a
    particular error
   Does not discover extraneous use cases ("features")
                      Static verification
 Verifying the conformance of a software system and its
  specification without executing the code
 Involves analysis of source text by humans or software
 on ANY documents produced as part of the software process
 Discovers errors early in the software process
 Usually more cost-effective than testing for defect detection at
  the unit and module level
 > 60% of program errors can be detected program
  inspections
 2 main techniques
      walkthroughs - 4-6 people examine code & list perceived problems,
       then meet to discuss
      program inspections (Fagin) - formal approach using checklists
                Fagin Software Inspections
   Typically involve:
        Author – the programmer
        Inspector – finds errors, omissions, inconsistencies
        Reader – paraphrases the code/ document at the meeting
        Scribe – records results
        Chairman – manages the process
        Chief moderator – improving the inspection process, updating the checklist
   Checklist may look for
        Data faults e.g. variables not initialised, variables declared but not used,
         variables assigned twice but not used in between, array bound violations,
         undeclared variables
        Control faults e.g. unreachable code, unconditional branches
        I-O faults e.g. variables output twice but nothing changes in between
        Interface faults e.g. parameter type mismatch, parameter number mismatch,
         uncalled methods
        Storage management faults e.g. unassigned pointers, pointer arithmetic.
                     Stress Testing

 Some     systems need to handle specified loads
       transaction processing systems
   E.g.
   Web-based systems

 Stress   Testing
        the failure behaviour of the system
   Tests
   Shows up defects that only occur under high volume
        Integration Testing Strategy

 The order in which the subsystems are selected for
 testing and integration determines the testing
 strategy
   Big bang integration (Nonincremental)
   Bottom up integration
   Top down integration
   Sandwich testing
Example: Three Layer Call Hierarchy

                A
                              Layer I




        B           C   D      Layer II




    E       F           G      Layer III
Integration Testing: Big-Bang Approach
    Unit Test
        A
                       Not a good idea!
    Unit Test
        B
    Unit Test
        C
                           System Test
    Unit Test
        D
    Unit Test
        E
    Unit Test
        F
            Bottom-up Testing Strategy
   Lowest layer subsystems tested individually
   Next subsystems tested that call those subsystems
   Repeat until all subsystems are included in the testing
   Need Special program = ―Test Driver‖
      Test Driver calls a subsystem and passes a test case to it


Test E                                                  Disadvantages
                                                             Tests the most important
             Test B, E, F                                     subsystem (UI) last
Test F                                                  Advantages
                    Test C                Test
                                       A, B, C, D,
                                                             Useful for integrating OO
                                         E, F, G              systems

                            Test D,G
 Test G
               Top-down Testing Strategy
    Test top layer i.e. controlling subsystem first
    Test each of the called subsystems
    Repeat until all subsystems are incorporated into the test
    Special program is needed to do the testing, Test stub (A dummy
     software component or object used (during development and testing) to simulate the
     behaviour of a real component. The stub typically provides test output. ):
       simulates the activity of a missing subsystem  Disadvantages
                                                           Writing stubs difficult
                                                           very large number of
                                               Test
                                                            stubs may be required
  Test A        Test A, B, C, D            A, B, C, D,
                                             E, F, G    Advantages
                                                           Test cases can be
Layer I                                                     defined in terms of the
                Layer I + II                                functionality of the
                                                            system (functional
                                           All Layers       requirements)
                   Sandwich Testing Strategy
     Combines top-down strategy with bottom-up strategy
     How do you select the target layer if there are more than 3 layers?
            Heuristic: Try to minimize the number of stubs and drivers


                 Test E                                                  Disadvantages
                                                                              Does not test
                                    Test B, E, F                               the individual
Bottom           Test F
Layer                                                                          subsystems
                                                           Test                thoroughly
 Tests                                                  A, B, C, D,            before
                                  Test D,G                E, F, G
                 Test G                                                        integration
                                                                         Advantages
                                     Test A,B,C, D
                                                                              Parallel Top
  Top            Test A                                                        and Bottom
 Layer                                                                         Layer Tests
 Tests
     Modified Sandwich Testing Strategy
                          Double
                          Test I
                      A
                                   Layer I

              B       C      D
                                   Layer II

Triple    E       F          G
                                   Layer III
Test I
                                 Double
                                 Test II

 Test   in parallel:
      Middle  layer with drivers and stubs
      Top layer with stubs
      Bottom layer with drivers
   Test in parallel:
      Top layer accessing middle layer (top layer replaces drivers)
      Bottom accessed by middle layer (bottom layer replaces stubs)
            Steps in Integration-Testing
1. Based on the integration             4. Do structural testing: Define
   strategy, select a component to         test cases that exercise the
   be tested. Unit test all the            selected component
 . classes in the component.            5. Execute performance tests
2. Put selected component               6. Keep records of the test cases
   together; do any preliminary fix-       and testing activities.
   up necessary to make the
   integration test operational         7. Repeat steps 1 to 7 until the full
   (drivers, stubs)                        system is tested.
3. Do functional testing: Define test
   cases that exercise all uses         The primary goal of integration
   cases with the selected                testing is to identify errors in the
   component                              (current) component
                                          configuration.
                       System Testing
 Functional  Testing (same as black box)
 Structure Testing (same as white box)
 Performance Testing
 Acceptance Testing
 Installation Testing


 Impact   of requirements on system testing:
     The more explicit the requirements, the easier they are to test.
     Quality of use cases determines the ease of functional testing
     Quality of subsystem decomposition determines the ease of structure
      testing
     Quality of nonfunctional requirements and constraints determines the
      ease of performance tests:
                       Performance Testing
   Stress Testing                                 Timing testing
        Stress limits of system (maximum #             Evaluate response times and
         of users, peak demands, extended                time to perform a function
         operation)                                Environmental test
   Volume testing                                      Test tolerances for heat,
        Test what happens if large amounts              humidity, motion, portability
         of data are handled                       Quality testing
   Configuration testing                               Test reliability, maintain- ability
        Test the various software and                   & availability of the system
         hardware configurations                   Recovery testing
   Compatibility test                                  Tests system‘s response to
        Test backward compatibility with                presence of errors or loss of
         existing systems                                data.
   Security testing                               Human factors testing
        Try to violate security requirements           Tests user interface with user
                  Acceptance Testing
      Demonstrate
 Goal:                              Alpha   test:
 system is ready for                    Sponsor   uses the software at
 operational use                         the developer’s site.
    Choice  of tests is made by        Software used in a controlled
     client/sponsor                      setting, with the developer
    Many tests can be taken
                                         always ready to fix bugs.
     from integration testing        Beta   test:
    Acceptance test is                 Conducted   at sponsor’s site
     performed by the client, not        (developer is not present)
     by the developer.                  Software gets a realistic
                                         workout in target environ-
                                         ment
                                        Potential customer might get
                                         discouraged
   Regression Testing – Example XUnit
 JUnit   - Erich Gamma and Kent Beck - http://www.junit.org
      Also Dunit, CPPUnit and others
 unit   tests in Java that can be run at any time
      Typically after some change has been made
 OSS  - Open Source Software – anyone can modify/develop
 Part of XP
 Automated
 Test First – i.e. write the tests before the code!
      New code only written when there is a test that doesn‘t work
Breakdown of topics for the Software Quality
Software Verification and Validation (V&V)
   is the process of checking that a software system meets specifications
    and that it fulfils its intended purpose. It is normally part of the software
    testing process of a project.
   Validation checks that the product design satisfies or fits the intended
    usage (high-level checking) — i.e., you built the right product. This is done
    through dynamic testing and other forms of review.
   According to the Capability Maturity Model (CMM):
        Verification: The process of evaluating software to determine whether the products of a
         given development phase satisfy the conditions imposed at the start of that phase.
         [IEEE-STD-610].
        Validation: The process of evaluating software during or at the end of the development
         process to determine whether it satisfies specified requirements. [IEEE-STD-610]
        In other words, validation ensures that the product actually meets the user's needs, and
         that the specifications were correct in the first place, while verification is ensuring that
         the product has been built according to the requirements and design specifications.
         Validation ensures that ‗you built the right thing‘. Verification ensures that ‗you built it
         right‘. Validation confirms that the product, as provided, will fulfill its intended use.
Difference between verification and validation

♦ Verification -> Are we building product right?
♦ Validation -> Are we building right product?
♦ Verification evaluates documents, plans, code, requirements, and
  specifications.
♦ Validation, on the other hand, evaluates the product itself.
♦ The inputs of verification are checklists, issues lists, walkthroughs and
  inspection meetings, reviews and meetings.
♦ The input of validation, on the other hand, is the actual testing of an actual
  product.
♦ The output of verification is a nearly perfect set of documents, plans,
  specifications, and requirements document.
♦ The output of validation, on the other hand is the actual/perfect product.
♦ Verification comes before Validation.
                   Lab Questions
  Identify five stages involved in the static analysis of
  verification and validation process.
 Why performance testing is important for Web based
  systems? Give an example to clarify your explanation.
 Identify the differences between white box testing and black
  box testing.
 Discuss four strategies in integration testing and compare the
  advantages and disadvantages of the strategies.
 Identify the differences between software validation and
  verification.

								
To top