Testing

							Software Testing
            Software Testing
•   Goals
•   Types of tests
•   Levels of tests
•   Test measures
•   Test plan
                  Goals
• Validation: Have we built the right
  software?
• Verification: Have we built the software
  right?
              Types of tests
• Black box
• White box
• Gray box
             Black box tests

input                                            output

                      interface




   1. Does it perform the specified functions?
   2. Does it handle obvious errors in input?
   Example: ordered list of ints
L=create()
L.insert(5)                      -1
                 class OrdInts
L.insert(-1)                     -1
                 create
L.insert(-1)     getFirst
                                 5
p=L.getFirst()   getNext         error
print (p)        insert

L.delete(p)      delete
                 print
p=L.getFirst()
print(p)
p=L.getNext(p)
print(p)
p=L.getNext(p)
            Black box tests
• Advantage: black box tester≠developer is
  unbiased by implementation details

• Disadvantage: black box tester is
  uninformed about implementation details
  – unnecessary tests
  – insufficient tests
        unnecessary tests
Input                 Code
        insufficient tests
Input                   Code
        insufficient tests
Input                   Code




                             complex
                              code
White box tests
 Based on code

                  test 1




                  test 2
     Example: ordered list of ints
class ordInts {
  public: …


    private:
     int vals[1000];
    int maxElements=1000;
    …
}
Example: ordered list of ints

bool testMax()
{
  L=create();
  num=maxElements;
   for (int i=0; i<=num; i++)
           L.insert(i)
   print maxElements;
}
            White box tests
• Advantage:
  – design tests to achieve good code coverage
    and avoid duplication
  – can stress complicated, error-prone code
  – can stress boundary values (fault injection)
• Disadvantage:
  – tester=developer may have bias
  – if code changes, tests may have to be
    redesigned
            Gray box tests
• Look at code to design tests
• But test through interface
   Example: ordered list of ints
L=create()
L.insert(1)                      1
                 class OrdInts
L.insert(2)                      2
                 create
L.insert(3)      getFirst
                                 3
                 getNext
                 insert
   …




                                 …
                 delete
                 print


L.insert(1001)
p=L.getFirst()
print(p)
p=L.getNext(p)
print p
    …
               Types of tests
• Black box:   test based on interface, through interface
• Gray box:    test based on code, through interface
• White box:   test based on code, through code




 Testing strategy can include all approaches!
              Levels of tests
•   Unit                        white

•   Component
•   Integration
•   System                      black

•   System integration
 Testing measures (white box)
• Code coverage
• Path coverage
• Code coverage based on complexity
             Code coverage
• How much of code is “tested” by tests?
  – manually
  – profiling tools
• Design new tests to extend coverage
• Is 100% good?
           Path coverage
• How many execution paths have been
  exercised by tests?
• 100% path coverage is usually impossible
• aim to cover commons paths and error
  prone (complex) paths
• aim to break code with tests
         convention wisdom
95% of errors are in 5% of the code



…or maybe 80% of the errors are in 20% of the code…
   code complexity measures
• cyclomatic complexity measure (McCabe)
  – basis path testing
  – ignores asynchronous interaction, fallibility of
    services
  – etc.
• developer’s best guess – what problems
  are likely and which will be hardest to
  diagnose
                Test plan
• Collection of tests: unit, component,
  integration, system
• Rationale for test: why these tests?
• Strategy for testing
                 Strategy
•   TDD – test driven development
•   Regression
•   Test harness
•   UI tests
        Test harness
input                       output




          everything else
         Test harness
input                        output




  test                         test
           everything else
 input                        output


             test harness