Verification and Validation by bac0A150


									  Software Testing
Strategies and Techniques
But First…
   Where have we been in this class
       gathering requirements
       defining the work to be done
       estimating time and effort
       scheduling work
       analyzing risks
       measuring our work
   Where are we going next
       testing our product
       managing change
    Starter Questions
   What is the purpose of testing software?
   Can we be sure the code is 100% correct?
   How much "correctness" should we shoot for?
   What needs to be tested?
   How do we test those parts/aspects?
   Given a finite schedule, what should a tester
    spend their time on?
   Given a finite budget, what can a manager do
    to improve testing effectiveness?
Testing Terms

   Verification - is the product correct

   Validation - is it the correct product
Laws of Testing
   The best person to test your code is
    someone else.
   A good test is one that finds an error.
   Testing can not prove the absence of
   Complete test coverage is impossible, so
    concentrate on problem areas.
   It cost a lot less to remove bugs early.
Testing Stages
      Unit Testing
           modules of code

      Integration Testing
           design

      Validation Testing
           requirements

      System Testing
           system engineering
Unit Testing Techniques
   White Box Testing
       testing a module of code based on the source code
       example - basis path testing

   Black Box Testing
       testing a module based just on what it is supposed
        to do
       example - boundary value analysis
Code Coverage
   Types of "Code Coverage"
       Statement coverage
            each line of code has been tested

       Conditional coverage
            each true/false decision has been tested

       Path coverage
            each unique path through the code has been tested
Level of Coverage
   Coverage Types are Apples and Oranges
       100% statement coverage ≠ 100% path coverage
       50% path coverage is probable
        better than 90% statement coverage

   100% path coverage is usually impractical
       N if-statements create up to 2n paths
       loops are worse
Basis Path Testing
     Objective is to test each conditional statement
                  as both true and false

1. Draw a Flow Graph

2. Determine the Cyclomatic Complexity
     count the regions, or
     CC = E - N + 2

3. Max Number of tests = CC

4. Derive a basis set of independent paths

5. Generate data to drive each path
void delete (int val, struct node **head)
   struct node *temp, *prev;
   prev = NULL;
   temp = *head;
1 while (temp!=NULL)                            1 - while
2    if (temp->value == val)
3         break;
     else                               5 - if              2 - if
4         prev = temp;
          temp = temp->next;
        }                             6        7      3 - break      4
5 if (prev == NULL)
6      *head = (*head)->next;             8
7      prev->next = prev->next->next;
Example Continued
   Complexity = 4                1 - while

                         5 - if               2 - if
   Basis Paths
       1241578
                     6            7     3 - break      4
       123578
       123568             8
       1568
Example Continued
void delete (int val, struct node **head)
   struct node *temp, *prev;                Path 1 2 4 1 5 7 8
   prev = NULL;                             1 to 2 non empty
   temp = *head;                            2 to 4 deleting non head
1 while (temp!=NULL)                        5 to 7 deleting non head
2    if (temp->value == val)
3        break;                             Path 1 2 3 5 7 8
                                            1, 2, 3 deleting the head
4        prev = temp;                       5 to 7 deleting non head
          temp = temp->next;
        }                                   Path 1 2 3 5 6 8
5 if (prev == NULL)                         1, 2, 3 deleting the head
     {                                      5 to 6 deleting the head
6     *head = (*head)->next;
     }                                      Path 1 5 6 8
                                            1 to 5 empty list
7     prev->next = prev->next->next;        5 to 6 deleting the head
Generating Paths
   If a path is not possible, then re-generate the
    list of basis paths

   An algorithm to generate only feasible paths
    is impossible (provably un-decidable)
Boundary Value Analysis
   Errors usually happen at the ends,
    not in the middle

   Example
       delete the head, size=1
       delete the head, size>1
       delete from end of the list, size>1
       delete when size=0
       delete when size≠0, not in list
   Create some test cases to verify the new
    "add alarm" feature.
Homework Results

12:00am                  12:00pm            11:59pm

          1.   12:00am
          2.   12:01am
          3.   11:59am
          4.   12:00pm     Most
          5.   12:01pm     error   11:59 = 11:59am
          6.   1:00pm              12:00 = 12:00pm
                                   12:01 = 12:01pm
          7.   12:59pm             13:00 = 1:00pm
Integration Testing
                        Objective is to test the
                     interfaces between modules
   Bottom-Up
         requires lots of drivers
         gives real answers early                 B       C       D
         big errors are not found until the end
                                                    E          F       G
   Top-Down
         requires lots of stubs
         gives fake answers until the very end
         big errors found early
                                                   Integration Testing
   Sandwich                                          necessitates
   Depth-First                                    Regression Testing
When do we stop unit and
integration testing?
   Answer: when the cost of finding bugs
    becomes greater than the cost of removing

of bugs
per day

Validation Testing
                Objective is to test the
              Conformity to Requirements

   Alpha Testing

   Beta Testing
Usability Testing

   Interface Design Heuristics

   Cognitive Walkthroughs
System Testing
   Recovery Testing
   Security Testing
   Stress Testing
   Performance Testing

Types of Testing Tools
   Regression Testing
         retest to find side-effects
   Database Testing
         test the queries
         build tests based on tables and fields
         load test the database
   Load Testing
         simulate a lot of users - stress test
   Functional Testing
         record a user's actions
         try lots of random user actions
         build test cases based on Use Case Scenarios
   Testing Web Applications
   Unit Testing
Example - load testing

 Example - functional testing - WinRunner
To create a test, Mercury WinRunner simply records a typical business process by emulating
user actions, such as ordering an item or opening a vendor account. During recording, you can
directly edit generated scripts to meet the most complex test requirements.
Next, testers can add checkpoints, which compare expected and actual outcomes from the test
run. Mercury WinRunner offers a variety of checkpoints, including test, GUI, bitmap, and Web
links. It can also verify database values to ensure transaction accuracy and database integrity,
highlighting records that have been updated, modified, deleted, and inserted.
With a few mouse clicks, Mercury WinRunner’s DataDriver Wizard lets you convert a recorded
business process into a data-driven test that reflects the unique, real-life actions of multiple
users. For further test enhancement, the Function Generator presents a quick and reliable way
to program tests, while the Virtual Object Wizard enables you to teach Mercury WinRunner to
recognize, record, and replay any unknown or custom object.
As Mercury WinRunner executes tests, it operates the application automatically, as though a
real user were performing each step in the business process. If test execution takes place after
hours or in the absence of a QA engineer, Mercury WinRunner’s Recovery Manager and
Exception Handling mechanism automatically troubleshoot unexpected events, errors, and
application crashes to ensure smooth test completion.
Example - unit testing java code
Teams that have tried manual creation of unit tests, will find that the automation
provided by Agitator makes developer testing practical for the first time.
Organizations that have deployed automated solutions for requirements
management, configurations management, systems, load, and performance testing
will find that Agitator is the perfect complement to deliver improved software quality
and reduced overall software lifecycle costs.

It is said that an effort is not worth doing if it can not be measured. Developer
Testing is no different. For successful application of Developer Testing, a total
of 20-40% of time dedicated to the project will be spent on the creation and
execution of unit tests. It is therefore critical that teams can evaluate that they
are allocating resources in the best possible way. The Agitar Management
Dashboard is the first of its kind,
created specifically to monitor and manage
Developer Testing efforts.
Rest of Semester …
   Nov 6 - Configuration Management
   Nov 13 - Software Quality Assurance
   Nov 20 - Quality Assurance Programs
   Nov 27 - Student Presentations

To top