Representational Theory of Measurement _chapter 2_

Document Sample
Representational Theory of Measurement _chapter 2_ Powered By Docstoc
					          SOEN345 – Winter 2011
      Software Testing,
    Verification and Quality
         Instructor: Dr. Olga Ormandjieva

Lecture 13: Assignment 2 discussion. Sample
         questions for the midterm

                  SOEN345-W11-L13: A2, review   1
   A2 discussion
   Review for the midterm

              SOEN345-W11-L13: A2, review   2
Due on Monday March 7, 2011
A2 Deliverables:
 test plan, traceability matrix, test
  progress report and defects report

              SOEN345-W11-L13: A2, review   3
Test plan

            SOEN345-W11-L13: A2, review   4
Test case design

          SOEN345-W11-L13: A2, review   5
    Order of Implementation and Testing
   Indicate the order in which units are to be tested and
        Classes need to be implemented (and fully tested) from least-coupled to
         most coupled see Figure below.
          1   address : Address        1
                                                                       1                                                      2
              name : Text
                                                                               3                         ProductSpecification
                                                                                                        description : Text
                                                          ...                          Contains
                                                                                                        price : Money
                     1                 Looks-in       1                            1               *
                                                                                                 1..    itemID : ItemID
                     1         1                                              5                         Describes
                                   6                                                                                 *
                    Register                              date : Date                                                         4
                                                          isComplete : Boolean                                SalesLineItem
                                           Captures       time : Time                  Contains
                                                                                                   quantity : Integer
              endSale()                1              1                       1                  *
              enterItem(...)                                                                       getSubtotal()

                                                                       1                                                      1
                         Logs-completed                     *                                                 Payment
                                                                                                  amount : Money

                                                   SOEN345-W11-L13: A2, review                                                    6
Status report on your testing
   For those tests that you have
    completed, indicate which tests passed
    and which failed. If you have not yet
    completed testing, give a schedule and
    resource estimate for the remainder of
    your testing process.

              SOEN345-W11-L13: A2, review   7
Testing traceability matrix

         SOEN345-W11-L13: A2, review   8
Defect report and tracking

         SOEN345-W11-L13: A2, review   9
One possible classification of
defect types

         SOEN345-W11-L13: A2, review   10
Defect report and tracking
   Follow the template provided in the
    following webpage:
   As a quick defect tracking tool, just put
    all these columns in an excel

               SOEN345-W11-L13: A2, review   11
     Defect life cycle
   What is a Bug Life Cycle?
    The duration or time span between the first time bug is found („New‟) and
    closed successfully (status: „Closed‟), rejected, postponed or deferred is
    called as „Bug/Error Life Cycle‟.

    (Right from the first time any bug is detected till the point when the bug is
    fixed and closed, it is assigned various statuses which are New, Open,
    Postpone, Pending Retest, Retest, Pending Reject, Reject, Deferred, and
    Closed. For more information about various statuses used for a bug during a
    bug life cycle, you can refer to article „Software Testing – Bug & Statuses
    Used During A Bug Life Cycle‟)

                             SOEN345-W11-L13: A2, review                     12
      Defect life cycle

           SOEN345-W11-L13: A2, review           13
   A2 discussion
   Review for the midterm

              SOEN345-W11-L13: A2, review   14
Homework 1 solution: discussion

      In class

                  SOEN345-W11-L13: A2, review   15
Homework 2 solution: discussion

     In class

                 SOEN345-W11-L13: A2, review   16
Exercise 2 answer

        SOEN345-W11-L13: A2, review   17
Sample exam questions
   What is a test case? What is test-case
   What are test plans and test cases?
    Illustrate each with an example.

              SOEN345-W11-L13: A2, review   18
         Sample exam questions
   Explain the difference between error, fault, failure.
   Explain the difference between debugging and testing

                       SOEN345-W11-L13: A2, review   19
Sample exam questions
   What is the difference between black-
    box testing and white-box testing? List
    their advantages and disadvantages.

               SOEN345-W11-L13: A2, review   20
Sample exam questions
   What are test oracles?
                                                 Test Set

             User Expectations, Specification,
            anticipated behaviour requirements                    Program Under Test
                or reasonable expectations

                     Expected Output                               Observed Output

                                                Are they

                                          No                Yes

                                 Fault is detected      Conformed

                      SOEN345-W11-L13: A2, review                                      21
Sample exam questions
   Suppose a developed software has
    successfully passed all the three levels
    of testing, i.e., unit testing, integration
    testing, and system testing. An we
    claim that the software is defect-free?
    Justify your answer

                SOEN345-W11-L13: A2, review   22
         Sample exam questions
   What are drivers and stub modules in the context of
    integration testing of a software product?
       a test driver sets up the environment and makes a series of calls to
        the unit-under-test. Effectively it mimics the units that call the unit-
        under-test, if these units are not yet ready for integration.
       • a test stub acts as a dummy for a dependent of the unit-under-
        test. Effectively it mimics the units that the unit under-test calls if
        these are not yet ready for integration. A simple way to provide a
        stub is to provide a dummy function with the same name and
        parameters as the real dependent unit, but which merely asks the
        tester to enter a sensible return value to be passed back to the unit-

                               SOEN345-W11-L13: A2, review                  23
Sample exam questions
   What is regression testing? Why is it
    important? When is it used?

               SOEN345-W11-L13: A2, review   24
    Regression testing
 It tends to be far too expensive to re-run every single
  test case every time a change is made to software.
 Hence only a subset of the previously-successful test

  cases is actually re-run.
 The tests that are re-run are called regression tests.

 Regression test cases are carefully selected to cover as

  much of the system as possible.
The “law of conservation of bugs”:
The number of bugs remaining in a large system is
 proportional to the number of bugs already fixed

                     SOEN345-W11-L13: A2, review        25
    Regression testing
 Regression testing can become very
tedious if a test set contains many tests that
require user actions:
Test 23.4: Help functions.
 • Move the cursor to the Help icon.
 • Click the left mouse button once. A popup menu
 • Move the cursor to the Contents box.
 • Click the left mouse button once.
 • ....
                  SOEN345-W11-L13: A2, review   26
    Regression testing
 Regression testing on a test set is carried
out as follows:
 Perform a subset of tests from the test set
one after another.
 When a test fails, stop testing and wait until
the fault has been corrected.
 Start the regression testing again

                  SOEN345-W11-L13: A2, review   27
Black-box testing
                                                     Inputs causing
   Input test data           I                       behaviour


                                                     Outputs which reveal
                                                     the presence of
 Output test results        Oe                       defects

                       SOEN345-W11-L13: A2, review                          28
Example: ATM
   For a simple ATM banking transaction system
       Conditions (inputs)
            C1:   Command is credit
            C2:   command is debit
            C3:   account number is valid
            C4:   transaction amount is valid
       Actions (outputs)
            E1:   Print “invalid command”
            E2:   Print “ invalid account number”
            E3:   Print “debit amount not valid”
            E4:   debit account
            E5:   Credit account

                         SOEN345-W11-L13: A2, review   29
ATM Cause-effect decision
table            Don’t Care
Cause\effect   R1      R2            R3           R4   R5
C1             0       1             x            x    1
C2             0       x             1            1    x
C3             x       0             1            0    1
C4             x       x             0            1    1
E1             1       0             0            0    0
E2             0       1             0            0    0
E3             0       0             1            0    0
E4             0       0             0            1    0
E5             0       0             0            0    1

                    SOEN345-W11-L13: A2, review             30
     Equivalence classes
   It is inappropriate to test by brute force, using every
    possible input value
        Takes a huge amount of time
        Is impractical
        Is pointless!

   You should divide the possible inputs into groups which
    you believe will be treated similarly by all algorithms.
        Such groups are called equivalence classes.
        A tester needs only to run one test per equivalence class
        The tester has to
             understand the required input,
             appreciate how the software may have been designed

                             SOEN345-W11-L13: A2, review             31
Examples of equivalence

        SOEN345-W11-L13: A2, review   32
Combinations of equivalence
   Combinatorial explosion means that you cannot
    realistically test every possible system-wide
    equivalence class.
        If there are 4 inputs with 5 possible values there are 54
         (i.e.625) possible system-wide equivalence classes.
   You should first make sure that at least one test is
    run with every equivalence class of every
    individual input.
   You should also test all combinations where an
    input is likely to affect the interpretation of
   You should test a few other random combinations
    of equivalence classes.
                      SOEN345-W11-L13: A2, review                    33
Example equivalence class
   One valid input is either „Metric‟ or „US/Imperial‟
        Equivalence classes are:
             Metric, US/Imperial, Other
   Another valid input is maximum speed: 1 to 750
    km/h or 1 to 500 mph
        Validity depends on whether metric or US/imperial
        Equivalence classes are:
             [-∞..0], [1..500], [501..750], [751.. ∞]
   Some test combinations
             Metric, [1..500]                   pass
             US/Imperial, [501..750]            fail
             Metric, [501..750]                 pass
             US/Imperial, [1..500]              pass
                           SOEN345-W11-L13: A2, review       34
Testing at boundaries of
equivalence classes
   More errors in software occur at the boundaries of
    equivalence classes
   The idea of equivalence class testing should be
    expanded to specifically test values at the
    extremes of each equivalence class
        E.g. The number 0 often causes problems

   E.g.: If the valid input is a month number (1-12)
        Test Equivalence classes are: [-∞..0], [1..12], [13.. ∞]
        Test 0, 1, 12 and 13 as well as very large positive and
         negative values

                        SOEN345-W11-L13: A2, review                 35
Example: ATM
   Consider data maintained for
     User should be able to access the
      bank using PC and modem
     User should provide six-digit
     Need to follow a set of typed
             SOEN345-W11-L13: A2, review   36
Data format
   Software accepts
       Area code:
            Might be blank or three-digit
       Prefix:
            three-digit number not beginning with 0 or 1
       Suffix:
            four digits number
       Password: six digit alphanumeric value
       Command:
            {“check”, “deposit,” “ bill pay”, “transfer” etc.}

                        SOEN345-W11-L13: A2, review               37
Input conditions for ATM
   Input conditions
       Area code:
            Boolean: the area code may or may not be present
            Range: values defined between 200-999
            Specific value: no value > 905
       Prefix: range –specific value >200
       Suffix: value (four-digit length)
       Password:
            Boolean: password may or may not be present
            Or value – six char string
       Command: set containing commands noted
                      SOEN345-W11-L13: A2, review               38
Boundary Value Analysis
       complements equivalence partitioning
       Works with single fault assumption in reliability
       Focuses is on the boundaries of the input
         If input condition specifies a range bounded by a certain
          values, say, a and b, then test cases should include
             The values for a and b
             The values just above and just below a and b
       If an input condition specifies any number of
        values, test cases should be exercise
             the minimum and maximum numbers,
             the values just above and just below the minimum and
              maximum values

                      SOEN345-W11-L13: A2, review                    39
More on BVA
   Generalizing BVA are done
       Using the number of variables
            Example for Function of F with n variables, need at least
             4n+1 test cases
       The type of ranges
            Logical quantity vs physical bounded quantities
                  E.g. of discrete and bounded: months of year,
                  E.g., discrete and unbounded: length of sides in triangle
                      Need to create artificial min/max bounds

                          SOEN345-W11-L13: A2, review                     40
H3 solution

SOEN345-W11-L13: A2, review   41
Valid partitions
   The valid partitions can be
     0<=exam mark <=75

     0<=coursework <=25

               SOEN345-W11-L13: A2, review   42
Invalid partitions
   The most obvious partitions are
       Exam mark > 75
       Exam mark < 0
       Coursework mark > 25
       Coursework mark <0

                SOEN345-W11-L13: A2, review   43
Exam mark and c/w mark

        SOEN345-W11-L13: A2, review   44
Less obvious invalid input EP
   invalid INPUT EP should include

              SOEN345-W11-L13: A2, review   45
Partitions for the OUTPUTS
   EP for OUTPUTS should include

              SOEN345-W11-L13: A2, review   46
The EP and boundaries
   The EP and boundaries for total mark

              SOEN345-W11-L13: A2, review   47
Unspecified Outputs
   Three unspecfied Outputs can be
    identified (very subjective)
       Output = “E”
       Output = “A+”
       Output = “null”

                 SOEN345-W11-L13: A2, review   48
Total PE

           SOEN345-W11-L13: A2, review   49
Test Cases corresponding to
PE exam mark (INPUT)

         SOEN345-W11-L13: A2, review   50
Test Case 4-6 (coursework)

        SOEN345-W11-L13: A2, review   51
test case for Invalid inputs

         SOEN345-W11-L13: A2, review   52
Test cases for outputs:1

         SOEN345-W11-L13: A2, review   53
Test cases for outputs:2

         SOEN345-W11-L13: A2, review   54
Test cases for invalid

         SOEN345-W11-L13: A2, review   55
Minimal Test cases:1

         SOEN345-W11-L13: A2, review   56
Minimal Test cases:2

         SOEN345-W11-L13: A2, review   57
For Boundary Value analysis

         SOEN345-W11-L13: A2, review   58
   For each boundary three values are
       One on the boundary
       One either side of it

                 SOEN345-W11-L13: A2, review   59
Test cases corresponding to
BVA (Exam mark)

         SOEN345-W11-L13: A2, review   60
Test cases corresponding to
BVA (Course work)

         SOEN345-W11-L13: A2, review   61

       SOEN345-W11-L13: A2, review   62
Test case derived from

         SOEN345-W11-L13: A2, review   63
Test case derived from

         SOEN345-W11-L13: A2, review   64
   Feb 16: midterm
   Tutorial in support of A2

   Feb 14: Happy Valentine’s Day!

               SOEN345-W11-L13: A2, review   65