Software Fault Interactions and Implications for Software Testing

Document Sample
Software Fault Interactions and Implications for Software Testing Powered By Docstoc
					  Software Fault Interactions and
 Implications for Software Testing


Presented By:
Sai Praneeth Purighella.
                              Introduction
   For example, 20 inputs, each having 10 possible values, yield a total of 1020
    combinations of settings.

   Only a few hundred test cases can be built and executed under most budgets

   Empirical research into quality and reliability suggests that software failures in
    a variety of domains were caused by combinations of relatively few
    parameters.

   If ‘n’ be the number of parameters involved, then testing all n-tuples of
    parameters is effectively equivalent to exhaustive testing (pseudo exhaustive).

   The number of conditions required to trigger a failure is referred as the failure-
    triggering fault interaction (FTFI) number.
                     Related Work

1.   Nair et al.’s case study of combinatorial testing for a small
     subsystem of a screen-based administrative database.

2.   Wallace and Kuhn’s review of medical device recall data
     gathered by FDA.

3.   Smith et al.’s investigation on the pairwise testing of RAX
     software on NASA’s Deep Space 1 mission.
                Empirical Analysis
   Analysis of 329 error reports from development and integration
    of a large distributed system at NASA Goddard Space Flight
    Center.

   Results of analysis are shown in last column of Table 1 and in
    Table 2.

   Distribution of failure-triggering conditions appears to follow a
    power law. (See figure 1, last four columns of Table 1).
Figure 1
         Implications of Testing
   Consider the example with 20 inputs, each with
    10 possible values.

   Exhaustive testing requires 1020 test cases.

   But empirical studies show that most failures are
    triggered by a single erroneous parameter.
    However, all could be triggered by fewer than 6
    parameters.
        Background Calculations
   Consider the problem of n-tuples of k-
    parameters, each of which has ‘v’ possible
    values.

   The number of n-tuples drawn from k-
    parameters is
              Implications (Continued)
   Since each parameter has ‘v’ values, total number of test cases
    would be C(k,n).vn.


   Let us see at the example previously considered.


   For a given number N, of test cases, and a specifies level n-tuple,
    how many values can or must be covered can becalculated using
    the expression:
                 vn<=N, so n log v <= log N.
                           Conclusions
   If all errors in a particular class of software are triggered by finite
    combinations of ‘n’ values or less, then testing all combinations of n or fewer
    values would provide a form of “pseudoexhaustive testing”.

   Appropriate levels of ‘n’ appear to be 4<=n<=6 when considering
    pseudoexhaustive testing, according to dependability requirements.

   This approach might not be good effective real-time or other software that
    depends on testing event sequences, but may be applicable to subsystems
    within real-time software.

   Many more empirical studies of other classes of software are needed to
    evaluate the applicability of combinatorial testing for other classes of systems.
Thank You One and All!

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:4/22/2013
language:Unknown
pages:12