Software Fault Interactions and
Implications for Software Testing
Sai Praneeth Purighella.
For example, 20 inputs, each having 10 possible values, yield a total of 1020
combinations of settings.
Only a few hundred test cases can be built and executed under most budgets
Empirical research into quality and reliability suggests that software failures in
a variety of domains were caused by combinations of relatively few
If ‘n’ be the number of parameters involved, then testing all n-tuples of
parameters is effectively equivalent to exhaustive testing (pseudo exhaustive).
The number of conditions required to trigger a failure is referred as the failure-
triggering fault interaction (FTFI) number.
1. Nair et al.’s case study of combinatorial testing for a small
subsystem of a screen-based administrative database.
2. Wallace and Kuhn’s review of medical device recall data
gathered by FDA.
3. Smith et al.’s investigation on the pairwise testing of RAX
software on NASA’s Deep Space 1 mission.
Analysis of 329 error reports from development and integration
of a large distributed system at NASA Goddard Space Flight
Results of analysis are shown in last column of Table 1 and in
Distribution of failure-triggering conditions appears to follow a
power law. (See figure 1, last four columns of Table 1).
Implications of Testing
Consider the example with 20 inputs, each with
10 possible values.
Exhaustive testing requires 1020 test cases.
But empirical studies show that most failures are
triggered by a single erroneous parameter.
However, all could be triggered by fewer than 6
Consider the problem of n-tuples of k-
parameters, each of which has ‘v’ possible
The number of n-tuples drawn from k-
Since each parameter has ‘v’ values, total number of test cases
would be C(k,n).vn.
Let us see at the example previously considered.
For a given number N, of test cases, and a specifies level n-tuple,
how many values can or must be covered can becalculated using
vn<=N, so n log v <= log N.
If all errors in a particular class of software are triggered by finite
combinations of ‘n’ values or less, then testing all combinations of n or fewer
values would provide a form of “pseudoexhaustive testing”.
Appropriate levels of ‘n’ appear to be 4<=n<=6 when considering
pseudoexhaustive testing, according to dependability requirements.
This approach might not be good effective real-time or other software that
depends on testing event sequences, but may be applicable to subsystems
within real-time software.
Many more empirical studies of other classes of software are needed to
evaluate the applicability of combinatorial testing for other classes of systems.
Thank You One and All!