SREPT Software Reliability Estimation and Prediction Tool

Document Sample
SREPT  Software Reliability Estimation and Prediction Tool Powered By Docstoc
					     SREPT: Software Reliability Estimation and Prediction Tool

                                            User Manual
SREPT greets the user with a welco me screen consisting of the CACC (Center for Advanced Co mputing
and Co mmunication) logo. The user can then choose a specific engine to use, depending on the data
available and type of analysis required.

The currently imp lemented modules in SREPT are :
1. Product/process metrics – for early predict ion of software fault content
2. ENHPP (Enhanced non-homogeneous Poisson process) - for analyses using inter failu re times and
    coverage information fro m testing
3. Debugging policies – to analyze the effect of debugging policies

1. Product/Process metrics
SREPT currently offers the “Fault density” approach to obtain early estimates of software quality.
The “Fault density” module is chosen by selecting the appropriate engine in the menu as in Figure 1.

                                                 Figure 1

The interface for the fault density module lets the user enter a module name (for examp le, the name o f a C
source file), and the number of lines in that module. The user needs to then enter “Experience fro m past
projects”. This consists of the number of faults observed in the past project and the number of lines of code
in that project. The table containing the module names and the number of lines can also be loaded fro m a
file, by clic king the button labeled “Click to read data into table fro m file”. To calculate the number of
expected faults in the current project, the user needs to click the button labeled “Estimate the number of
faults”. An examp le is shown in Figure 2.

                                                  Figure 2

If the table containing the module names and the number of lines in each module is to be loaded fro m a file,
the file should have the follo wing fo rmat for each line:

Module_name blank spaces and/or tabs Number_of_lines

For examp le,

analyze.c         946
mu ltpath.c       387

If there is a third field in a line of the file, that field is displayed under the column labeled “Nu mber of
faults”. This could be the case for example, if the table was previously saved after the estimation of the
number of faults was done.

The ENHPP module is chosen fro m the menu as shown in Figure 3.

The ENHPP module allo ws the follo wing types of input:
 Inter failure times alone
 Inter failure times and coverage data from testing
 Estimate of the total nu mber of faults in the software and coverage d ata fro m testing

The type of input data can be chos en using the ENHPP module interface , as shown in Figure 4.

                                                 Figure 3

                                                    Figure 4

When the type of input data chosen is “Use inter failure times data onl y”, the user should specify the file
containing the inter failure t imes. This file should contain one inter failure time per line. Thus the format is:

Inter failure time 1
Inter failure time 2

When inter failure t imes data alone are used as input, the user can either specify a coverage function to
use, or let SREPT help find the best of the four coverage models built into SREPT currently. These are :
 Exponential (GO) model
 Weibull
 S-shaped
 Log-logistic
coverage models.

The user can override the default init ial values in the numerical solution of the models when a
configuration window (shown in Figure 5) pops up. SREPT reports the Kolmogorov distances for each
model based on the u-plot (Bias test) and also the sum of square errors (Goodness of fit test). The results of

                                                  Figure 5

choosing the “Use best model” under “Coverage models to use” in the interface and the u-plot generated
are shown in Figure 6 and Figure 7 respectively.

Figure 6

Figure 7
Based on the sum of square errors, the best model for this data set is the log -logistic coverage function. The
user can now choose the log-logistic coverage function, as coverage model to use and let SREPT estimate
the model parameters by clicking on the “Estimate” button. This causes a configuration window to pop up
allo wing the user to override the default in itial values used in the numerical solution. The result s of the
solution are shown in Figure 8.

                                                  Figure 8

Figure 8 also shows that SREPT p rints out the model parameters estimated in a “textarea” labeled
“Messages”. The Mean value function is also displayed. The coverage function fitted is shown in a separate
“textfield”. By clicking on the “Obtain release time” button, the user can now use SREPT to estimate the
release time for the software. When the button “Obtain release time” is clicked, a window with the release
criteria pops up. Figure 9 shows the interfaces when the “Faults remain ing” criteria is chosen.

                                     Figure 9

Plots of “Mean value function”, “Failure intensity” “Faults remain ing”, “Conditional reliabiu lity”,
“Estimated coverage” can be obtained by choosing “Plots” from the main menu. Figure 10 sho ws the
interface fo r the plots window with the mean value function plotted. The irregular line is the actual failure
data plotted, whereas the smooth curve is the fitted curve corresponding to the log -logistic coverage model
in this case.

                                                  Figure 10

In Figure 10, and everywhere the same p lot package is used, the number of points to plot, as well as the
minimu m and the maximu m “t” values can be altered by the user. By clicking on the button labeled “Plot
it!”, the user can redraw the plot with the specified values. By dragging the mouse over a section of the
plot, the plot can be zoo med-in. Clicking on the “Plot it!” button resets the plot to how it was before any
zoo m-in.

The ENHPP module also allows the input data to consist of the inter failure times and coverage information
fro m testing. This coverage informat ion fro m actual measurements will yield better estimates than before.
The coverage information may be either in the form o f a function (a distribution function, imp lying that it is
a function of time “t”, approaching 1.0 as “t” approaches infinity, and 0 at “t” = 0), o r in the form o f
coverage values.

If coverage in formation is supplied as “coverage values”, the values should be specified in a file. Each line
of the file should contain the inter failure t ime (wh ich is also specified as an input separately) and the
coverage value at that time. So, the format of the file containing coverage informat ion is,

Interfailure time 1 blank spaces and/or tabs coverage at this time
Interfailure time 2 blank spaces and/or tabs coverage at this time

The interface to choose the type of coverage informat ion to be used is shown in Figure 11.

                                                 Figure 11

When the coverage information is specified as coverage values fro m a file, SR EPT does not currently fit a
coverage function, and hence does not at this time allow estimates of release times. The plots also appear
different as can be seen from Figure 12.

                                                       Figure 12

The plot package shown in Figure 12 (it is also used in some other modules in SREPT), allows the user to
zoo m in by dragging the mouse over a region wh ile holding down the “Shift” key. Dragging the mouse
while holding down the “Ct rl” key translates the plot. Clicking on the “Reset” button restores the plot to its
default appearance.

When the type of input available to the ENHPP module is an “estimate of the total number of faults” and
the coverage informat ion fro m testing, SREPT again allows the coverage information to be a coverage
function (of time variab le “t”), or coverage values in a file. The format of this file is,

time 1 blank spaces and/or tabs coverage at this time
time 2 blank spaces and/or tabs coverage at this time

The user also needs to supply an estimate of the total number of faults in the software. Figure 13 shows the
interface fo r this input type.

                                                    Figure 13

3. Debugging policies
The “Debugging policies” module can be used to study the effect of non -zero fault removal times as well
as different debugging policies on the observed failure intensity and the number of faults remaining. An
example of this module is shown in Figure 14.

In the interface shown in this figure, it can be seen that the user has to specify a “Truncation level”. This is
the level at wh ich the infinite state space of the underlying NHPP (non-homogeneous Poisson process) is
truncated. This is necessary for numerical solution. The truncation level should be chosen by trial and error,
for examp le by try ing out a slightly higher truncation level and ver ify ing that the results don’t differ by too
much. An initial estimate of the truncation level could be the expected total number of fau lts in the
software. For examp le, this estimate could be got if inter failure t imes for this software was availab le and
was used to fit a coverage function using the ENHPP module. In that case, a truncation level that is greater
than the expected total number of faults can be used.

Next, the user needs to supply a “failure intensity”. This again can come fro m the ENHPP module. SREPT
reports the coverage function fitted (c(t)). If a is the expected total number of faults (given infin ite testing
time), wh ich is one of the parameters estimated fo r all the coverage functions fitted, then the failu re
intensity is

where c’(t) is the derivative of c(t) with respect to time t.
The interface ensures that the failure intensity is of the form a*c’(t), by having separate entry fields for a
and c’(t).

                                                 Figure 14

The process of fault removal can be optionally specified as “Delayed”, in which case the “Fault removal
rate” will consist of two phases, each with possibly a differen t rate. The two phases represent the time
before debugging actually starts and the time it takes to debug, once debugging starts. If the “Delayed”
checkbox is not checked, the user needs to enter just one “Fault removal rate”.

The process of fault removal can also be “Deferred”. This option can be used to specify that debugging
does not start till the specified number of faults are logged.

Clicking the button labeled “Solve” brings up a configuration window that lets the user specify the time up
to which the solutions are required and the time steps to use in the solution. Choosing very small time steps
improves accuracy but slows down the solution, whereas choosing large time steps speeds up solution at
the cost of accuracy.

The plots of the faults remaining and the perceived and actual failure intensities can be got by choosing
“Plots” fro m the main menu. An example is shown in Figure 15.

                                               Figure 15

4. Architecture of SREPT
The source files in the current version of SREPT and their function is indicated in Table 1.

                      File name                                                  Functi on                                            GUI that brings up a configuration window to
                                                          specify initial values for the ENHPP coverage
                                                          function solvers                                      GUI spec. for the configuration window for the
                                                          Debugging policies module                                       Solves NHCTM C by invoking SHARPE                                                Starting point for the ENHPP engine                                         Routines for the coverage functions in the ENHPP
                                                          model                                    GUI spec for the dialog that pops when a ENHPP
                                                          coverage function is being fitted. The dialog
                                                          prompts for the estimat ion to be stopped if desired.                                         Interfaces for the "Fault density" module                                           Generates a "file v iewer" window that displays the
                                                          contents of the file whose name is passed as an
                                                          argument to the constructor.                                           GUI that brings up a configuration window to
                                                          specify initial values for the Exponential (GO)
                                                          coverage function solver.                                                 Specify help file names here                                           Construct the help window                                           Dialog to display messages. Mainly used to display
                                                          error messages.                                       Routines to determine the best coverage model                                            GUI that brings up a configuration window to
                                                          specify initial values for the Log-logistic coverage
                                                          function solver.                                       Used in association with to
                                                          generate the message text.                                              Generates plots. Derived fro m the “graph” package.                                            GUI spec. for the "Plots" frame when data points
                                                          rather than functions are plotted.                                                GUI spec. for the "Plots" frame when functions
                                                          rather than data points are plotted.                                          Routines for co mputing the release times
                                                          based on the ENHPP coverage functions.                                    Starting point for SREPT.                                      Contains the interfaces for the main SREPT window                                        GUI spec. for the "Welco me screen"                                        GUI that brings up a configuration window to
                                                          specify initial values for the S-shaped coverage
                                                          function solver.                                               GUI spec. for p lotting the u-plots                                            GUI that brings up a configuration window to
                                                          specify initial values for the Weibull coverage
                                                          function solver.
                                                  Table 1

5. Acknowledgements

The following packages were obtained fro m the web and used in the SREPT software.
1. graph – a graph plotting package authored by Leigh Brookshaw available under GPL fro m
2. lundin – a package authored by Patrik Lundin that performs symbolic differentiation. Available fro m
    The package is free for NON COMM ERCIA L use, a license for co mmercial use is $30 and can be
    payed online through a secure server

In addition a routine to obtain roots of equations by the bisection method was obtained from
http://www.ny ley/cs3113/ ml


pptfiles pptfiles