# PowerPoint Presentation

Document Sample

```					ECE 453 – CS 447 – SE 465
Software Testing &
Quality Assurance

Lecture 23

Instructor
Paulo Alencar
1
Overview
Software Quality Metrics
Black Box Metrics
White Box Metrics
Development Estimates
Maintenance Estimates

2
White Box Metrics
• White Box Metrics
– Linguistic:
• LOC
• Halstead’s Software Science
– Structural:
• McCabe’s Cyclomatic Complexity
• Information Flow Metric
– Hybrid:
• Syntactic Interconnection
3
Types of White Box Metrics
– Linguistic Metrics:
• measuring the properties of program/specification text without
interpretation or ordering of the components.

– Structural Metrics:
• based on structural relations between objects in program;
• usually based on properties of control/data flowgraphs [e.g.
number of nodes, links, nesting depth], fan-ins and fan-outs of
procedures, etc.

– Hybrid Metrics:
• based on combination (or on a function) of linguistic and
structural properties of a program.

4
Linguistic Metrics
Lines of code/statements (LOC)
• perhaps the simplest:
– count the number of lines of code (LOC) and use as a
measure of program complexity.
– Simple to use:
• if errors occur at 2% per line, a 5000 line program should have
• If it required 30 tests to find an error, then could infer the
expected number of tests needed per line of code.

5
LOC
• Examples of use include:

–   productivity    KLOC/person-month
–   quality         faults/KLOC
–   cost            \$\$/KLOC
–   documentation   doc_pages/KLOC

6
LOC
• Various studies indicate:
– error rates ranging from 0.04% to 7% when measured
against statement counts;
– LOC is as good as other metrics for small programs
– LOC is optimistic for bigger programs.
– LOC appears to be rather linear for small programs
(<100 lines),
– but increases non-linearity with program size.
– Correlates well with maintenance costs
– Usually better than simple guesses or nothing at all.

7
LOC
• One study: a rough estimate of the average LOC
to build one function point (FP):

Language                LOC/FP
(average)
Assembly                 300
COBOL                    100
FORTRAN                  100
Pascal                    90
00 Language               30
4GL                       20
Code Generator            15             8
Halstead's Software Science Metrics
• Halstead [1977] based his "software
science" on:
–   common sense,
–   information theory and
–   psychology.
–   His 'theory' is still a matter of much
controversy.
• A basic overview:
9
Halstead's Software Science Metrics
• Based on two easily obtained program parameters:
– n1  the number of distinct operators in program
– n2  the number of distinct operands in program
• Paired operators such as: "begin...end",
"repeat...until" are usually treated as a single
operator. Other examples: =, do, if, goto, etc.
• Operands: primitive variables, array variables
• the program length is defined by:
H  n1 log 2 n1  n2 log 2 n2
• define the following:
– N1 = total count of all operators in program.
– N2 = total count of all operands in program.
10
Halstead's Software Science Metrics
• then the actual Halstead length is given as:

N  N1  N 2

– [usually accepted as more appropriate]
– which is basically a static count of the number of tokens
in program.

• the vocabulary of a program is defined as the sum
of the number distinct operands and operators :
n  vocabulary  n1  n2
11
Halstead's Software Science Metrics
• Various other measures can also be
obtained, for example:
– program volume:             V  N log 2 (n1  n2 )
• varies with the language and represents the
information (in bits) to specify the program.
– potential volume:
• V* is the volume of the most succinct program in
which the algorithm can be coded.
• Consider op (d1 , d 2 ,..., d n ) where
n*1  2 and n2  n thus
V *  (2  n2 ) log2 (2  n2 )                      12
Halstead's Software Science Metrics
V*
• program level:       L
V
– which is a measure of the abstraction of the formulation
of the algorithm.    *
ˆ n1 n2
– also given as : L 
n1 N 2

• program effort:       E
V
L

– the number of mental discriminations needed to
implement the program (the implementation effort)
– correlates with the effort needed for maintenance for
small programs.
13
Use of Halstead’s Metrics
– A motivated programmer who is fluent in a
language, the time to generate the preconceived
program is
TˆE V
S SL

where S is a constant (Stroud).
• Letting n1* = 2 and substituting for L and V we get:
ˆ n N H log2 n
T 1 2
2Sn2
14
Use of Halstead’s Metrics
Known weaknesses:
– Call depth not taken into account
• a program with a sequence 10 successive calls more
complex than one with 10 nested calls

– An if-then-else sequence given same weight as
a loop structure.
– Added complexity issues of nesting if-then-else
or loops not taken into account, etc.
15
Structural Metrics

– McCabe’s Cyclomatic Complexity
• Control Flow Graphs

– Information Flow Metric

16
McCabe’s Cyclomatic Complexity

• Determines the logical complexity of a graph
– typically the graph is a flow graph of a function or procedure
– can also be a graphical representation of an FSM.
• Define:
– independent path: any path that introduces at least one new set of
statements or a new condition; must move along at least one edge
that has not been traversed before.
– Basis Set: a set of independent paths (covers all conditions and
statements).

17
McCabe’s Cyclomatic Complexity
Example: set of independent paths which comprise a basis set for
the following flow graph:

Path1: a, c, f            R1                  a
Path2: a, d, c, f              R2                 R5
Path3: a, b, e, f                   b         c             d
Path4: a, b, e, b, e, f        R3       R4
Path5: a, b, e, a, c, f          e            f
18
McCabe’s Cyclomatic Complexity
• Alternate methods to calculate V(G):
– V(G) = #of_predicate_nodes + 1
– V(G) = #of_edges - #of_nodes + 2
• Note: in previous graph node ‘a’ has an out degree of 3,
thus it counts as 2 predicate nodes.
• The previous graph could also have been drawn as:
a1

a2

b         c        d

e1        f

19
e2
McCabe’s Cyclomatic Complexity
• A predicate node is a node in the graph with 2 or more out
going arcs.

• In the general case, for a collection of C control graphs
with k connected components, the complexity is equal to
the summation of their complexities. That is
k
V (G )  V (Gi )
i 1

20
Information Flow Metric
• By Henry and Kafura:
– attempts to measure the complexity of the code by measuring the
flow of information from one procedure to another in terms of fan-
ins and fan-outs.

• fan-in: number of local flows into a procedure plus the
number of global structures read by the procedure.
• fan-out: number of local flow from a procedure plus the
number of global structures updated by the procedure.

21
Information Flow Metric
• Flows represent:
– the information flow into a procedure via the argument
lists and
– flows from the procedure due to return values of
function calls.
• Thus, the complexity of the procedure, p, is given
by

Cp = (fan-in  fan-out)2
22
Hybrid Metrics
• An example is Woodfield’s Syntactic
Interconnection Model
– attemps to relate programming effort to time.
• A connection relationship between two modules A
and B:
– is a partial ordering between the modules
– i.e., to understand the function of module A
• one must first understand the function of module B
– denoted as: A  B
23
Hybrid Metrics
• Three module connections are defined:
1. control: an invocation of one module by the
other.
2. data: one module makes use of a variable
modified by another module.
3. implicit: a set of assumptions used in one
module are also used in another module.
•   If the assumption changes, then all modules using
that assumption must be changed.
• The number of times a module must be
reviewed is defined as the fan-in.
24
Hybrid Metrics
• The general form for the measure is given
by:
fan _ in1
Cb  C1b           RC k 1
k 2

– where Cb is the complexity of module B’s code,
– C1b is internal complexity of module B’s code,
– fan_in is the sum of the control and data
connections to module B
– RC is a review constant.
25
Hybrid Metrics
• The internal complexity can be any code metric

– e.g. LOC, Halstead’s, McCabe’s, etc.
– Halstead’s Program Effort Metric was used originally
– a review constant of 2/3 was suggested by Halstead.

• The previous Information Flow Metric can also be
used as a hybrid metric by taking into account the
internal complexity of the module in question as:
C p  C1 p  ( fan _ in  fan _ out) 2
26
Maintenance Predictions:
Case Study 1
• The intermetric results can be observed in Henry and
Wake’s paper in table 1, page 137 where the statistical
correlations are given between :
– the Length, N, V, E, McCabe, Woodfield, Information-L,
Information-E, and Information metrics,

• There is a high degree of correlation among the code
metrics:
– they all measure the same aspect of the code
– but low correlations among the code metrics, structure metrics and
hybrid metrics
• these measure different aspects of the code.
– These agree with other studies made.
27
Maintenance Predictions
• The goal of their study:
– develop a model using metric values as parameters to
predict the number of lines of code (NLC) changed and
the total number of changes (NCC) during the
maintenance phase of the software life-cycle.

– The NLC and NCC are the dependent variables for the
statistical model while the metric values are the
independent variables.

– They used various statistical analysis techniques (e.g.
mean squared error, different curve fitting techniques,
etc.) to analyze the experimental data.
28
Maintenance Predictions
• the tables below summarize the overall top candidate
models obtained in their research (see their paper for the
details):
Best overall NLC models

NLC = 0.42997221 + 0.000050156*E - 0.000000199210*INF-E
NLC = 0.45087158 + 0.000049895*E - 0.000173851*INF-L
NLC = 0.60631548 + 0.000050843*E - 0.000029819*WOOD -
0.000000177341*INF-E
NLC = 0.33675906 + 0.000049889*E
NLC = 1.51830192 + 0.000054724*E - 0.10084685*V(G) -
0.000000161798*INF-E
NLC = 1.45518829 + 0.00005456*E - 0.10199539*V(G)

29
Maintenance Predictions
• Best overall NCC models

NCC = 0.34294979 + 0.000011418*E
NCC = 0.36846091 + 0.000011488*E - 0.00000005238*INF-E
NCC = 0.38710077 + 0.000011583*E - 0.0000068966*WOOD
NCC = 0.25250119 + 0.003972857*N - 0.000598677*V +
0.000014538*E
NCC = 0.32020501 + 0.01369264*L - 0.000481846*V +
0.000012304*E

30
Maintenance Predictions
• Note:
– These models are best fits to the software components
used in the study.
– May not necessarily apply universally to all software
systems.
– Intuitively, one would suspect that the models would be
sensitive to:
• the application domain (i.e. real-time software vs. Data
processing, etc.),
• overall system complexity,
• maturity of the product area/team,
• programming language used,
• CASE tools used etc.
31
Maintenance Predictions:
Case Study 1
• Some good correlations were found:
– one procedure had a predicted NLC = 3.86
• the actual number of lines changed was 3;
– another had an NLC of 0.07
• the number of lines changed was zero.

• The experimental results look encouraging
but need further research.
32
Maintenance Predictions:
Case Study 1
• The basis for generating the models for NLC and
NCC:

– Not to get exact predicted values.
– But, rather these values could be used as an ordering
criteria to rank the components in order of likelihood of
maintenance.
– If performed before system release,
• future maintenance could be prevented (reduced) by changing
or redesigning the higher ranking components.
33
Maintenance Predictions
• To properly use these models (or others):
– the organization would be required to collect a significant amount
of error or maintenance data first
• before the models results can be properly interpreted.

• Models such as these could be used during the coding
stage:
– to identify high maintenance prone components,
– redesign/code them to improve the expected results.

• Useful during the test phase:
– to identify those components which appear to require more
intensive testing,
– help to estimate the test effort.                                34
Case Study 2
• Another interesting case study was performed by
Basili et al.

• Their major results were:
– the development of a predictive model for the software
maintenance release process,
– measurement based lessons learned about the process,
– lessons learned from the establishment of a
maintenance improvement program.
35
Case Study 2
• Maintenance types considered:
– error correction,
– enhancement and adaptation
• within the Flight Dynamics Division (FDD) of the NASA Goddard
Space Flight Center.
• the following table illustrates the average distribution of
effort across maintenance types:
Maintenance
Activity              Effort
Enhancement            61%
Correction                14%
Other                     20%                                    36
Case Study 2
• The enhancement activities typically involved more SLOC
(Source Lines of Code) than error corrections
– verifies the intuitive notion that error corrections usually result in
minor local changes.

• What about the distribution of effort within each
maintenance activity?
– the following efforts can be examined:
• analysis ( examine different implementations and their associated
costs),
• isolation (time spent to understand the failure or requested
enhancement),
• design (time spent in redesigning the system),
• code/unit test and,
• inspection, certification and consulting.
37
Case Study 2
• The following table illustrates the measured distributed
effort for error correction and enhancement maintenance:
Error                  Effort           Enhancement
Correction
6%        Analysis                         1%

26%       Isolation                       20%

26%       Design                          27%

38%       Code/Unit Test                  39%

4%        Inspection, Certification,      13%
Consulting
38
Case Study 2
• Attempt to distinguish between the two types of SCRs
(Software Change Request):
– user and tester generated SCRs.

• For example, during the implementation of any release,
errors may be introduced by the maintenance work itself.
– This may be caught by the tests which generate SCRs which
become part of the same release delivery.
– The SCR count and the SLOC differences between user and tester
change requests for 25 releases were given as:

SCRs         Origin       SLOC
35%          Tester        3%
65%           User        97%                     39
Case Study 2
• By estimating the size of a release, an effort estimate (for
enhancement releases) can be obtained as the following
equation:
Effort in hours = (0.36  SLOC) + 1040
• This indicates a certain amount of overhead from the
regression testing and comprehension activities which tend
to be independent of the size of the change.
• It appears that if productivity improvement is the main
goal,
– then better to avoid scheduling small error correction releases but,
– rather to package them with a release of larger enhancements
– better still if the enhancements require change to the same units as
the corrections.

40

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 5 posted: 6/16/2012 language: English pages: 40
How are you planning on using Docstoc?