Get documents about "Software Metrics Management Rajeev Vashista It was when I was searching on Internet about usefulness of metrics management I felt there was
Description
Software Design Metrics Template document sample
Document Sample
Software Metrics Management
Rajeev Vashista
It was when I was searching on Internet about usefulness of metrics
management; I felt there was urgent need for some white paper which will
explicitly deal with this topic and its usefulness to software development process.
Before writing this whitepaper I went through couple of books and innumerable
websites which cover this topic, however I feel that they lack the basic concept of
metrics management mainly in context to software. I am trying here to enlist what
are major metrics which are extremely useful in software development and some
metrics which can be somewhat useful.
What is metrics management?
It is made up of two almost correlated words ‘METRICS’ and ‘Management’.
Very fundamental definition of Metrics is ‘A Standard for measurement’.
Similarly for management ‘it is a act of managing or judicious use of means to
accomplish an end’.
Now we can easily conclude that we are dealing with managing different
measures to achieve our goals.
Just to elaborate on this, let’s take an example of obstacle free road of X miles in
length and car with speed of Y miles per hour. If we need to cover X miles in our
car it will reach destination in X/Y hours, However it will remain our projection of
time unless we achieve our goal in X/Y hours, for this we have make sure that
a) Car doesn’t spend time in gaining speed on Y miles/hour
b) Our assumption of obstacle free road hold true throughout our journey.
c) It reaches halt in no time.
Hence whole challenge will to keep speed monitored or managing it otherwise
we might reach slight late depending on the average speed.
How relevant is metrics management to Software Development?
This question can definitely gives nightmares to software managers who are
forced to use metrics to achieve software development goals. In fact software
development cycle is so fragile that it simply gives no scope for anything which is
measurable or every measurable unit in software has to be padded with so many
assumptions that whole purpose of measure seems defeated in the end.
Let us have a look at conventional software development waterfall model. It
comprises of below stages:-
a) Business Requirement gathering stage
b) Designing Stage
c) Implementation/Development Stage
d) Testing/Debugging Stage
e) Closing Stage
f) Control/Maintenance Stage
Page 1 of 7
Main challenges what seems to percolate from above stages are:-
• This cycle seems highly cohesive that it seems that most of the stages do
not have explicit end or explicit beginning.
• What are different things we can measure at different steps and how are
they correlated?
• How managing developers/designers/testers who are so un-predictive to
measure we can be sure we are progressing and are in right directions?
However one thing is for sure if we can find out what we can measure in software
development cycle we can take better control on
a) Quality of software
b) Committed timelines
c) Defects and faults
d) Customer satisfaction
The point I am trying to make is if we know what to measure and it’s HOW and
WHY, we can definitely have better control on major aspects of software
development stages.
What can be measured in Software Development?
Now this is the question if can be answered properly, to an extent we can justify
the usability of metrics management in relevance to software development.
Let us revisit software waterfall and check what its main ingredients are.
a) Business Requirement Gathering Stage:- This is where we identify our
main objective for the development. It is the most important stage where
all the stakeholders, end users, project managers envisage common goal.
(However in most of the cases the objective is not so simple that it can be
translated into one goal. In fact in real life scenarios it encompasses wide
numbers of different goals which has to be consolidated and unified into
one goal. ) Also remember this is the stage where we can get as much as
possible information about the Goal of the project, and don’t forget to
document or record them. Now based on the requirement document we
can easily find out (Taken from "Software Metrics and Reliability"- SATC NASA)
a. Textual Lines - Physical lines of text as a measure of size.
b. Imperatives - Words and phases that command that something
must be done or provided. The number of imperatives is used as a
base requirements count.
c. Continuances -Phrases that follow an imperative and introduce the
specification of requirements at a lower level, for a supplemental
requirement count.
d. Directives – References provided to figures, tables, or notes.
e. Weak Phrases - Clauses that are apt to cause uncertainty and
leave room for multiple interpretations measure of ambiguity.
Page 2 of 7
f. Incomplete – Statements within the document that have TBD (To
be Determined) or postponed for later stages.
g. Options - Words that seem to give the developer latitude in
satisfying the specifications but can be ambiguous.
The main challenge here will be to minimize the value of points e,f and
g. This metrics can be controlled at this stage only, so try to convert
most of the requirements into IMPERATIVE or don’t start them,
because they may change their definition moving forward in the
development cycle. We can call this ‘Requirement consistency metrics’
Another metrics which kind of take birth in this stage is Traceability
metrics. It may be in the shape of excel file and it may look like below
table:-
Requirement Design Implementation
Testing Stage Control Stage
Stage Stage Stage
Invoice bill HLD ver 1 Code file Name User Acceptance
number to be section 3.2.1 Test document / Test Testing
generated unique LLD ver 2 case number
every time. section 4.2.4
Figure 1 : Traceability metrics
One can easily understand the main aim of the traceability metrics
which makes sure all the requirement that were jotted down at the
beginning of the project have been converted into desired output or
not.
As third metrics at this stage we have ‘Requirement Testing Metrics’.
The main objective of this metrics is to capture
wrong/ambiguous/unachievable requirements in this stage and correct
them or flag them for more elaboration. Remember traceability metrics
must have only those requirements which are ‘Imperative’ or
‘Continuances’, all other weak one should be brainstormed and in case
of inconsistencies should be omitted.
Page 3 of 7
Logical Technical Test
Requirement Elicitation Analysis Complexity Complexity Cases
To validate
local
business Test
total balance document
against /Test
corporate case
numbers Yes Yes Moderate TBD number
Figure 2: Requirement Testing Metrics Template
Also we have Error Metrics whose genesis takes place at this stage only; we
keep on tracking the error identified at each stage and will link back to its main
stage where it should have been identified in ideal case. For example in ideal
situation we should have all the requirement errors to be fixed in requirement
stage only, however in practical situation some fail to get noticed and they are
trapped at either design or implementation stage.
Error Original
S.No. Error Description Stage Stage status
Design Requirement
Figure 3: Error Metrics
Error Control metrics purpose is just to keep track where the number of errors
was trapped and how much are fixed ideally it should have negative exponential
graph however we can expect random behavior too.
Error Error Fixed Error Numbers Error Fixed
Stage Numbers (I) (I) (II) (II)
Requirement
Design
Implementation
Testing
Control
Figure 3: Error Control Metrics (I= First Iteration, II=Second Iteration) (In fact number
iteration can be flexible it is up to the number of bugs discovered in first Iteration that
need for second or third iteration arises)
b) Designing Stage: Main aim of this stage is to convert all requirements into
technical design. First at the macro level (High Level Design) and then at
the granular level (Low Level Design). Hence this stage will continue using
almost all of for the metrics we identified in Requirement Stage. This
stage can have complexity metrics (based on Cyclomatic (out of scope of current
paper) complexity theorem) and Effort Metrics or measurement (based on
Page 4 of 7
parametric or analogy approach, Delphi method or personal gut feel
method).
For effort measurement the main idea can be features covered in
requirement stage. What I like personally is divide all requirements into
mutually exclusive features and give each features number of points.
Effort Metrics
First II nd Revised III rd
Projected Projected Revised
Complexity Efforts (in Efforts (in Projected Actual
S.No. Feature name Points hours) hours) Efforts Efforts
1 Requirement Doc 1.a.i 2 12 10 7 8
2 Requirement Doc 1.b 5 30 34 35 35
3 Requirement Doc 3.b 8 50 60 47 50
Figure 4:- effort estimation sample metrics
The point to be noted here is that
i) There is no direct correlation between complexity point and
efforts in hours.
ii) Complexity point is generally identifies by end developer
iii) Below 8 hours generally means less than a day, so revised
timelines makes generally no sense (above example is just for
information purpose), hence any tangible outcome must be
either associated with more than 16 hours or for below 8 hours
we will only fill first Projected Efforts and Actual Efforts.
iv) Number of revision should not be too frequent however it should
be revised keeping in mind Parkinson’s Theorem (Work always
fill Allocated time).
c) Implementation/Development Stage: - For this stage all metrics will flow
from previous stage. However for this stage design has already been
prepared now the main challenge will be to convert the design into the
code and keep measuring that too in terms of effort. All the design can be
converted into estimated Assembly Line of Code (ALOC), there are tools
available for that or some experienced person in team can do that. This
stage will also have complexity metrics (based on Cyclomatic complexity
theorem) and effort metrics as discussed in above stage. However we can
use parametric approach like Function Point and COCOMO and check
with our actual output.
d) Testing/De bugging Stage:- The powerful metrics in this stage will be:-
Page 5 of 7
a. Defects fixed versus defects introduced:- The basic definition of
defect fixed is handling the exception raised by the defect. Now
handled exception may also introduce new defects. The rule of
thumb is
Number of fixed Defects > Number of new defects in same
space of code
However the main challenge is sometimes integration brings in new
code to expose or since exception brings in new functionality to run,
and it exposes other unleashed errors, so where to put all those
errors. Obviously they will be part of new errors, so our new
equation should look like:-
Number of fixed Defects > Number of new defects in fixed
space of code
b. Defect frequency:- There should be a number of iterations to
remove maximum defects from the code before final rollout. One
can place some rules of thumbs like 2.5 defects per 100KLOC or 1
defect per 5 FP. However this number is not fixed and make sure it
should not overshoot customer satisfaction tolerance level.
There are some other Generic Metrics, which are currently out of scope of this
paper, since they are concepts in their own sense. Some of them are listed
below:
a) Earned Value or Schedule Status Metrics
b) Customer Satisfaction (for Control Projects)
c) Project Plan (WBS)
d) Risk Metrics Management
Page 6 of 7
Metrics Requirement Design Implementation Testing Closing Control
Name Stage Stage Stage Stage Stage Stage
Project Plan Y Y Y Y Y Y
Requirement Y N N N N N
consistency
metrics
Traceability Y Y Y Y Y N
Metrics
Error Metrics Y Y Y Y Y Y
Effort N Y Y Y N Y
Metrics
Figure 5: Example of Checklist for Metrics to be covered at each stage of Software Development
Conclusion:
Metrics are generally looked as a threat by the development team, however
they are excellent tools which can not only help curbing uncertainties but also
keep things realistic and manageable.
Page 7 of 7
Related docs
Other docs by sys20543
