Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out

Optimization in Software Test Management

VIEWS: 22 PAGES: 10

The process of managing the software test life cycle (STLC) of the application under test (AUT) is known as Test Management. It starts with test planning activity and ends with test results reporting. The objectives of test management are to manage requirements dynamically, effectively utilize the 'testing cycle' time and efficient use of testers in terms of their productivity. Test management tool known as 'Quality Center (QC)' is widely used for managing the STLC of the project. Test case prioritization techniques schedule test cases for execution in an order that attempts to maximize some objective function. A variety of objective functions are applicable; one such function involves rate of fault detection — a measure of how quickly faults are detected within the testing process. An improved rate of fault detection during regression testing can provide faster feedback on a system under regression test and let debuggers begin their work earlier than might otherwise be possible. In this paper we have presented a method of choosing risk-based test cases. Our risk analysis is based on a practical risk model, and is similar to that used by some organizations.

More Info
									IJCSN International Journal of Computer Science and Network, Vol 2, Issue 1, 2013                                                  141
ISSN (Online) : 2277-5420




                  Optimiza
                  Optimization in Software Test Management
                                            1
                                                Soumya Samal, 2Ayeskant Mohapatra, 3Sanjay Ku.Padhi
                                  1
                                      Computer Science, Bipu Patnaik University of Technology, KIST
                                                      Bhubaneswar, Odisha, India
                                      2
                                          Computer Science, Biju Patnaik University of Technology, HIT
                                                         Bhubaneswar, Odisha, India
                                  3
                                      Computer Science, Biju Patnaik University of Technology, KIST
                                                     Bhubaneswar, Odisha, India



                            Abstract                                       Software developers often save the test suites they develop
The process of managing the software test life cycle (STLC) of             for their software, so that they can reuse those suites later
the application under test (AUT) is known as Test Management.              as the software evolves [1]. Such test suite reuse, in the
It starts with test planning activity and ends with test results           form of regression testing, is pervasive in the software
reporting. The objectives of test management are to manage                 industry and, together with other regression testing
requirements dynamically, effectively utilize the 'testing cycle'
                                                                           activities, can account for as much as one-half of the cost
time and efficient use of testers in terms of their productivity.
Test management tool known as 'Quality Center (QC)' is widely              of software maintenance. Regression testing is well known
used for managing the STLC of the project. Test case                       to be an important software maintenance activity
prioritization techniques schedule test cases for execution in an          performed with the aim of gaining confidence in modified
order that attempts to maximize some objective function. A                 software. It is an essential part of an effective testing
variety of objective functions are applicable; one such function           process for ensuring software quality. Moreover,
involves rate of fault detection — a measure of how quickly                regression testing is also an important activity in object-
faults are detected within the testing process. An improved rate           oriented software development [2].
of fault detection during regression testing can provide faster
feedback on a system under regression test and let debuggers
begin their work earlier than might otherwise be possible. In this
paper we have presented a method of choosing risk-based test
cases. Our risk analysis is based on a practical risk model, and is
similar to that used by some organizations.

Keywords: Fault detection, Regression testing, Risk model,
Test cases, Test management.

1. Introduction
With the advent of newer technologies and platforms,
                                                                                                   Fig 1: RPN Cycle
today’s applications are becoming increasingly complex.
Huge competition and rising costs of application failure
and downtime have raised the need for testing to new
dimension. While the pressure to deliver high-quality                      2. Test Case Preparation
applications continues to mount, shrinking development
and deployment schedules, spatially distributed                            A test case in software engineering is a set of conditions or
organizations, outsourcing and high turnover rates for                     variables under which a tester will determine whether an
skilled employees make application testing challenging                     application or software system is working correctly or not.
[1]. To overcome these challenges, many organizations are                  The mechanism for determining whether a software
adopting test management methodologies and are turning                     program or system has passed or failed such a test is
to automated test management tools to help centralize,                     known as a test oracle. In some settings, an oracle could be
organize, prioritize and document their testing efforts.                   a requirement or use case; while in others it could be a
                                                                           heuristic [5].
IJCSN International Journal of Computer Science and Network, Vol 2, Issue 1, 2013                                          142
ISSN (Online) : 2277-5420


3. Workflow for Test Case Optimization
                                                                      We adopt two approaches for test Case optimization. One
3.1 Risk Failure Assignment for Test Case using                       is Failure mode Effect Analysis

FMEA and RBT




                                Fig 2: Work Flow Diagram for Test Case Optimization using FMEA and RBT.

(FMEA) and other is Risk Based Testing (RBT). FMEA                    average of Cost of the requirement attributes tested (CCt)
depends on Risk Priority Number, which is the                         ,Vendor Fault Cost (CVt) and Severity probability of test
multiplication of Severity, Occurrence and Detection. But             cases (Pt) is nothing but the multiplication of Number of
the problem with this method is that we are unable to                 Defects (Nt), Average Severity of Defects (St). Out of
calculate the risk exposure. On the other hand risk                   these two methodologies (i.e. FMEA & RBT), FMEA is
exposure can be calculated using risk based approach. For             still emerging techniques, while RBT has taken giant
finding the Risk Exposure Factor, we have to find out Ct              strides to test case optimization.
(Cost of test cases) & Pt (Severity probability of test
cases). Here Cost of test cases (Ct) is nothing but the
IJCSN International Journal of Computer Science and Network, Vol 2, Issue 1, 2013                                           143
ISSN (Online) : 2277-5420




                                Fig 3: Work Flow Diagram for Risk Failure Assignment using Risk exposure



4. Anatomy of a Test Case                                                        TEST CASE2: A=TRUE, B=FALSE, C=TRUE
                                                                                 TEST CASE3: A=FALSE, B= TRUE, C=TRUE
Implementation of test case optimization techniques like                         TEST CASE4: A=TRUE, B=TRUE, C=FALSE
Risk Based Testing Approach aid in reducing test cycle
times during reduced timelines [6][7]. Test Data                       NOTE: Benefit saved two Test Cases above
Optimization is one such technique to get around the issue             Risk Based Testing is technique of optimizing test
of lengthy test execution timelines. E.g., Selective Test              planning based on the defining risks levels for test cases
Data Refresh, where a subset of the entire test data is                and prioritizing for Test Execution [8].
considered while testing a particular functionality or build.
Risks are benchmarked at different levels [3]. The Test                4.1 Risk Priority Number (RPN)
Bed is optimized to contain higher ratio of test cases
addressing High Risks and a mix of medium risks and low                The RISK PRIORITY NUMBER (RPN) is the product of
risks [4].                                                             the SEVERITY (S), OCCURRENCE (O), and
                                                                       DETECTION (D) ranking. The RPN is a measure of
                                                                       software risk. The RPN is also used to rank order the
MCDC=       MODIFIED      CONDITION     DECISION
                                                                       concerns in processes (e.g., in Pareto fashion) [8]. The
COVERAGE Technique.                                                    RPN will be between “1” and “1,000.” For higher RPNs
Example 1: Unique Case MCDC                                            the team must undertake efforts to reduce this calculated
       TEST CASE1: A=FALSE, B=FALSE                                    risk through corrective action(s).
       TEST CASE2: A=FALSE, B=TRUE                                                RPN = (S) x (O) x (D)……………. (1)
       TEST CASE3: A=TRUE, B=FALSE
NOTE: Benefit saved one Test Case above
Example 2: Masking MCDC
       TEST CASE1: A=FALSE, B=FALSE, C=TRUE
IJCSN International Journal of Computer Science and Network, Vol 2, Issue 1, 2013                                                     144
ISSN (Online) : 2277-5420

4.2 Evaluation Criteria                                              Occurrence (O)

Severity (S)                                                        OCCURRENCE (O) is the likelihood that a specific
SEVERITY (S) is an assessment of the seriousness of the             cause/mechanism will occur i.e., Defect Occurrence in
effect of the potential failure mode [9] i.e., Defect Severity      software testing terms. OCCURRENCE is estimated on a
in software testing terms. SEVERITY applies to the effect           “1” to “10” scale [9].
only.
                                                                                                             Possible
    Effect         Criteria: SEVERITY of             Ranking              Probability of Failure             Failure
                             Effect                                                                           Rates            Ranking
                Very high defect severity                 10                                                  1 in 2             10
                ranking when a potential                               Very High: Failure is almost
 Hazardous      failure mode affects major                                     inevitable                     1 in 3             9
 without        software functionality and/or                                                                 1 in 8             8
                                                                         High: Repeated Failures
 Warning        involves noncompliance with                                                                  1 in 20             7
                regulations / software design                                                                1 in 80             6
                without warning.                                      Moderate: Occasional Failures          1 in 400            5
                Very high defect severity                 9                                                 1 in 2,000           4
                ranking when a potential                                                                       1 in
 Hazardous      failure mode affects major                                                                    15,000             3
                                                                      Low: Relatively Few Failures
 with           software functionality and/or                                                                  1 in
 Warning        involves noncompliance with                                                                  150,000              2
                regulations / software design                                                                  1 in
                                                                       Remote: Failure is Unlikely
                with warning.                                                                               1,500,000            1
                Software Product/component                8                    TABLE 2: possibility of Occurrence of Failure
                inoperable, with loss of
 Very high
                primary function.                                   Detection (D)
                Software Product/component                7         In order to achieve a lower ranking, generally the planned
                operable, but at reduced level                      testing activities (e.g., preventative, validation, and/or
 High           of performance. Customer                            verification activities) have to be improved. DETECTION
                dissatisfied.                                       is estimated on a “1” to “10” scale [9].
                Software Product/component                6
 Moderate       operable, but may cause                                 Detection        Criteria: Likelihood of           Ranking
                rework or a defect.                                                     DETECTION by Design
                Software Product/component                5                                      Control
                operable, but may cause                               Absolute         Test Case Design will not                10
 Low
                slight inconvenience to                               Uncertainty      and/or cannot detect a
                related operations.                                                    potential cause/mechanism
                Software Product/component                4                            and subsequent failure
                operable, but possesses some                                           mode; or there is no Test
 Very Low
                minor defects (cosmetic)                                               Case Designed.
                noticeable to most customers.                         Very             Very remote chance the                    9
                Software Product/component                3           Remote           Test Case Design will
                operable, but may possess                                              detect a potential
 Minor          some minor defects                                                     cause/mechanism and
                noticeable by discriminating                                           subsequent failure mode.
                customers.                                            Remote           Remote chance the Test                    8
                Software Product/component                2                            Case Design will detect a
 Very           operable, but is in                                                    potential cause/mechanism
 Minor          noncompliance with                                                     and subsequent failure
                company policy.                                                        mode.
 None           No effect.                                1           Very Low         Very low chance the Test                  7
                                                                                       Case Design will detect a
         TABLE I: Ranking Effects Depending on Severity
                                                                                       potential cause/mechanism
IJCSN International Journal of Computer Science and Network, Vol 2, Issue 1, 2013                                                      145
ISSN (Online) : 2277-5420

                     and subsequent failure
                     mode.
  Low                Low chance the Test Case              6                Co          Mnemo
                     Design will detect a                                                                 Description
                                                                       de            nic
                     potential cause/mechanism                                          Unorder
                     and subsequent failure                                 µ1                         no prioritization (control)
                                                                                     ed
                     mode.                                                  µ2          Random         randomized ordering
  Moderate           Moderate chance the Test              5                                           ordered to optimize rate of
                     Case Design will detect a                              µ3          Optimal
                                                                                                       fault detection
                     potential cause/mechanism
                                                                                                       prioritize in order of total
                     and subsequent failure                                             FEP-
                                                                            µ4                         probability of exposing
                     mode.                                                           total
                                                                                                       faults
  Moderately         Moderately high chance                4
                                                                                                       prioritize in order of total
  High               the Test Case Design will
                                                                                       FEP-            probability of exposing
                     detect a potential                                     µ5
                                                                                     addtl             faults, adjusted to consider
                     cause/mechanism and
                                                                                                       effects of previous tests
                     subsequent failure mode.
  High               High chance the Test Case             3                     TABLE 4: Prioritization for Rate of Fault Detection
                     Design will detect a
                     potential cause/mechanism                      5.1 Risk-based Test Case Selection of Safety Tests
                     and subsequent failure
                                                                    (µ1 for no prioritization)
                     mode.
  Very High          Very high chance the Test             2
                                                                    The test case prioritization techniques schedule test cases
                     Case Design will detect a
                                                                    for execution in an order that attempts to maximize some
                     potential cause/mechanism
                                                                    objective function [11]. A variety of objective functions
                     and subsequent failure
                                                                    are applicable; one such function involves rate of fault
                     mode.
                                                                    detection — a measure of how quickly faults are detected
  Almost             Test Case Design will                 1        within the testing process. An improved rate of fault
  Certain            almost certainly detect a                      detection during regression testing can provide faster
                     potential cause/mechanism                      feedback on a system under regression test and let
                     and subsequent failure                         debuggers begin their work earlier than might otherwise be
                     mode.                                          possible.
            TABLE 3: Detection of Risks Based on Ranking
                                                                    First, we calculate the Risk Exposure for each test case
                                                                    (REt), the subscript stands for test case. Then, we choose
In this section we have taken the Risk exposure factor for
                                                                    test cases with the highest REt for our regression suite of
each test case. From each test case, risk exposure for each
                                                                    Safety Tests. There are four main steps in our approach
scenario has been generated. Then Regression testing has
                                                                    [11]:
been performed with the test suite having highest REs
value. The regression test suite that includes the risk test        Step 1: Assess the cost (impact of potential failure covered
scenario and test case can be revisited from time to time in        by this test case) Ct for each test case.
order to look at further test optimization opportunity.
REscan be calculated as:                                            In our case, we are not sure which should be weighted
                                                                    heavier than the other. However, our approach is
   REs = ΣRE , {1 ≤ i ≤ n | test case ti is covered by this         sufficiently flexible to allow weights to favor either
                it
                                                                    customer or vendor interests. So we have taken the
                    scenario}…. (2)                                 average of CC and CV as our cost C. The formula for
Where S = ∑t (Test Scenario is a group of test cases)               calculating C is therefore [12] [13]:

5. Test Case Prioritization for Rate of fault                                       C = (CC + CV) / 2 …………… (3)
   Detection                                                        Where C denotes the average of CC and CV, CC denotes
                                                                    the failure as seen by the customer & CV denotes failure
We consider five different test case prioritization                 as seen by the vendor
techniques. Table 4 lists these techniques; we next discuss         Here we define failure cost (or simple cost) of test case Ct
them in an order that facilitates their presentation [10].          as the cost of the requirement attributes tested by this test
IJCSN International Journal of Computer Science and Network, Vol 2, Issue 1, 2013                                                   146
ISSN (Online) : 2277-5420

case. Therefore, we have the following formula to                            …                       ....     ...   ……
calculate Ct:
                                                                                    TABLE 6: Risk Exposure Ret of Test Cases
           Ct = (CCt + CVt) / 2 …………… (4)
Where Ct = cost of the requirement attributes tested by this        Step 4: Select regular test cases as Safety Tests
test case. CCt & CVt = value on a one to five scale (1 –
Low, 5 – High).
                                                                    In this section, we provide our approach for selecting
Step 2: Derive severity probability for each test case              feature-oriented, regular test cases as Safety Tests.A
(randomized ordering µ2)                                            regular test case is often designed to test a specific local
                                                                    feature. In our case, we selected 30% of the full test suite
We believe that using severity probability instead of               to form regression test suite. 30% was selected to make
probability in our risk mode is important, because severity         our regression test suite comparable to the manually
probability weighs the test cases that had serious defects in       selected regression test suite of historical data that had
previous test execution. Severity probability is calculated         30% coverage.
based on test profiles. The value of severity probability
will change after new test execution. But testers are not                Test         Full Test             Regression Test Suite
allowed to change the value manually.                                   Case          Suite                 (30%)
                                                                        t0010               1                          1
    Test       Number of              Average              Nt ×         t0020               1                          1
    Case       Defects (Nt)          Severity of            St          t0030               1                          0
                                     Defects (St)                       t0040               1                          0
    t0010            1                    2                 2
                                                                        t0050               1                          1
    t0020            1                    3                 3
                                                                        t0060               1                          1
    t0030            1                    1                 1
                                                                        t0070               1                          0
    t0040            1                    4                 4
                                                                         …..               …..                        …..
    t0050            2                    3                 6
    t0060            3                   3.5               10.5                           TABLE 7: Test Case Selected
    t0070            0                    0                 0       At this stage, we could stop since, we have systematically
    ……               ….                 …..                 ….      achieved a good level (better than the manual level) of
                                                                    regression coverage with the combined Targeted Tests and
             TABLE 5: Severity Probability of Test Cases            Safety Tests.

Where Nt = Number of Defects, St = Average Severity of              5.2 Customer Fault Cost (Cc)
Defects, Pt = Severity probability of test cases.
                                                                    When testers test a software system, they should simulate
Step 3: Risk Exposure for each test case                            the behaviors of customers running the operational system.
                                                                    Testers are therefore highly encouraged to understand how
The Risk Exposure REt is our basis for choosing test cases          customers use the system. Based on their experience and
as Safety Tests [9]. To enhance or focus the results, we            knowledge of the customers, testers are often able to
can also add weights to test cases that we must give                estimate CCt. Once scores for all test cases have been
preference to.                                                      tabulated, we assess CCt in this way [12]:
For example, we might like to choose more test cases for
some corporate flagship features, such as database                       •    For test cases in the top 20% of scores, CCt = 5
features, if we are a database systems provider.                         •    For test cases scoring between the top 20% and
                                                                              40%, CCt = 4
            Test Case          Ct     Pt    REt = Ct × Pt                • For test cases scoring between the top 40% and
                                                                              60%, CCt = 3
              t0010            4.5    2           9
                                                                         • For test cases scoring between the top 60% and
              t0020             4     3          12                           80%, CCt = 2
              t0030             3     2           6                      • For test cases in the bottom 20% of scores, CCt
              t0040            2.5    3          7.5                          =1.
              t0050             2     4           8                 For example, test case t0010 has score 17, which belongs
              t0060            3.5    5         17.5                to the top 20% of course. Thus, its CCt is assigned 5.
              t0070            1.5    1          1.5
IJCSN International Journal of Computer Science and Network, Vol 2, Issue 1, 2013                                          147
ISSN (Online) : 2277-5420

     Test Case      Q1    Q2     Q3   …       Total   CCt                           ……         …..      …..         …..
     t0010           2    0      2    ..…      17      5
     t0020           0    2      2    ..…      8       3                    TABLE 10: Cost of a Few Real Test Cases
     t0030           2    0      2    ..…      6       2
     t0040           0    2      1    ..…      12      4            6. Risk-based Test Scenario Selection
     t0050           0    0      1    ..…      3       1
     t0060           2    0      0    ..…      5       2            Risk-based test scenarios simulate common user profiles
     t0070           0    0      1    ..…      3       1            of system use. Often they are designed based on use cases
                                                                    documented while capturing requirements or early in the
             TABLE 8: Cc of Some Actual Test Cases                  system design phase. Some of them are equivalent to use
                                                                    cases.
5.3 Vendor Fault Cost (Cv)
                                                                    6.1 Rules for Scenario Selection
For a vendor, the maintenance cost is the major part of the
cost of a fault. In this paper, we consider the cost rather at      When test time is limited, we might be not able to run all
the component-to-component level than at the individual             end-to-end scenarios for the system. Thus, to reduce risk
LOC level. We employ a questionnaire instead of applying            efficiently, we give the following two rules for scenario
metrics from complexity theory, such as McCaler’s                   selection [15][16]:
Complexity Cycle metrics, for the estimation [12].                     R1: Select scenarios that test the most critical
                                                                    requirement attributes.
     Test     Comp       Comp       Compt =           CVt              R2: Have the scenario tests cover as many requirement
     Case      CFt       Dt(d)     CompCFt ×                        attributes as possible.
                                    CompDt
     t0010     3           4          12              4             An end-to-end scenario runs across several components of
                                                                    the system under test. Often it is a serialized sequence of
     t0020     5           3          15              5
                                                                    regular test cases for these components. Since every
     t0030     2           5          10              4
                                                                    regular test case tests one or more requirement attributes,
     t0040     2           1           2              1
                                                                    by the requirements traceability the rules shown above are
     t0050     4           2           8              3             equivalent to [13]:
     t0060     4           5          20              5
     t0070     1           4           4              2               T1: Select scenarios that contain the most critical
     …        …..         …..         …..             …..           regular test cases.
                                                                      T2: Have the test suite of scenarios cover as many
             TABLE 9: CV of Some Actual Test Cases                  regular test cases as possible.To be brief, in this section,
                                                                    we use term “scenario” for “scenario test cases”, and “test
In table 9, CompCFt denotes complexity of control flows,            case” for “regular test case”.
CompDt denotes complexity of test data cost to the vendor
CVt denotes measuring the complexity of the system for              6.2 Scenarios and Coverage of Test Cases
each test case.
                                                                    Before doing scenario selection, for every end-to-end
5.4 Case cost estimates (Ct)                                        scenario, we have to know which test cases are covered by
                                                                    each scenario [15].
Now that a cost estimate has been derived for each test
case, we can derive a failure probability enhanced by                       Scena                    Test Case
severity weights, called severity probability [12].                         rio
                                                                             s001     t0010, t0020, t0030, t0040, t0050,
             Test Case      CCt       CVt      Ct                                    …
               t0010         5         4       4.5                           s002     t0030, t0050, t0060, t0070, …
               t0020         3         5        4                            S003     t0040, t0070, …
               t0030         2         4        3                             ….      …
               t0040         4         1       2.5
               t0050         1         3        2                                    TABLE 11: Test Case Selected
               t0060         2         5       3.5
               t0070         1         2       1.5
IJCSN International Journal of Computer Science and Network, Vol 2, Issue 1, 2013                                                148
ISSN (Online) : 2277-5420

Since one scenario is a sequence of test cases, we keep this        The scenario with the highest REs covers the most critical
information in a table 12.                                          test cases, or has the highest coverage of risk severity-
                                                                    covering test cases. According to our selection rules, it
6.3Traceability between test cases and scenarios                    should be included in the regression test suite. In our
                                                                    example, scenario s001 has the highest REs in Table 13.
            Test                Scenario                            Therefore, it is put first into the regression scenario test
           Case                                                     suite [13].
                     s001     s002 s003         ....
           t0010       1        0      0        ….                  Step 3: Update the traceability matrix and re-build REs
           t0020       1        0      0        ….                  table     (Prioritize in order of total probability of
           t0030       1        1      0        ….                  exposing faults, adjusted to consider effects of previous
           t0040       1        0      1        ….                  tests µ5)
           t0050       1        1      0        ….
           t0060       0        1      0        ….                  In our approach, after the chosen scenario has been
           t0070       0        1      1        ….                  executed, we delete the column for the chosen scenario
             …        …..      ….    …..        …                   and rows for all the test cases that have been covered by
                                                                    this scenario from the traceability matrix, which is given in
              TABLE 12: Test Case Selected                          Table 12 in our example.

Where ‘0’ denotes the scenario that does not cover the test
case and ‘1’ denotes the scenario that covers the test case.                    Test                Scenario
                                                                               Case      s001     s002 s003           .....
6.4 End-to-end Safety Test Scenarios Selection                                 t0010       1        0      0          .....
   Method (Ordered to optimize rate of fault                                   t0020       1        0      0          .....
   detection µ3)                                                               t0030       1        1      0          .....
                                                                               t0040       1        0      1          .....
Step 1: Calculate Risk Exposure for each scenario                              t0050       1        1      0          .....
                                                                               t0060       0        1      0          .....
In table 6 we show the Risk Exposure REt for each test                         t0070       0        1      1          .....
case. Since scenarios consist of test cases, we can simply
calculate the Risk Exposure for each scenario REs by
summing up the Risk Exposure REt of all test cases that                      TABLE 14: Process of Updating Traceability Matrix
this scenario covers [13]. If a test suite consists of n test
cases, then Res can be Calculated as:                               Based on Table 14, we can calculate REs again for the
                                                                    remaining scenarios and re-build Table 15.
  REs = ΣREit, {1 ≤ i ≤ n | test case ti is covered by this
                        scenario}                                                        Scenario        REs
                                                                                           s001          736
                     Scenario      REs                                                     s002          356
                       S001        985                                                     s003          611
                       S002        463                                                     s004          176
                       S003        732                                                     s005          180
                       S004        213                                                     s006           96
                       S005        195                                                     s007           68
                       S006        127
                       S007         70                                          TABLE 15: Risk Exposure Res for Scenarios
                        …..        …..
                                                                    It can be seen from table 15 that scenario s001 is the one
           TABLE 13: Risk Exposure REs for Scenarios                with the highest REs and is our next choice for regression
                                                                    testing. After adding s003 to our regression scenario test
Step 2: Select scenarios that has the highest REs (prioritize       suite, we repeat the process (Step 2 and Step 3) until it is
in order of total probability of exposing faults µ4)                time to stop.
IJCSN International Journal of Computer Science and Network, Vol 2, Issue 1, 2013                                       149
ISSN (Online) : 2277-5420

The size of our test suite of scenarios can be dependent on          [3] Ozarin the Omnicon Group Inc.40 Arkay Drive,
the time and resources available. Regression test selection              Hauppauge, New York 11788 USA The Role of
is terminated whenever we run out of time or resources.                  Software Failure Modes and Effects Analysis for
Each time we select one test scenario, the Risk Exposure                 Interfaces in Safety- and Mission-Critical Systems
for un-selected scenarios are decreased. Therefore, we can               SysCon 2008 - IEEE International Systems
also decide when to stop choosing more regression test                   Conference Montreal, Canada, April 7-10, 2008.
scenarios based on the Pareto principle (also known as the           [4] Kelly J. Hayhurst Langley Research Center,
80:20 Rule and the law of the vital few, which states that               Hampton, Virginia A Practical Tutorial on
for many phenomena 80% of consequences stem from                         Modified      Condition/      Decision     Coverage
20% of the causes) or based on whether we have reached a                 NASA/TM-2001-21087
pre-defined Risk Exposure range of coverage.
                                                                     [5] Proceedings of the International Conference on
                                                                         Software Maintenance, Oxford, UK, September,
        Test Cases        REs Before        REs After                    1999, IEEE Copyright Test Case Prioritization: An
           s001              985              736                        Empirical Study, First published in Software
           s002              463              356                        Testing and Quality Engineering Magazine, 11/99
           s003              732              611                        Copyright 1999, James Bach Heuristic Risk-Base.
           s004              213              176                    [6] [Amland 99a] Stale Amland, Risk Based Testing
           s005              195              180                        and Metrics: Risk analysis fundamentals and
           s006              127               96                        metrics for software testing including a financial
           s007               70               68                        application case study, 5th International Conference
                                                                         EuroSTAR’99, November 1999, Barcelona, Spain.
 TABLE 16: Comparison of Res Values before and after Optimization    [7] Risk and Decision Analysis 1 (2009) 21–33 21 DOI
                                                                         10.3233/RDA-2008-0002 IOS Press A risk
7. Conclusion                                                            management approach to RBAC. Ebru Celikel a,
                                                                         Murat Kantarcioglu a, Bhavani Thuraisingham a
In test optimization out of all software test life cycle                 and Elisa Bertino b a University of Texas at Dallas
phases, test case preparation phase has the highest scope of             Richardson, TX 75083, USA b Purdue University,
optimization. The result shows that test case preparation                West Lafayette, IN 47907, USA
phase scores over other phases in terms of being a                   [8] N. C. Audsley, A. Burns, M. F. Richardson, and A.
candidate for optimization. That test case preparation                   J. Welling, “Software test management in real-time
phase scores over other phases in terms of being a                       scheduling: The deadline monotonic approach,” in
candidate for optimization. The possible methods of                      Proc. 8thIEEE Workshop on Real-Time Operating
optimization for test case preparation phase area are                    Syst. Software., Atlanta, GA, 1991, pp. 133–137.
FMEA & RBT. Out of these two methodologies, FMEA is                      Santiago Delgado National Instruments Parallel
still emerging techniques, while RBT has taken giant                     Testing Techniques for Optimizing Test Program
strides to words test case optimization. In this paper, an               Execution and Reducing Test Time IEEE
attempt has been made to implement RBT and then rolling                  AUTOTESTCON 2008 Salt Lake City, UT, 8-11
it up and defining a risk at the test scenario level. The                September 2008
regression test suite that includes the risk test scenario and
                                                                     [9] The Object-FMA Based Test Case Generation
test case can be revisited from time to time in order to look
                                                                         Approach for GUI Software Exception Testing Jing
at further test optimization opportunity.
                                                                         Lai, Hong Zhang, Baiqiao Huang School of
                                                                         Reliability and System Engineering, BeiHang
References                                                               University Beijing, Chin jingial@163.com
                                                                    [10] Jacques Sauve, Member, IEEE, Rodrigo Santos,
 [1]   First published Implementation of Effective Test
                                                                         Rodrigo Reboucas, Antao Moura, and Claudio
       Management System - Version 1.0 5th
                                                                         Bartolini, Member, IEEE Change Priority
       International Conference Euro STAR '99,
                                                                         Determination in IT Service Management Based on
       November 8 – 12
                                                                         Risk Exposure IEEE TRANSACTIONS ON
 [2]   201O International Conference, IEEE Copyright on                  NETWORK AND SERVICE MANAGEMENT,
       Computer Design And Applications (ICCDA                           VOL. 5, NO. 3, SEPTEMBER 2008.
       2010). An Optimization to Automatic Fault Tree
                                                                    [11] Gregg Rothermel Department of Computer Science
       Analysis and Failure Mode and Effect Analysis
                                                                         Oregon State U. Corvallis, OR grother@cs.orst.edu
       Approaches for Processes
                                                                         Test Case Prioritization: An Empirical Study
IJCSN International Journal of Computer Science and Network, Vol 2, Issue 1, 2013   150
ISSN (Online) : 2277-5420

       Proceedings of the International Conference on
       Software Maintenance, Oxford, UK, September,
       1999, IEEE Copyright.
[12]   5th International Conference EuroSTAR '99,
       November 8 - 12, 1999, Barcelona, Spain Risk
       Based Testing and Metrics Risk Analysis
       Fundamentals and Metrics for software testing
       including a Financial Application case study.
[13]   Stale Amland Hulda, Risk Based Testing and
       Metrics 5th International Conference Euro STAR
       “99, November 8-12, 1999, Barcelona, Spain
[14]   An Analysis of the Requirements Traceability
       Problem Orlena C. Z. Gotel & Anthony C. W.
       Finkelstein Imperial College of Science,
       Technology & Medicine Department of
       Computing, 180 Queen's Gate London SW7 2BZ
       (oczg; acwf@doc.ic.ac.uk)
[15]   Heuristic Risk-Based Testing By James Bach, First
       published in Software Testing and Quality
       Engineering Magazine, 11/99 Copyright 1999,
       James Bach

 First Author Mr.Soumya Samal, did M.Tech as well as B.Tech in
 Computer Science. Presently working as Sr. Software Executive
 in IBM Business Partner .Previously worked as Software
 Engineer in Symmetric Technology Pvt Ltd for three years in
 Embedded System..His areas of interest in Embedded System,
 Programming in C, Driver Programming and Software Testing.
 Second Author Mr. Ayeskant Mohapatra, did M.Tech and
 B.Tech in Computer Science .Presently working as Assistant
 Professor in Computer Science Department in Hi-Tech Institute
 of Technology.
 Third Author Mr. Sanjay Kumar Padhi , did M.Tech and B.Tech
 in Computer Science. Presently working Assistant Professor in
 Computer Science Department in Konark Institute of Science and
 Technology.

								
To top