Advantages - PowerPoint by j26QdXa


									Testing: A Roadmap
    Mary Jean Harrold
      Presented by : Atul Vij

      CSCI 589 Fall 2004
 A verification method that
  applies a controlled set of
  conditions and stimuli for
  the purpose of finding
 Paper Says:
   It’s a process that is performed to
    support quality assurance
 Making test Cases
 Executing the application with those test
 Examining the results produced by those
          Why Further Research On
 Expensive
     Testing accounts for 50% of the cost of software
     Higher for critical systems

   Cumbersome
       New tools need to be developed for easy and
        automatic Testing.
    What this paper is about
 A report written in 2000
 assesses the state of the art in software
 outlines future directions in software
 gives pointers to software testing
Software Testing Roadmap
   Component Based Systems
 Increased size and
  complexity has led to
  component based                         Component
 Advantages:
      Saves Money
      Saves Time
      Code Reusability
      Reduces Complexity
 Examples                     Provider               User
Java beans, MFC, Java       Views component     Views Component
  Applets, ActiveX             independent       Relevant to the
                              of the context       Application
          Component User
     Repository                 Service

     Component                             Understand


System Under Assemble

                  Executable                Compile

             Component User
 Problems Faced:
     Availabilty of source code.
     Heterogenity
     Component Trust Problem
     Lack of testing mechanisms.
 Suggestions
   Theory of Test Adequacy by Rosenblum
   Ongoing Research on providing testing tool
    along with the component.
 Efficient Techniques and Tools for:
   Testing various aspects like security,
    dependability, safety.
   Representing and computing the types of
    testing information that a component user
    might need.
   Storing and using testing information in the
           Precode Artifacts
 Based on:
   Design
   Requirements
   Architecture
     Description of elements from which system is built.
     Interaction among those elements.
     Pattern that guide their composition and their
      constraints on these patterns.
   Testing based on Architecture
 Chemical Abstract Machine (CHAM)
   CHAM defined by a set of molecules(m1,m2…),
    solutions(s1,s2,…) and transformation rules(t1,t2,…)
   Transition Graph depicting all possible behavior of the
    system is derived using software architecture
   Derive a specific subgraph by selecting an
    appropriate criterion
   Derive a test plan to cover all the transitions in these
          Example of CHAM
 Lets say the criterion is concurrent
 reduced graph

 whatever scheduling policy the system will
 adopt it will certainly have to consider this
 subgraph (for detailed algorithms refer [5])
Transition Graph for sequential compiler   Concurrent Reduced Graph
 Techniques and tools for:
   Test case requirements for accessing various
    elements of the system
   Systematic development of test cases early in
    the development plan.
   Automatic generation of test cases.
   Evaluate software architecture for testability
    and hence choose the best architecture early
    in design phase.
     Testing Evolving Software
 Regression Testing : attempts to validate
  modified software and ensure no new errors are
  introduced into previously tested code.

 Most expensive activity in a software system

 Selective Regression Testing
 Techniques and tools for:
   Assist in managing the size of the test suite
   Reduce the size of test suite
   Identification of test cases that because of
    modification is no longer needed.
   Prioritize test cases.
   Assess the testability of both software and
    test suites.
  Demonstrating Effectiveness of
      Testing Techinques
 HOW???
  Increase confidence of developer
  Software behavior
  Provide visual interface
  Identify classes of faults for a given test
  Test complex boolean expression – Modified
   condition/decision coverage
  Modified Condition/Decision

 Every point of entry and exit in the program
  has been invoked atleast once
 every condition in a decision in the program
  has taken on all possible outcomes atleast
 each condition has been shown to
  independently affect the decisions outcome
 Determine adequacy of test suites and test
    cases that inspire confidence.
   Determine interaction between the various test-
    selection criteria and find ways to combine them
    to perform more effective testing
   Test cases based on data flow
   use existing testing techniques to test visual
    programming languages
   test complex Boolean expressions
Effective Processes for Testing
 Process used for planning and
  implementing it.
 Perpetual testing
 Microsoft uses a different approach
   Synchronize and Stabilize as project
   Build and Test a version every night.
 Regression Testing
 Does Microsoft nightly rebuild,
    minimize testing later ?
   Test and Validate the existing models.
   Generalized Results obtained by
   Combine various techniques and tools
    for testing
   Combine static analysis with testing
       Using Testing Artifacts
 Artifacts: Execution Traces, Results of Test case

 How to use Artifacts:
   Dynamic Program Slides
   Identifying the invariants in the Program.
   Predicting the percentage of test suite that need to be
    used after modification
   Regression Testing
   Visualization Technique
 Verifying that the existing techniques
  provide useful information
 New Techniques for testing to be used by
  Software Engineers
 Better Visualization
            Other Techniques
 Some other techniques helpful in reaching end goal
  (quality software)

  Need for a scalable automatic test data generation

   Static analysis required but expensive. Need for a
  scalable analysis technique that can be used to compute
  required information.

  data-flow analysis expensive and hence need for
  efficient instrumentation and recording techniques.
              Other Topics
 Methods and Tools
   Scalability
   Computational tradeoffs
   Automatic generation
 Empirical Studies
 Testing Resources

• Emphasizes the basic stages-Sets up the next paper.

• Examining Current Techniques to demonstrate
  scalability & usefulness of techniques in practice-

• Relevance To Embedded Systems
   • Talks about evolving systems-ES evolve by the second.
   • Talks about component based testing-COTS.
   • Emphasizes the need for Testing based on Precode Artifacts-
     Software Architecture.
 [1] Antonia Bertolino, Paola Inverardi, Henry Muccini,
    Andrea Rosetti, “An Approach to Integration Testing
    based on Architectural Descriptions”
   [2] Antonia Bertolino, Andrea Polini, “A Framework for
    Component Deployment Testing”
   [3] M.J.Harrold, R.Gupta and M.L Soffa, “A methodology
    for controlling the size of test suite”
   [4] G. Rothermel , M.J. Harrold, “Analyzing Regression
    Test Selection Techniques”
   [5] A.Bertolino,P Inverardi,H.Muccini, “Software
    Architecture and integration testing”
Thank You!

Any Questions??

To top