VIEWS: 13 PAGES: 29 POSTED ON: 11/3/2012
Testing: A Roadmap Mary Jean Harrold Presented by : Atul Vij CSCI 589 Fall 2004 Definition A verification method that applies a controlled set of conditions and stimuli for the purpose of finding errors. Paper Says: It’s a process that is performed to support quality assurance Testing Making test Cases Executing the application with those test cases. Examining the results produced by those executions. Why Further Research On Testing??? Expensive Testing accounts for 50% of the cost of software development Higher for critical systems Cumbersome New tools need to be developed for easy and automatic Testing. What this paper is about A report written in 2000 assesses the state of the art in software testing outlines future directions in software testing gives pointers to software testing resources. Software Testing Roadmap Component Based Systems Increased size and complexity has led to component based Component systems. Advantages: Saves Money Saves Time Code Reusability Reduces Complexity Examples Provider User Java beans, MFC, Java Views component Views Component Applets, ActiveX independent Relevant to the of the context Application Component User Reusable Repository Service Select Semantics Component Understand Composible Assembly System Under Assemble Executable Compile System Component User Problems Faced: Availabilty of source code. Heterogenity Component Trust Problem Lack of testing mechanisms. Suggestions Theory of Test Adequacy by Rosenblum Ongoing Research on providing testing tool along with the component. Roadmap Efficient Techniques and Tools for: Testing various aspects like security, dependability, safety. Representing and computing the types of testing information that a component user might need. Storing and using testing information in the component. Precode Artifacts Based on: Design Requirements Architecture Description of elements from which system is built. Interaction among those elements. Pattern that guide their composition and their constraints on these patterns. Testing based on Architecture Specification Chemical Abstract Machine (CHAM) CHAM defined by a set of molecules(m1,m2…), solutions(s1,s2,…) and transformation rules(t1,t2,…) Transition Graph depicting all possible behavior of the system is derived using software architecture Derive a specific subgraph by selecting an appropriate criterion Derive a test plan to cover all the transitions in these subgraphs Example of CHAM Lets say the criterion is concurrent reduced graph whatever scheduling policy the system will adopt it will certainly have to consider this subgraph (for detailed algorithms refer ) Transition Graph for sequential compiler Concurrent Reduced Graph Roadmap Techniques and tools for: Test case requirements for accessing various elements of the system Systematic development of test cases early in the development plan. Automatic generation of test cases. Evaluate software architecture for testability and hence choose the best architecture early in design phase. Testing Evolving Software Regression Testing : attempts to validate modified software and ensure no new errors are introduced into previously tested code. Most expensive activity in a software system lifetime. Selective Regression Testing Roadmap Techniques and tools for: Assist in managing the size of the test suite Reduce the size of test suite Identification of test cases that because of modification is no longer needed. Prioritize test cases. Assess the testability of both software and test suites. Demonstrating Effectiveness of Testing Techinques HOW??? Increase confidence of developer Software behavior Provide visual interface Identify classes of faults for a given test criteria. Test complex boolean expression – Modified condition/decision coverage Modified Condition/Decision Coverage Every point of entry and exit in the program has been invoked atleast once every condition in a decision in the program has taken on all possible outcomes atleast once each condition has been shown to independently affect the decisions outcome Roadmap Determine adequacy of test suites and test cases that inspire confidence. Determine interaction between the various test- selection criteria and find ways to combine them to perform more effective testing Test cases based on data flow use existing testing techniques to test visual programming languages test complex Boolean expressions Effective Processes for Testing Process used for planning and implementing it. Perpetual testing Microsoft uses a different approach Synchronize and Stabilize as project proceeds. Build and Test a version every night. Regression Testing Roadmap Does Microsoft nightly rebuild, minimize testing later ? Test and Validate the existing models. Generalized Results obtained by Testing Combine various techniques and tools for testing Combine static analysis with testing Using Testing Artifacts Artifacts: Execution Traces, Results of Test case execution. How to use Artifacts: Dynamic Program Slides Identifying the invariants in the Program. Predicting the percentage of test suite that need to be used after modification Regression Testing Visualization Technique Roadmap Verifying that the existing techniques provide useful information New Techniques for testing to be used by Software Engineers Better Visualization Other Techniques Some other techniques helpful in reaching end goal (quality software) Need for a scalable automatic test data generation Static analysis required but expensive. Need for a scalable analysis technique that can be used to compute required information. data-flow analysis expensive and hence need for efficient instrumentation and recording techniques. Other Topics Methods and Tools Scalability Computational tradeoffs Automatic generation Empirical Studies Testing Resources Conclusion • Emphasizes the basic stages-Sets up the next paper. • Examining Current Techniques to demonstrate scalability & usefulness of techniques in practice- Empirical. • Relevance To Embedded Systems • Talks about evolving systems-ES evolve by the second. • Talks about component based testing-COTS. • Emphasizes the need for Testing based on Precode Artifacts- Software Architecture. References  Antonia Bertolino, Paola Inverardi, Henry Muccini, Andrea Rosetti, “An Approach to Integration Testing based on Architectural Descriptions”  Antonia Bertolino, Andrea Polini, “A Framework for Component Deployment Testing”  M.J.Harrold, R.Gupta and M.L Soffa, “A methodology for controlling the size of test suite”  G. Rothermel , M.J. Harrold, “Analyzing Regression Test Selection Techniques”  A.Bertolino,P Inverardi,H.Muccini, “Software Architecture and integration testing” Thank You! Any Questions??
Pages to are hidden for
"Advantages - PowerPoint"Please download to view full document