Introduction To Software Testing by nirajng

VIEWS: 289 PAGES: 28

									Introduction to Software Testing

Introduction to Software Testing

Introduction to Software Testing

What is Testing ?

Process of exercising or evaluating a system or system component:
? To verify that it satisfies specified requirements ? To identify differences (failures) between:
? ?

Expected results Actual results

? To reduce the risks of not working to acceptable levels

Expected Result


Actual Result

Introduction to Software Testing

Testing v.s. Debugging
The process of analyzing and locating bugs when software does not behave as specified

Testing Testing
The process of executing software in a controlled manner, in order to answer the question “Does the

software behave as specified?”

Introduction to Software Testing


? Human action that results in a defect in the software

What we did to put bugs there


? The actual defectin the software as a result of an error. Faults are

typically referred to as “bugs”.

What the trouble shooter found


? Inability of the software to perform its required function

What the user saw


Introduction to Software Testing

Goals of Testing
1950’s view: Show it Works (Demonstration) 1970’s view: Find Defects (Detection) 1980’s view: Measure Quality (Prediction) 1990’s view: Control Quality (Prevention)

Today, all four of these perspectives are still valid as a reason to test. The issue is not which perspective to choose, but rather how to prioritize the set. The priorities are: ? prevention ? detection ? prediction ? demonstration (if ok)

Introduction to Software Testing

The Traditional vs. Modern Approach
Integration Detailed Design H.L. Design Requirements Code Test & Fix Effort

Integration Detailed Design H.L. Design Requirements Code Identify and Resolve Defects Effort



Release 6

Introduction to Software Testing

W - Whole Life Cycle

Write Req.

Test Req.


Acceptance Test

Logical Design

Test Design
Build Software

Build System

System Test

Physical Design

Test Design

Integration Test


Unit Test

Introduction to Software Testing

Software Testing Methodology What’s in it?
A systematic discipline, akin to that of Software Development, with the aid of strategies, guidelines and procedures


Introduction to Software Testing

Systematic Test and Evaluation Process STEP

Introduction to Software Testing

A Software Testing Methodology
? Systematic Test and

Evaluation Process

* A Systematic Software Testing Method since 1985 * Developed by Software Quality Engineering Co.
? ANSI/IEEE Testing Standards

* ANSI/IEEE 829 Test Documentation Standard * ANSI/IEEE 1008 Unit Test Process Standard


Introduction to Software Testing

STEP Phases

Plan Strategy - for each level
* Master test plan * Detailed test plans


Acquire Testware
* Inventory test objectives * Design tests * Implement tests development by plans and requirements


Measure Behavior
* Execute tests * Check for adequacy * Evaluate testing process

Introduction to Software Testing

What is a Master Test Plan?
? ? ?

The testing policy of the project Reference for all testing reviews It will:
? state test levels ? for each level it will state test targets,
? ? ? ?

responsibilities, environments revisions time tables, etc.


Introduction to Software Testing

Planning Phase Process
Overall Resources Project Info DEVELOP Master Plan Clarifications Master Test Plan Master Test Plan • Identify & prioritize product risks • Determine resource constraints • Develop overall testing strategy • Determine features and their aspects to be tested • Determine resource & training rqmts • Identify responsibilities • Specify an overall schedule • Review the test plan Software Info Level Specific Resources DEVELOP Detailed Plans

Level Specific Plans Level Specific Plans • Identify & prioritize feature risks • Refine testing strategy • Specify features and their aspects to be tested • Specify special resource & tool rqmts • Specify a detailed schedule • Review the test plan 13

Introduction to Software Testing

Testing Levels

? To ensure that each software module complies with the functional

specifications for that module based on design specs. ?

? To test gradually the interfaces among system modules, components and

subsystems. ?

? To test the system as a whole (End-to-End testing) including hardware and

interfacing with all other related production systems. To test the functional and non-functional requirements based on customer requirements for the overall system.


Introduction to Software Testing

Testing Levels (Cont.)

? Alpha (Internal) - To subject approved integrated software to test

conducted in a production-like environment.
? Beta (External) - To conduct a live application test of the software at

selected customer sites before releasing the software product to all customers. ?

? To ensure that the software satisfies all business needs and objectives, as

they are defined by functional and non-functional requirements, through a customer focus. Usually it is a subset of system tests agreed upon with the customer


Introduction to Software Testing

Functional/Non-Functional Testing

? To verify requirements, specifications, rules, objectives,

algorithms, etc. ?

Non-Functional (Operational)
? To verify that the system will work in production and will provi de

consistent and predictable level of service. It includes measuring resource utilization, performing high volume stress tests, testing system break points and testing performance bottlenecks.


Introduction to Software Testing

System Testing Types(1)
? ? ? ? ? ?

Capacity Concurrence HW Config. Installation Interface Performance Recovery Reliability

- Limits on input variables as volumes - Simultaneous execution of multiple processing - System running on full range of permission - Installation procedures - Interface to existing system - Response time, throughput rates and total job run times - Restart/recovery procedures - Extended periods of normal processing

? ? ?

Resource Usage - Consumption of CPU and both main and secondary storage

Introduction to Software Testing

System Testing Types (2)
? ? ?


- Attempting to breach security mechanisms Adaptation to changing loads The range of permissible software system configuration Human factors aspects of the user interfaces including the user procedures

Sensitivity SW Config. Usability -



Introduction to Software Testing

Testing Principles

Introduction to Software Testing

Testing Principles
Errorcentric Testing ? Prioritize tests so that whenever you stop testing you have done the best testing in the time available ? Building Block Approach
? Unit

testing for technical validation errors ? Integration testing for test interfaces across system
? System

testing for test of business functionality


Experience Based & Formal Test Case Design

Introduction to Software Testing

Testing Principles

? Thoroughness

of tests related to the risks of failures ? Concentration on those parts which deliver the main benefits
? ?

? System

test cases design as Requirements are prepared

Independent Testers ? Test Automation


Introduction to Software Testing

Bug - Quality Improvement Opportunity ? Testing is an “ART” ? Testing is the Single Largest Activity in Development ? Automating Chaos Gives Faster Chaos ? We Don’t Have Time to Get it Right, We Are Too Busy ? Testing Drives the Project


Introduction to Software Testing

Introduction To Testing Techniques


Introduction to Software Testing

Test Case Design Approaches
Goal: Design a Necessary and Sufficient Set of Test Cases, To Ensure That All Functional Requirements & Non Functional Requirements Have Been Met!
? ? ? ?

Experience Based Combinatorial Machines Rigorous Algorithms Code -Based


Introduction to Software Testing

Types of Testing (Partial List)
? ? ? ? ? ? ? ? ? ? ?

Static Testing Dynamic Testing Black Box Testing White Box Testing Gray Box Testing Ad Hoc Testing Exploratory Testing Function Testing Non-Function Testing Regression Testing Stress Testing

Introduction to Software Testing

Static Testing
Testing a work product without running it

Static Analysis
• Should be done for all documents during the development of a system • Adherence to coding standards • Identification of anomalities ?Control flow ?Data flow ?Semantics • Executable/ Non-Executable code • Requirements versus code • Design versus code

• Walkthrough • Review • Inspection • Static analysis with the auxiliary of tools. It generates:

?Metrics (Complexity, size,…) ?Code/Data Cross references ?Call Trees ?Structure Charts


Introduction to Software Testing

What is Software Inspection ?
A type of Review, but we differentiate it as following: ? Walkthrough - Main purpose: understanding
? author guides the group through a document, so all understand the same thing, concensus on changes to make

? Review - Main purpose: decision making
? group discusses document and makes a decision about the content, e.g. how something should be done, go or no-go decision

? Inspection - Main purpose: find defects
? formal individual and group checking, using sources and standards, according to detailed and specific rules ? inspection is a well-defined process


Introduction to Software Testing

What can be Inspected ?

Anything written down can be inspected, i.e.

policy, business plans, marketing material, contracts

? system requirements, systems designs ? software code ? test plans, test designs, test cases, test results ? user manuals, procedures, training material


Introduction to Software Testing

The Inspection Process
Software Development Stage Change Request Process Improvement

Entry Kick off Ind Chk Meet Edit


Next Software Development Stage


Introduction to Software Testing

Documents Used in Inspection
Company Customer
Project related Process related


Check Lists


Source(s) (in writing) Inspection Product Log / Editing / Summary


Introduction to Software Testing

Inspection Effectiveness & Return on Investment

Most forms of testing are less than 30% efficient in finding errors or bugs The measured defect removal efficiency of both formal design inspections and formal code inspections is about 60% efficient, and according to Tom Gilb some of them reached 88%! The best defect removal efficiency level technique Mature Inspection - for 1 hour of inspection work you save 8 - 30 hrs of work


? ?


Introduction to Software Testing

If Inspections are so Good Why doesn’t Everyone Use them ?
? ? ?

Mainly “brain” work Almost no tools No company tries to “sell” inspections while there are many vendors selling testing tools Needs discipline Management issue

? ?


Introduction to Software Testing

Industrial Trends

Inspections become major engineering activity
? Motorola - Fagan inspections ? NASA - reduction of 90% rework


Testing to complement inspection and to allow quality certification Inspection can be introduced with payoff in first project Main issue: Cost (Inspection + Testing (new) < Cost (Testing (old))

? ?


Introduction to Software Testing

Dynamic Testing
Testing a program based on code ’s behavior while running code’

Black-Box Testing
• Program is viewed as a Black-Box •Test cases are derived directly from specifications, not from internal logic

White-Box Testing
• Program is viewed as a White- Box • Test cases are derived from internal logic not from specifications • Also called structured testing


Introduction to Software Testing

Kinds of Dynamic Testing

EXHAUSTIVE TESTING I.e. to test A = B+C if B and C are in the range -32767 to32768, 4,294,836,225 test cases are needed to do exhaustive testing!!!

? is ? does ? does


a poor technique not provide an optimal subset of test cases

not provide a high probability of detecting most errors ? perhaps, the most widely used technique

INTELLIGENT TESTING ? Develop test cases using Black-Box techniques ? Supplement these test cases using White-Box techniques

Introduction to Software Testing

Black-Box Testing Techniques
? Equivalence

Partitioning ? Boundary Value Analysis ? State Transitions ? Rigorous Algorithms ? Exploratory Testing


Introduction to Software Testing

Equivalence Partitioning
Partition the input domain into a finite number of equivalence classes ? Create test cases to invoke as many different conditions as possible

* It is based on the premise that if a test case in an equivalence class detects an error, all other test cases in the same class detect the same error

Introduction to Software Testing

Equivalence Partitioning
Step 1
? Identify

the input parameters ? Identify the allowable classes of input ? Select a representative value for each class
? Create

test cases to cover as many classes as possible


Step 2
? Select

invalid values for each class ? Create additional test cases to cover invalid classes

Step 3
? Identify ? Identify

the output parameters

the possible classes of output ? Create test cases to generate each possible output


Introduction to Software Testing

Equivalence Partitioning Example
External Condition 1. Input Hours a. Regular b. Over Time 2. Input Rate 3.5 to 13.5 3. Gross pay 10 to 2000 Valid Invalid Equivalence Equivalence Class Class 25 43 12 0 -1 2.1, -2.5 15.0, ax2 9, 2010

Boundary Value


Introduction to Software Testing

Boundary Value Analysis
Boundary Value Analysis technique tests conditions on, above and beneath the edges of input and output Equivalence Classes ? Test cases are created to test the edge of each Equivalence Class ? Test cases are created to test edges of both input and output classes


Introduction to Software Testing

Boundary Value Analysis Steps
For each equivalence class identified: a value on the class boundary ? Pick a value just under the boundary ? Pick a value just over the boundary
? Select


Introduction to Software Testing

Boundary Value AnalysisInvalid Example Valid
External Equivalence Equivalence Condition Class Class 1. Input Hours a. Regular 25 0 b. Over Time 43 -1 2. Input Rate 3.5 to 13.5 12 2.1, -2.5 15.0, ax2 3. Gross pay 10 to 2000 450 9, 2010 Boundary Value 40, 39.9, 40.1 59.9, 60, 60.1 3.5, 3.49, 3.51 13.5, 13.51, 13.49 10.0, 10.01, 9.99 2000, 2000.1, 1999

Introduction to Software Testing


Process: Create test cases to:
? Force

each transition ? Force each action

Introduction to Software Testing

Exploratory Testing
Some people can design test cases that will discover failures based on their experience ? Allow these people to write test cases ? Address complex areas ? Address changes ? Report failures, faults and errors so that experience level may grow


Introduction to Software Testing

White-Box Testing Techniques

Control Flow
? ? ? ? ? ?

Statement Coverage Decision (Branch) Coverage Condition Coverage Decision/Condition Coverage Multiple Condition Coverage Loop Coverage All Definitions All Predicates Uses All Computation Uses All P-uses/some C-uses All Uses


Data Flow
? ? ? ? ?

Introduction to Software Testing

White-Box Testing

? To

write test cases such that every statement is executed at least once ? It is a weak criteria ? It will not test the validity of logical operators

? Decision


Coverage (also called Branch Coverage) states that test cases must be written such that each decision has a True and False outcome at least once ? Decision Coverage emphasizes that each branch direction must be executed at least once ? It usually satisfies Statement Coverage


Introduction to Software Testing

White-Box Testing
? Condition

Coverage technique states that test cases must be written such that every condition in a decision takes all possible outcomes at least once ? Condition Coverage technique may not cover all decision outcomes

? Decision/Condition

coverage technique states that each condition should take on all possible outcomes at least once and each decision must take on all possible outcomes at least once


? Multiple

Condition coverage technique states that test cases must be written such that all possible combinations of conditions in each decision are taken at least once

Introduction to Software Testing

White-Box Testing


? Loop

coverage technique states that test cases must be written to test the loop counters

Process: • Treat the loop counter as an equivalence class • Test the boundary conditions


Introduction to Software Testing

Testing Automation

Introduction to Software Testing

What’ s Wrong with Manual Testing ?

Covers functionality only
* * * * * *

at human speed major investment in process and not in errors detection limited by resources availability and not necessity inexact repetition of tests inaccurate results checking difficult regression testing means NO regression testing POOR QUALITY SYSTEMS COSTLY MAINTENANCE

Introduction to Software Testing

GUI Test Automation Techniques
? Capture/Replay ? Test Programming ? Data Driven


Introduction to Software Testing

Computer Aided Software Testing Tools

Introduction to Software Testing

Categories of CAST Tools
CAST Tools CAST Tools

Static Static Testing Testing
Static Static Analysis Analysis Review Review Support Support Requirements Requirements

Dynamic Dynamic Testing Testing

Test Test Management Management Results Results Checking Checking
Comparator Comparator
Test Planning Test Planning

Test Test Preparation Preparation
Test Design Test Design Data Data Preparation Preparation

Test Test Running Running
Test Harness Test Harness Capture Capture Replay Replay Simulator Simulator

Dynamic Dynamic Analysis Analysis
Debug Debug

Testware Testware Management Management Metrics Metrics

Coverage Coverage Error Tracking Error Tracking Performance Performance


Introduction to Software Testing


Introduction to Software Testing

The Integrated View
Requirements Management

Visual Modeling

Development Tools


Manual & Automated Software Testing

Software Configuration Management 55

Introduction to Software Testing



To top