Docstoc

Testing Principles and Concepts - SISQA

Document Sample
Testing Principles and Concepts - SISQA Powered By Docstoc
					CSTE Training
          by
 Mark B. Curtiss, CSTE




              Skill Category 1 :
            Testing Principles and
                  Concepts
                Topic Review
        Test Principles and Concepts

   Definition of Test Specification
   Testing Techniques
   Testing Methods
   Independent Testing
   Commercial Off The Shelf (COTS) Software
   Testing Code Developed Under Outside Contract
   Test Quality
   Testing Life Cycle
   Vocabulary
                Topics Covered
         Test Principles and Concepts

   Definition of Test Specifications
    –   Test Objective
    –   Purpose
    –   Approaches
    –   Pass/Fail Criteria
    –   Entrance/Exit Criteria
   Vocabulary
Testing Life Cycle
     Test Questions to be Covered

   What is COTS?

   List five of the phases in a typical testing life
    cycle
            Testing Life Cycle

   Risk Analysis
   Planning Process
   Test Design
   Performing Test
   Defect Tracking and Management
   Quantitative Measurement
   Test Reporting
                Testing Life Cycle
                     (Risk Analysis)

   Risk is a major driver in testing
   In risk analysis, consider these components:
    –   Is this event positive or negative?
    –   The probability that the negative event will occur
    –   The potential gain/loss or impact associated with
        the event
                  Testing Life Cycle
                        (Risk Analysis)

   There are two phases to Risk Analysis
    –   Risk Identification
            Software Risk
            Testing Risk
            Premature Release Risk
            Business Risk
            Risk Methods
    –   Risk Management
            Risk Magnitude
            Risk Reduction Methods
            Contingency Planning
               Testing Life Cycle
        (Risk Analysis – Risk Identification)

   Software Risk
    –   Knowledge of the most common risks associated
        with software development, and the platform you
        are working on
   Examples
    –   Development time vs. testing time
    –   Security holes in Windows
    –   “Scope Creep”
                Testing Life Cycle
        (Risk Analysis – Risk Identification)

   Testing Risk
    –   Knowledgeable of the most common risks
        associated with software testing for the platform
        you are working on, tools being used, and test
        methods being applied
   Examples
    –   Budget
    –   Test environment
    –   New technology
    –   Number of qualified test resources
                Testing Life Cycle
        (Risk Analysis – Risk Identification)

   Premature Release Risk
    –   Ability to determine the risk associated with
        releasing unsatisfactory or untested software
        products
   Examples
    –   Short testing cycle
    –   No expanded defect regression
    –   No tester training
                Testing Life Cycle
        (Risk Analysis – Risk Identification)

   Business Risks
    –   Most common risks associated with the business
        using the software
   Examples
    –   Health care (100% reliability)
    –   National Defense
                Testing Life Cycle
        (Risk Analysis – Risk Identification)

   Risk Methods
    –   Judgment and instinct (Personal experience)
    –   Dollars estimate (Cost of failure)
    –   Identification and weighting of risk attributes
    –   Software risk assessment packages
                  Testing Life Cycle
        (Risk Analysis – Risk Management)

   Risk Magnitude
    –   How likely is this event going occur?
    –   What is the cost to the project if this event occurs?
   Risk Reduction Methods
    –   Prior planning
    –   Avoidance?
   Contingency Planning
    –   Prepare risk reduction plans
    –   Change technology
    –   Manage event
             Testing Life Cycle
               (Planning Process)

   Pre-Planning Activities
   Test Planning
   Post-Planning Activities
               Testing Life Cycle
                 (Planning Process -
               Pre-Planning Activities)


   Success Criteria / Acceptance Criteria

   Test Objectives

   Assumptions

   Entrance Criteria / Exit Criteria
              Testing Life Cycle
                (Planning Process –
                   Test Planning)


   Test Plan
   Requirements / Traceability
   Estimating
   Scheduling
   Staffing
   Approach
   Test Check Procedures
                     Testing Life Cycle
                        (Planning Process -
                           Test Planning)


   Test Plan
    –   The deliverables to meet the test’s objectives; the
        activities to produce the test deliverables; and the
        schedule and resources to complete the activities.
            Test Objectives
            Entrance Criteria / Exit Criteria
            Schedule
            Test Approach
            What tested and what not tested
                    Testing Life Cycle
                       (Planning Process –
                          Test Planning)


   Requirements / Traceability
    –   Defines the tests needed and relates those tests
        to the requirements to be validated.
            Written or verbal
            Informal user requirements
            Organized by requirement or test case
                    Testing Life Cycle
                       (Planning Process -
                          Test Planning)


   Estimating
    –   Determines the amount of resources required to
        accomplish the planned activities.
            Personnel / Equipment / Space
            Time Frame (hours, days)
            Estimating tools (MS Project)
                     Testing Life Cycle
                        (Planning Process -
                           Test Planning)


   Scheduling
    –   Establishes milestones for completing the testing
        effort.
            Identification of risk at these points
            Major checkpoints, not day-to-day activities
            External testing efforts
            Avoid over-utilization of resources
                    Testing Life Cycle
                       (Planning Process -
                          Test Planning)


   Staffing
    –   Selecting the size and competency of staff
        needed to achieve the test plan objectives.
            Staffing based on budget and skills needed
            Experienced vs. new employees
                    Testing Life Cycle
                       (Planning Process -
                          Test Planning)


   Approach
    –   Methods, tools, and techniques used to
        accomplish test objectives.
            Test Automation
            Black Box vs. White Box
            Code Coverage Tool
            Top Down Testing vs. Information Flow
                    Testing Life Cycle
                       (Planning Process -
                          Test Planning)


   Test Check Procedures
    –   Set of procedures based on the test plan and test
        design, incorporating test cases that ensure that
        tests are performed correctly and completely.
            Checklist
            Milestone Review
            Defect Tracking Analysis
            Traceability Matrix
                  Testing Life Cycle
                   (Planning Process -
                 Post-Planning Activities)


   Check Management
    –   Modifies and controls the plan in relationship to
        actual progress and scope of the system
        development.


   Versioning / Change Control / Change
    Management / Configuration Management
    –   Methods to control, monitor, and achieve change.
                Testing Life Cycle
                         (Test Design)

   Design Preparation
    –   Test Bed / Test Lab
    –   Test Coverage
   Design Execution
    –   Specifications
    –   Cases
    –   Scripts
    –   Data
                   Testing Life Cycle
                          (Test Design)

   Design Preparation
    –   Test Bed / Test Lab
            Adaptation or development of the approach to be used
             for test design and test execution.
    –   Test Coverage
            Adaptation of the coverage objectives in the test plan to
             specific system components.
                   Testing Life Cycle
                          (Test Design)

   Design Execution
    –   Specifications
            Creation of test design requirements, including purpose,
             preparation and usage.
    –   Cases
            Development of test objectives, including techniques
             and approaches for validation of the product.
             Determination of the expected result for each test case.
                   Testing Life Cycle
                          (Test Design)

   Design Execution
    –   Scripts
            Documentation of the steps to be performed in testing,
             focusing on the purpose and preparation of procedures;
             emphasizing entrance and exit criteria.
    –   Data
            Development of test inputs, use of data generation tools.
             Determination of the data set or sub-set needed to
             ensure a comprehensive test of the system. The ability
             to determine data that suits boundary value analysis and
             stress testing requirements.
                  Testing Life Cycle
                      (Performing Test)

   Execute Tests
    –   Perform the activities necessary to execute tests in
        accordance with the test plan and test design (including
        setting up tests, preparing databases, obtaining technical
        support, and scheduling resources).
   Compare Actual vs. Expected Results
    –   Determine if the actual results met expectations (note:
        comparisons may be automated).
   Test Log
    –   Logging tests in a desired form. This includes incidents not
        related to testing, but still stopping testing from occurring.
   Record Discrepancies
    –   Documenting defects as they happen including supporting
        evidence.
               Testing Life Cycle
        (Defect Tracking and Management)

   Defect Tracking
    –   Defect Recording
    –   Defect Reporting
    –   Defect Tracking
   Testing Defect Management
    –   Validation
    –   Regression Testing
    –   Verification
             Testing Life Cycle
          (Quantitative Measurement)

   Test Completion Criteria

   Test Metrics

   Management By Fact
                   Testing Life Cycle
               (Quantitative Measurement)

   Test Completion Criteria
    –   Code Coverage
            Purpose, methods, and test coverage tools used for
             monitoring the execution of software and reporting on
             the degree of coverage at the statement, branch, or path
             level.
    –   Requirement Coverage
            Monitoring and reporting on the number of requirements
             exercised, and/or tested to be correctly implemented.
                   Testing Life Cycle
               (Quantitative Measurement)

   Test Metrics
    –   Metrics Unique to Test
            Includes metrics such as Defect Removal Efficiency,
             Defect Density, and Mean Time to Last Failure.
    –   Complexity Measurements
            Quantitative values accumulated by a predetermined
             method, which measure the complexity of a software
             product.
                   Testing Life Cycle
               (Quantitative Measurement)

   Test Metrics
    –   Size Measurements
            Methods primarily developed for measuring the software
             size of information systems, such as lines of code,
             function points, and tokens. These are also effective in
             measuring software testing productivity.
    –   Defect Measurements
            Values associated with numbers or types of defects,
             usually related to system size, such as 'defects/1000
             lines of code' or 'defects/100 function points'.
                    Testing Life Cycle
               (Quantitative Measurement)

   Test Metrics (cont’d)
    –   Product Measures
            Measures of a product’s attributes such as performance,
             reliability, failure, usability
              –   Performance Benchmarking
              –   Usability Evaluation
              –   Component Failure Rate
                  Testing Life Cycle
               (Quantitative Measurement)

   Management By Fact
    –   Management by fact is using quantitative
        measures and metrics to manage the planning,
        execution, and reporting of software testing.
            Test Case Executed per Hour
            Test Cases per Defect
            Testing Hours by Defect
            Access Time
            Hits per Minute
                     Q&A

   What are your questions about the Software
    Testing Lifecycle?
                     Test Answers

   What is COTS?
    –   Commercial Off-The-Shelf (shrink-wrap)
   List five of the phases in a typical testing life cycle
    –   Risk Analysis
    –   Planning Process
    –   Test Design
    –   Performing Test
    –   Defect Tracking and Management
    –   Quantitative Measurement
    –   Test Reporting

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:6
posted:8/24/2011
language:English
pages:39