Carnegie Mellon University by sofiaie

VIEWS: 9 PAGES: 23

									Carnegie Mellon University                    CMU-MSE-KRYPTO-SQA Plan–1.0
School of Computer Science                                    Spring 2001
Master of Software Engineering




                                                    Krypto Team


                                 Software Quality Assurance Plan
                                                           Version 0.1
                                                          April 5, 2001



                                                            Pisey Huy
                                                         Grace Lewis
                                                        Ming-hsun Liu
Document Information

Title                   Software Quality Assurance Plan
Author(s)               Pisey Huy
Reviewer(s)             All Team Members, Erin Harper (Spring 2001)
Team name               Krypto
Team members            Pisey Huy, Ming-hsun Liu, Grace Lewis
Project mentors         Cliff Huff, Lynn Carter
Editor                  Erin Harper (Spring 2001)
Type of report          Managerial
Software used           Microsoft Word 2000
Templates used          University of Florida Software Quality Assurance Plan
Style guide             Software Engineering Institute Style Guide




Krypto  SQA Plan version 1.0                                                   2
Document Revisions
Rev.   Date           Author(s)   Comment




Krypto  SQA Plan version 1.0               3
Document Approvals
The following signatures are required for approval of this document. Note, however, that since the MSE
Studio role assignments will change during the course of the project, the respective names should be
updated as needed.



___________________________________________                           ______________________
Ming-hsun Liu, CMU MSE Program                                        Date
Krypto Team Lead and Process Manager


___________________________________________                           ______________________
Pisey Huy, CMU MSE Program                                            Date
Quality Assurance and Planning Manager


___________________________________________                           ______________________
Grace Lewis, CMU MSE Program                                          Date
Development and Support Manager


___________________________________________                           ______________________
Cliff Huff, SEI Technical Staff                                       Date
Studio Mentor


___________________________________________                           ______________________
Lynn Carter, SEI Technical Staff                                      Date
Studio Mentor




Krypto  SQA Plan version 1.0                                                                            4
Table of Contents
Document Information ................................................................................................................................. 2
Document Approvals .................................................................................................................................... 4
Table of Contents.......................................................................................................................................... 5
List of Figures ............................................................................................................................................... 6
List of Tables ................................................................................................................................................. 7
1    Acronyms .............................................................................................................................................. 8
2    Glossary ................................................................................................................................................. 9
3    Purpose .................................................................................................................................................10
4    Documentation .....................................................................................................................................11
5    Standards, Practices, and Conventions .............................................................................................12
6    Training ................................................................................................................................................13
7    Informal Walk-Throughs....................................................................................................................14
8    Inspection Process ...............................................................................................................................15
  8.1     Planning Phase ...............................................................................................................................15
  8.2     Preparation Phase ..........................................................................................................................16
  8.3     Inspection Meeting ........................................................................................................................17
  8.4     Follow-up ......................................................................................................................................18
9    Software Testing ..................................................................................................................................19
  9.1     Unit Test ........................................................................................................................................19
  9.2     System Test ...................................................................................................................................19
10      References ........................................................................................................................................20
Appendix A: Inspection Process References .............................................................................................21
  A1.0 Inspection Roles ...............................................................................................................................21
  A2.0 Rules for Inspections ........................................................................................................................21
  A3.0 Inspection Checklists ........................................................................................................................22
  A3.1 Implementation Inspection Checklist ...............................................................................................22




Krypto  SQA Plan version 1.0                                                                                                                                  5
List of Figures



Figure 1. Inspection Process ..........................................................................................................................15




Krypto  SQA Plan version 1.0                                                                                                                         6
List of Tables

Table 1. Governing Documentation ..............................................................................................................11
Table 2. Process Standards ............................................................................................................................12
Table 3. Inspection Phases ............................................................................................................................15
Table 4. Planning Phase ................................................................................................................................16
Table 5. Preparation Phase ............................................................................................................................16
Table 6. Inspection Meeting ..........................................................................................................................17
Table 7. Follow-up ........................................................................................................................................18




Krypto  SQA Plan version 1.0                                                                                                                              7
1 Acronyms

              Acronym           Definition
              IEEE              Institute of Electrical and Electronics Engineers
              NDBS              Netscape Database KeyStore
              PC                Project Coordinator
              QA                Quality Assurance
              SE                Software Engineer
              SEI               Software Engineering Institute
              SOW               Statement of Work
              SPMP              Software Process Management Plan
              SQA               Software Quality Assurance
              SRS               Software Requirements Specification
              STP               Software Task Plan
              TN                Technical Note




Krypto  SQA Plan version 1.0                                                       8
2 Glossary

              Term               Definition
              Author             The producer of the software element(s) that are inspected.
              Defect             Either a deviation from standards and specifications, a non-
                                 conformance to applicable standards, or an instance of an unmet
                                 specification
              Inspection         "A statistical analysis technique that relies on visual
                                 examination of development products to detect errors, violation
                                 of development standards and other problems." [IEEE90]
              Moderator          The chief planner and meeting manager of an inspection.
              Reader             During an inspection, the person who leads an inspection team
                                 through the software element(s) being inspected.
              Recorder           The person who records defects identified during an inspection.
              Software element   "A deliverable or in-process document." [IEEE90]




Krypto  SQA Plan version 1.0                                                                      9
3 Purpose
The purpose of this plan is to specify how software quality assurance (SQA) will be performed during the
software process. The QA manager will ensure that software development is performed according to the
process described in this SQA Plan and help to improve and refine the process. Following this process will
help to improve the overall quality of software products. This plan describes the SQA activities to be
performed and defines a set of standardized techniques for performing those activities.




Krypto  SQA Plan version 1.0                                                                          10
4 Documentation
The documents identified in Table 1 should be generated during the software process. The table specifies
both the inspection procedures that should be used to check a document for accuracy, and the criteria
against which the document will be checked.

   Documents                  Inspection                    Criteria
   SOW                        Document Review               SOW Checklist
   SPMP                       Document Review               SPMP Checklist
   SRS                        Document Review               SRS Checklist
   Design Document            Document                      Design Review Checklist, Design
                              Review/Inspection             Standards, Software Architecture
                                                            (Rational Rose Models)
   Software Architecture      Document Review               SRS
   SEI TN                     Document                      TN Checklist, Design Document
                              Review/Inspection

                               Table 1. Governing Documentation




Krypto  SQA Plan version 1.0                                                                          11
5 Standards, Practices, and Conventions
The following table lists all standards, practices, and conventions to be used in the process and identifies by
the QA manager:


   Standard, Practice or Convention                 Inspection Type
   Design Standard                                  Design Inspection
   Coding Standard                                  Code Inspection
   Unit Test Standard                               Test Plan Inspection

                                     Table 2. Process Standards




Krypto  SQA Plan version 1.0                                                                               12
6 Training
The QA manager will train the team at the beginning of the first project cycle. This training will consist of
a lecture that describes SQA activities and an inspection simulation. It will also include a review of the
inspection procedures.




Krypto  SQA Plan version 1.0                                                                             13
7 Informal Walk-Throughs
The objective of the walk-through is to evaluate the product and eliminate as many defects, omissions, and
deviations from standards as possible, before proceeding to a formal inspection. A walk-through is an
informal procedure performed by a software engineer who is not the author of the product. All of the
review procedures for a walk-through may be done via e-mail.

The following actions occur during a walk-through:

       The author of the product or the lead software engineer sends the document to team members,
        mentors, and technical writer.
       The reviewers will review the product for compliance with standards and good practice.
       The reviewers will report any defects found to the author.
       The author will correct and resolve all noted defects.
       The author will notify QA manager when the walk-through is complete.




Krypto  SQA Plan version 1.0                                                                           14
8 Inspection Process
Inspections are used to increase software quality and improve the productivity and manageability of the
development process. The inspection process follows a specific set of steps, which define what will be
inspected, when it will be inspected, who will do the inspection, what data will be collected, and what
follow-up actions must be taken.

Inspections are conducted for Rational Rose models and code. These inspections may be done
incrementally as development progresses and need not be delayed until the product is finished. The
inspection process may identify reworks at the planning, preparation, or inspection step. The overall
inspection process is as follows:




      Planning                   Preparation                  Inspection                   Follow-Up




                                                  Rework




                                    Figure 1. Inspection Process

The purpose of the inspection process is to formally review the product and identify defects. The author of
the product to be inspected should notify the team lead that the product is complete and ready for
inspection.


   Actions                    Responsibility
   Planning                   Author/Moderator
   Preparation                Inspectors
   Inspection                 Inspectors
   Follow-up                  Moderator/Author

                                     Table 3. Inspection Phases

The appropriate inspection checklist (Appendix A) will be used to validate inspections. The moderator will
certify that all necessary actions of the inspection are complete. The inspection report and combined defect
log are created and filed in the Visual SourceSafe as a result of the inspection process.



8.1    Planning Phase
The purpose of the planning phase is to ensure that the product to be inspected is properly prepared. The
author should assemble all software elements necessary for the inspection in Visual SourceSafe to allow
easy access for the entire inspection team. The team lead will then notify the team members that the



Krypto  SQA Plan version 1.0                                                                               15
product is ready for inspection. Notification may be in the form of an E-mail message. Minimum criteria,
if applicable, shall include:

          Check of spelling and grammar
          Check for compliance with standards (see Section 2.0)
          For code inspection, check to ensure inspection item is line numbered for easy reference by
           inspectors.
          Check to ensure code compiles cleanly

      Actions                                                            Responsibility
      Assemble items to be inspected                                     Author
      Select moderator from SQA group                                    QA Manager
      Review inspection items for completeness                           Moderator
      Select and notify inspection team                                  QA Manager
      Assign roles                                                       QA Manager
      Schedule inspection                                                Moderator


                                        Table 4. Planning Phase
Note:
To keep everyone informed, it is strongly recommended that all of the team members participate in each
inspection.

The moderator will certify that the product to be inspected is complete, the team is selected and notified,
and all scheduling actions are accomplished. The outputs for this phase include the assembled product to be
inspected, the defect log form, and the appropriate inspection checklist (Appendix A), if applicable.



8.2       Preparation Phase
The purpose of the preparation phase is to ensure that the team conducting the inspection is properly
prepared. Each member of the inspection team should be provided access to the product to be inspected.

      Actions                                                            Responsibility
      Review all items to be inspected                                   Inspectors
      Record identified defects on individual defect log                 Inspectors
      Record inspection preparation times on individual defect log       Inspectors
      Send copy of individual defect log to moderator 48 hours prior
                                                                         Inspectors
      to inspection meeting
      One day prior to the inspection, notify any inspectors who
                                                                         Moderator
      failed to provide a Defect Log*
      Combine all individual defect logs in a logical order to present
                                                                         Moderator
      during the inspection meeting **
      Disperse the combined defect log to all inspection participants
                                                                         Moderator
      for review 24 hours prior to the meeting

                                      Table 5. Preparation Phase

*The moderator may reschedule the inspection to provide all participants an additional opportunity to
prepare adequately for the inspection.
**After reviewing the combined list of defects, the moderator will decide whether the inspection can be
completed in a reasonable time. If the moderator feels that the inspection could take excessive time (based
on the number of defects reported), the moderator may consult with the author in order to determine if the



Krypto  SQA Plan version 1.0                                                                            16
document or code is really ready to be inspected. If not, the author is responsible for making all changes
necessary to allow for an inspection to occur. As a result the scheduled inspection may be rescheduled.

Notes:
Each inspector should use the appropriate checklist (Appendix A) and standards (Section 2.0) to review the
inspection items. Defects identified by each inspector are documented on the combined defect log as a
result of the preparation phase.

If more than a couple spelling and/or grammatical mistakes are found, then the inspector may make a
general comment to the author to check spelling and grammar rather than trying to record each on the
defect log. A redlined copy of the inspection item may also be given to the author at the end of the
inspection to aid in defect location.


8.3     Inspection Meeting
The purpose of the inspection meeting is to formally review the product and the identified defects.

      Actions                                                           Responsibility
      Review rules for inspections (Appendix A)                         Moderator
      Confirm and record inspection preparation times                   Moderator
      Systematically review the product being inspected*                Reader
      Present and clarify defects identified                            Inspectors
      Note the status of each defect as agreed upon by the inspection
                                                                        Recorder
      team and categorize valid defects
      Determine results of the inspection                               Moderator
      Determine estimated rework effort and completion date             Author
      Notify team members of inspection results                         Moderator
      Review rules for inspections (Appendix A)                         Moderator


                                     Table 6. Inspection Meeting
*A systematic review normally involves checking each identified defect. The moderator may choose, if
appropriate, to read the product line by line.

The moderator, with input from the team members, will determine the inspection disposition (accepted,
conditional, or re-inspect, as defined below). The team lead must approve all re-inspections and rework
efforts recommended by the inspection team. The combined list of defects will be created as a result of the
inspection meeting.


Accepted - The product is complete, without any further verification of rework. No defects were found that
caused the product to deviate from its specifications. The product may contain a very few trivial defects
that can be left to the author to correct.

Conditional - The product is conditionally accepted, with verification of the rework by the moderator. In
this case there are some major defects, but they are few, and rework is not expected to create any
substantial changes in the major design of the product.

Re-inspect - The author's rework must be reinspected. The moderator will inform the team lead if a re-
inspection is required. This disposition requires the moderator, the author, and at least one other member of
the team to examine the rework in a reiteration of the inspection meeting. For this case, there are either a
substantial number of major defects or rework that will change the original design of the product. The re-
inspection may or may not comprise the entire original product. The moderator will determine if the




Krypto  SQA Plan version 1.0                                                                                17
product needs to be re-inspected in its entirety. In this case the inspection process for this product will
commence at the beginning.



8.4     Follow-up
The purpose of the follow-up is to ensure the inspection results and decisions are properly recorded and
resolved. All inspectors must agree that the list of defects is accurate and complete. The inspection meeting
can then be concluded.

      Actions                                                             Responsibility
      Correct or resolve all noted defects.                               Author
      Validate corrections made by author (conditional disposition).      Moderator
      Complete the inspection report.                                     Moderator
      Post the inspection report to the appropriate directory in Visual
                                                                          Moderator/Recorder
      SourceSafe (configuration management tool).
      Record metrics in the appropriate section of the process
                                                                          Moderator
      checklist.


                                            Table 7. Follow-up

The moderator will certify correction of all defects (conditional disposition) and completion of all required
forms. The inspection report will be created as a result of the follow-up.




Krypto  SQA Plan version 1.0                                                                                 18
9 Software Testing
A Software Test Plan (STP) will be written to satisfy the requirements found in the Software Requirement
Specification. The STP may contain separate sections for each Software Sub-Section (SSS). The plan will
provide an overview of the test activities, schedules and resources required to perform testing. The plan
will describe how the testing specifications found in the following sections will be implemented.



9.1    Unit Test
All code will be unit tested to ensure that the individual unit (class) performs the required functions and
outputs the proper results and data. Proper results will be determined by using the design limits of the
calling (client) function as specified in the design specification which defines the called (server) function.
Unit testing is typically white box testing and may require the use of software stubs and symbolic
debuggers. This testing will help ensure proper operation of a module because tests are generated with
knowledge of the internal workings of the module.


9.2    System Test
System testing will begin when sufficient hardware and software have been integrated that enable the
operation and functions of the NDBS 2.0. The purpose of system testing is to validate and verify the system
in order to assure a quality product. This is the responsibility of the Krypto team. The QA manager will
ensure that the system tests have been run and the results recorded as specified in the STP. A team member
will ensure that the results of the system tests are addressed if the results show that the functionality does
not meet requirements identified in the Software Requirement Specification. System testing is actually a
series of different tests intended to fully exercise the computer-based system. Each test may have a
different purpose, but all work to expose system limitations. System testing will follow formal test
procedures based on hardware, software and science requirements as specified in the STP.

The Krypto Team must finish the system test before delivering NDBS 2.0 to the client on August 3,
2001.




Krypto  SQA Plan version 1.0                                                                                19
10 References
[1]    IEEE Std 610.12-1990, Standard Glossary of Software Engineering Terminology (ANSI)

[2]    Wiegers, Karl. “Improving Quality with Software Inspections,” Software Development, April
       1995.

[3]    Freedman, Daniel and Gerald Weinberg, “Handbook of Walkthroughs, Inspections, and Technical
       Reviews,” IEEE Transactions on Software Engineering, 1983




Krypto  SQA Plan version 1.0                                                                      20
Appendix A: Inspection Process References
This appendix provides additional information and checklists to be used in support of the inspection
process. The included checklists may be tailored to fit specific projects. Blank inspection forms are
available from Visual SourceSafe. To be able to audit the effectiveness of the inspection process, the
following data should be collected:

        Size of materials inspected
        Defects by severity, class, and type
        Preparation times prior to inspection
        Estimates of rework effort


A1.0 Inspection Roles

The following paragraphs define the roles and responsibilities of the various inspection participants. The
minimum roles required for an inspection are moderator/recorder, author/reader, and inspector (3
participants). The Krypto team may also get other members from other teams to inspect the product.

Author - The author’s role is to address specific questions that the reader is not able to answer, and to detect
defects based upon their special understanding of the product. The author must not serve as moderator or
recorder.

Moderator – The moderator ensures that the inspection procedures are followed, and that the other
inspectors perform their responsibilities for each step of the inspection process. The moderator may also
serve as the recorder and/or reader.

Reader - The reader leads the team through the product in a comprehensive and logical manner. The reader
should be prepared to describe the various parts and functions of the product.

Recorder - The role of the recorder is to record and classify all of the defects detected at the inspection
meeting, and to assist the moderator in preparing the inspection reports.

Inspector - All of the inspection team members will act as inspectors, including the author, regardless of
their other roles.


A2.0 Rules for Inspections

        The maximum time scheduled for an inspection meeting should be 2 hours.
        The meeting should be started on time and conducted so that it finishes on schedule.
        Arguments over style or technique should be deferred.
        Solutions or alternatives may be mentioned, but spending too much time on group rework should
         be avoided.
        Any useful references not already provided to team members should be brought to the inspection
         meeting, such as standards and reference documents.
        The product should be addressed directly, do not attack the author.




Krypto  SQA Plan version 1.0                                                                                 21
A3.0 Inspection Checklists

The following checklists are provided to aid in inspections. These checklists are not only useful to the
formal inspectors, but to the team members that must be both author and peer reviewer during walk-
throughs. The checklists are not intended to be all-inclusive, but are only a starting point for inspections.
Not all questions are pertinent to each inspection.

A3.1 Implementation Inspection Checklist

Support Material:

The following material should be provided to inspectors (if applicable):

        All source code pertaining to the inspection, including code listings before and after changes were
         made (in line-numbered listings) and difference files as appropriate
        All documentation pertaining to the inspection before and after changes were made (in line-
         numbered listings) and difference files as appropriate
        Standards and Guidelines referenced in Section 2.0

Inspection Issues:

        Do all documents and code conform to the applicable standards and guidelines, and if not is a
         variance provided?
        Are all products, including documentation, necessary to implement the system complete and
         correct?
        Has the team lead certified that unit testing was performed in accordance with standards?
        Have all items passed the walk-through process?
        Does the proposed change do what it is supposed to do? That is, is it correct?
        Does the proposed change ONLY do what is supposed to? That is, is it consolidated? Review
         potential for changed code to cause undesirable side effects (ways in which a change can cause
         affect existing code and documentation).
        Does the proposed change keep the code from looking patched? That is, is it clean? Take a
         generalized look at the new code in context, or review the new code against standards.

Variables:

        Are the variables properly defined and given meaningful, clear, and consistent names?
        Did all assigned variables have proper type consistency and casting?
        Are there any redundant or unused variables?
        Are all array references in bounds?

Loops and Branches:

        Are all loops, branches, and logic constructions complete, correct, and properly nested?
        Are all cases covered in an IF-THEN-ELSE or SWITCH block, including ELSE or DEFAULT
         clauses?
        Does each case in a SWITCH statement have a break?
        Does every switch statement have a default?
        Are loop termination conditions obvious and invariably achievable?
        Are indexes or subscripts properly initialized just prior to the loop or in the loop?




Krypto  SQA Plan version 1.0                                                                                   22
Defensive Programming:

       Are indexes, pointers, and subscripts tested against array, or file bounds?
       Are input arguments tested for validity and completeness?
       Are divisors tested for zero and noise?
       Are all output variables assigned?
       Are the correct data being operated on in each statement?
       Are files checked for existence before attempting to access them?
       Are cleanup routines used to make sure all files and devices are left in the correct state upon
        program termination?

Documentation:

       New Name: Are new data name, file name, and/or labels documented?
       Old Name Deleted: Are all documentation references to deleted names resolved?
       New Error Message: Are any new error messages included in their proper place in the
        documentation?
       Deleted Error Message: Are all references to deleted error messages deleted?
       Accepts New Data/Rejects Old Data: Are changes in input to the program communicated to
        operators and users?
       New Data Interpretation: Question a design, which changes the interpretation of existing, inputs to
        the system.
       Is the code clearly and adequately documented?
       Are there any inconsistencies between code and comments?
       Is the commenting style easy to maintain?

Checklists adapted from:

Improving Quality with Software Inspections
Handbook of Walkthroughs, Inspections, and Technical Reviews




Krypto  SQA Plan version 1.0                                                                             23

								
To top