Test Strategy by wuzhenguang

VIEWS: 15 PAGES: 32

									            Test Director User Guide


                                 Date Last Revised: 09/07/04
                                       Version #: 1.0




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0
                                                  Test Director User Guide



Table of Contents

TABLE OF CONTENTS ............................................................................................................................................ I

1.0 INTRODUCTION ................................................................................................................................................3
    1.1 OVERVIEW ..........................................................................................................................................................3
1.0 LOGIN/SECURITY STANDARDS ....................................................................................................................3
    1.1 SECURITY GROUPS ..............................................................................................................................................3
    2.2 TEST DIRECTOR ACCESS .....................................................................................................................................4
3.0 REQUIREMENTS ...............................................................................................................................................7

4.0 TEST PLANNING ................................................................................................................................................9
    4.1 TEST TREE ORGANIZATION .................................................................................................................................9
       4.1.1 Folder organization ...................................................................................................................................9
    4.2 TEST SCRIPT GUIDELINES ................................................................................................................................. 10
       4.2.1 Test Script Process Flow .......................................................................................................................... 10
       4.2.2 Test Script Standards (Non-Expert Execution / Future Reuse & Automation) ........................................ 11
       4.2.2.1 Test Script Content............................................................................................................................ 11
       4.2.2.2 Characteristics of a Good Test Case.............................................................................................. 12
       4.2.2.3 Test Step Examples ........................................................................................................................... 13
       4.2.3 Test Script Standards (Expert Execution / Limited Reuse) ....................................................................... 13
       4.2.3.1 Test Script Content............................................................................................................................ 13
    4.3 TEST PLAN TAB ................................................................................................................................................ 14
       4.3.1 Test Plan Tab - Overview ......................................................................................................................... 14
       4.3.2 Test Plan Tab – Creating a Test Case ...................................................................................................... 15
       4.3.3 Test Plan Tab – Associating a Test Case with a Requirement ................................................................. 17
       4.3.4 Test Plan Tab – Creating a Design Step .................................................................................................. 18
5.0 TEST LAB ........................................................................................................................................................... 19
    5.1 TEST SET ORGANIZATION ................................................................................................................................. 19
       5.1.1 Naming conventions ................................................................................................................................. 19
       5.1.2 Test Set Guidelines ................................................................................................................................... 20
       5.1.3 Content guidelines .................................................................................................................................... 20
    5.2 TEST LAB TAB .................................................................................................................................................. 21
       5.2.1 Test Lab Tab - Overview .......................................................................................................................... 21
       5.2.2 Test Lab Tab – Creating a Test Set .......................................................................................................... 22
       5.2.3 Test Lab Tab – Adding Test Cases to a Test Set....................................................................................... 23
       5.2.4 Test Lab Tab – Running Test Sets ............................................................................................................ 24
       5.2.5 Valid Status Options ................................................................................................................................. 25
6.0 DEFECT MANAGEMENT ............................................................................................................................... 26
    6.1 DEFECT MANAGEMENT PROCESS FLOW ........................................................................................................... 26
    6.2 DEFECT MANAGEMENT OVERVIEW .................................................................................................................. 26
    6.3 DEFECT WORKFLOW ......................................................................................................................................... 27
       6.3.1 Defect Status Options ............................................................................................................................... 27




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                                                                                         i
                                               Test Director User Guide


  6.4 DEFECT CLASSIFICATION .................................................................................................................................. 29
     6.4.1 Severity Options ....................................................................................................................................... 29
  6.5 DEFECT TAB ..................................................................................................................................................... 30
     6.5.1 Defect Tab – Creating a Defect................................................................................................................ 30
     6.5.2 Defect Tab – Required Fields................................................................................................................... 31
     6.5.3 Defect Tab – Additional Customized Fields ............................................................................................. 31




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                                                                                     ii
                                 Test Director User Guide




1.0 Introduction
1.1 Overview
The purpose of this document is to provide information for Test Director for projects for the
Release. Each tab within Test Director is addressed in this document and standards and
guidelines are defined for each.


1.0 Login/Security Standards
1.1 Security Groups
The standard Enterprise Test Director server user access groups are being used. The groups and
a description of each are listed below. _QA Tester is the primary group to which most testers
will be added. This is the most common group which provides add and modify access to each of
the modules within Test Director but not delete.

Test Director users will be placed in the appropriate group based on the type of test activities
they will be involved in.

      Group                                                            Rights
_Project Admin          The group members have full privileges in a Test Director project for all tabs. The group also has some
                        administration privileges.
_Project Manager        The group members have full privileges in the following Test Director modules: Requirements, Test Plan,
                        Test Lab and Defects.
_QA Tester              The group members have full privileges in the following Test Director modules: Requirements, Test Plan,
                        Test Lab and Defects. The group members do have delete access for any modules.
_Developer              The group members privileges are limited to modifying the attachments in the following modules:
                        Requirements, Test Plan and Test Lab. In the Defects module, the group can only add and modify defects,
                        but not delete them.
_Viewer                 The group members have read-only privileges in a Test Director project, and can only change their own
                        passwords and properties.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0
                                 Test Director User Guide



2.2 Test Director Access
To access the Project in Test Director, enter the following URL:
    http://enterprisetd.bankone.net/tdbin/ and click the Test Director link on the General
       Access page




                                                                       Please note: project users
                                                                       Please note: May Cert Release
                                                                       must request access to Test
                                                                       project users must request
                                                                       Director from the project
                                                                       access to Test Director from the
                                                                       administrator.
                                                                       May Cert administrator:
                                                                       • Bill Stroud




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                  4
                                 Test Director User Guide


Once you have been granted access by the administrator, the User ID is the same as your email
address before the @ symbol and the password is blank.
NOTE FOR NEW USERS ON ENTERPRISE TD SERVER: Before logging into Test
Director, the password must be changed. To change your password, click on customize in the
upper right corner, a Login box will appear. Enter your User ID and select OK. This will display
the project customization screen where you can change your password.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                5
                                 Test Director User Guide


Click on the Change Password link, and leave the old password field blank and enter your new
password and click OK. Select LOGOUT to return to the Test Director General Access page.




The project can now be accessed within Test Director.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                             6
                                 Test Director User Guide




  To access the ER105
  project in Test
  Director, select
  ER105_Domain as
  the Domain and
  ER105_Project as the
  project. Enter the
  UserID and password
  you created.




3.0 Requirements
The Requirements module of Test Director is designed to store requirements of a project. It is
not a full-blown requirements tool that allows for version tracking of business requirements,
functional and technical specifications. All requirements must be loaded into Test Director and
covered by Test Cases within Test Director.

The requirements can be organized into parent and child requirements within Test Director.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                7
                                 Test Director User Guide




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                              8
                                 Test Director User Guide




4.0 Test Planning
4.1 Test Tree Organization
It is important to organize the test tree logically so that users/testers can easily identify the
purpose of the scripts.

   A well organized test subject tree will:
       –Allow tests to be located quickly;
       –Allow new resources to quickly become familiar with existing tests
       –Reduce resource contention among Test Director users.

   Provide descriptions to all subject tree folders allowing others to better understand the folders
    and structure in use.

4.1.1 Folder organization
Subject folder level set-up (Test Cases will need to be created under the following folders).
Subsequent levels can be broken down as necessary by functional area, test objective, screen,
application, etc… The lowest levels in the Test Plan Subject Tree are the actual Test Cases.
Additional high-level folders will be added as needed for additional LOB/groups.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                      9
                                               Test Director User Guide




4.2 Test Script Guidelines
Just as it is important for the test tree structure to be easy to understand, it is important that other
users be able to easily find what they need in the test scripts. The scripts should be set up/written
in such a way that any tester, regardless of knowledge of the application, can execute the tests.
Do not assume that the tester has any knowledge of the application and therefore needs specific
instruction and exact details on what to click on next.


4.2.1 Test Script Process Flow



                                          Script Management Process Flow


    ROLE                                                                        ACTIVITIES




   LOB/IT                                                                      Document Test Cases/                                             Add/Update Test Cases/
                                                                                                                      Execute Test Sets
  PROJECT                                                                       Sets in Test Director                                                   Sets
   TEAM

                                                                           Document all required Test Cases                                 Defects found without associated
                                                                           Link Test Cases to Requirements                                   test cases require the addition of
                                                                           Combine Test Cases into Test Sets                                 new test cases




                                                Collect Test Case/Set           Track Status of Test
                 Conduct Test Scripting
                                                Count Estimates from           Case/Set Entry in Test
                   Overview Meeting
                                                        Project                       Director


                 Review Test Scripting
                                                                           Requirements Coverage Reports
                  Guidelines & Criteria
                                                                           Test Case/Set Completion % Reports
     ITE         Review Test Scripting
                                                                           Test Case/Set Scorecard
   CENTRAL        Timeline & Process
                 Demonstrate Use of
    TEAM
                  Mercury Test Director to
                  Enter Test Cases (Scripts)
                  and Test Sets including
                  Test Case/Set Tree                                            Conduct Script Quality
                  Structure                                                       Review Process



                                                                           Complete Test Script Assessment
                                                                            Checklist
                                                                           Execute Reviews of Subset of Test Cases




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                                                                                               10
                                     Test Director User Guide



4.2.2 Test Script Standards (Non-Expert Execution / Future Reuse & Automation)
Test Scripts which will be executed by a non-expert user of the application during testing or will
be automated should be written to the guidelines/criteria defined below. Test Scripts which will
be leveraged going forward for future project and regression testing are also encouraged to follow
the guidelines/criteria defined below.

4.2.2.1 Test Script Content

Scripts are written based upon the assumption that the author will not be performing the test, and
that a person not familiar with the application or process will be executing the script. Test result
documentation must verify the expected results were obtained.
  Test Case Creation Standards                                  Benefits of a Good Test Case

  • Write the script for someone not familiar with the          • Decreased Cost
    application or business process                                   – Less time required to execute test cases
  • Each step of the script should have an expected result            – Less time fixing/augmenting cases during
  • Each script should have a clear and concise purpose.                 execution
  • Each script is to test one condition single objective)            – Easier maintenance of cases
    only. Attachments are acceptable, however the                     – Improved cycle times
    attachment may not contain multiple scripts for various     • Increased ability to estimate
    scenarios or conditions.     It is not acceptable to
    attach or refer to an application‟s B1B development               – Consistency and standards across all
    scripts within one business script                                   applications
  • Test results must be stored in Test Director                • Decreased project risk
  • Test result documentation must provide proof of the               – Improved coverage
    steps of the Test Case passed. The amount of detail               – Increased ability to pinpoint testing effort
    required is based upon an assumption that the               • Common Structure
    documentation will be reviewed by someone unfamiliar
                                                                      – Cross-training opportunities
    with the process and that the documentation will be
    reviewed after the testing phase is complete.                     – Improved resource allocation resulting in
                                                                         decreases of peaks and valleys
  • Each Test Step within a Test Case must be marked                  – Aids in ability to automate
    with a Pass/Fail status
  • Each Test Director Requirement must be linked
    to at least one Test Case



The following content should be included in the Test Director Test Cases:

  Test Case Content                                           Test Case Step Guidelines

  • Description Information                                   Each Test Script (Test Case within Test Director) will be
        – Prerequisites                                         made up of one to many Test Steps. Each step
        – Assumptions/Dependencies                              should be created with the following guidelines in
                                                                mind:
        – Objective
        – Variables in the Script
                                                              • Keep number of steps within a test script to less than
        – Notes                                                 50 steps
        – Constraints                                         • Use the default step name within Test Director so
  • How to Navigate within the Application                      steps can be renumbered easily if needed
  • Test Case Steps with Expected Results                     • Limit the amount of details in one step to one user
  • Other Required Fields within Test Director                  action (i.e. enter information in a specific field, or open
    determined by your project                                  a specific screen)
  • Actual results                                            • Break down into small details to help testers that have
                                                                limited experience with the application
Created By: Kotti RavindraNath                                • Each step must be reportable with a status of pass or
Date Last Revised: 09/07/04                                     fail
Version #: 1.0                                                                                                                11
                                       Test Director User Guide


4.2.2.2 Characteristics of a Good Test Case


      • Test Single Objective
      • Clear and concise purpose
            – One to Two sentences
      • Length
            – Less than 50 steps; each test step should have similar timing
            – Consistent timing - aids in test planning
            – Promote forward progress - less likely to get lost and make mistakes
            – Results are easier to track
            – Decreased defect testing time
      • Ordered according to business scenarios/use (Priority)
      • High probability of finding a defect
      • Traced to requirement
      • Self-standing
            – Get the same result each time for each tester
            – Contains all information required to generate test
            – Does not require explanation or clarification
            – Do not assume the tester has utilized the application
            – When possible, avoid cumulative cases – those requiring the generation of dependent tests (need to run X test
                prior to generating next test)
                   • Only use when input is really needed from another test
                   • Use name and number when referring to other tests (numbers may change)




      • Concise Steps
            – Steps are written as clear inputs or tasks (could also include set of logically related inputs – i.e. login, password,
                company id, etc should be included on the same step)
            – Results not always required for each step (no system response no result)
      • Efficient
            – Does not contain unnecessary steps or rehash basics
            – Reusable
      • Maintainable
            – Need to be reused for maintenance release regression
      • Clear Pass/Fail Criteria
      • Simple Language
            – Be exact and consistent in wording
                  • Do not refer to the same thing different ways; common language
            – Naming of fields should not be generic and needs to match requirements
                  • Must be updated/reviewed for each release
            – Active case - do this, do that, system does this, navigate, compare
                  • Verb usage – most cases to begin with a verb (click, select, navigate)
                  • Clear description of who is performing the action – tester or system
      • Clear Descriptions
            – Allow a training opportunity




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                                                         12
                                                 Test Director User Guide


4.2.2.3 Test Step Examples

Detailed below are examples of expected results within test steps. The left column indicates an
unclear example and the right column indicates a better way of writing an expected result within
the test step:
        N o t C le a r                                                              G o o d E x a m p le
        V e r if y t h a t r e c o r d      is   s a ve d     / d e le t e d        V e r if y t h a t „x y z ‟ t a b le in “ a b c ” p a g e
                                                                                    r e f le c ts t h e n e w r e c o r d w it h f o llo w in g
                                                                                    v a lu e s :
        H o w    d o   w e    v e rify ?                                            A c c o u n t # : “”
                                                                                    T y p e : “”
                                                                                    S ta tu s : “”
                                                                                    E x p ir y D a t e : “ ”

        N o t C le a r                                                              G o o d E x a m p le
        V e r if y t h a t lis t b o x       c o n t a in s    f o u r v a lu e s   V e r if y t h a t lis t b o x   c o n t a in s   th e   lis t o f
        n o w .                                                                     f o llo w in g v a lu e s :
                                                                                    1 . a b c
                                                                                    2 . d e f
        W h a t a re      a ll th e      v a lu e s ?                               3 . g h i
                                                                                    4 . xyz

        N o t C le a r                                                              G o o d E x a m p le
        V e r if y t h a t e n t r y   is    n o t a c c e p te d                   V e r if y t h a t in v a lid v a lu e s in „a b c ‟ t e x t
                                                                                    f ie ld g iv e s t h e f o llo w in g e r r o r m e s s a g e :
                                                                                    “”
        T e s t s te p s s h o u ld c le a rly s ta te w h a t
        h a p p e n s o r w h a t m e s s a g e a p p e a rs .                      F u r t h e r V e r if y t h a t „x y z ‟ t a b le d is p la y s
                                                                                    t h e o ld r e c o r d s a n d n e w r e c o r d is n o t
                                                                                    a d d e d t o t h e t a b le

        N o t C le a r                                                              G o o d E x a m p le
        V e r if y t h a t a c c o u n t h o ld e r c a n m a k e              8    V e r if y t h a t a n e r r o r   m e s s a g e is
        t r a n s a c t io n s a t t h e m o s t p e r m o n t h                    d is p la y e d “ Y o u c a n      n o t . … … … … . . . ” if
                                                                                    a c c o u n t h o ld e r a t t e   m p ts t o m a k e t h e ir
        T   h e te s t s te p o r s c rip t s h o u ld                              9 t h t r a n s a c t io n in a     m o n th .
        e   x p lic itly s ta te w h a t c o n d itio n s                 m u s t
        b   e m e t fo r a n e rro r m e s s a g e to
        a   p p e a r.




4.2.3 Test Script Standards (Expert Execution / Limited Reuse)
Test Scripts which will be executed by an expert user skilled in the application can be written to
the guidelines/criteria defined below. Test Scripts which are project-specific and will not be
leveraged going forward for future project and regression testing can also follow these
guidelines/criteria.

4.2.3.1 Test Script Content

Scripts are written based upon the assumption that someone familiar with the application or
process will be executing the script. Test result documentation must verify the expected results
were obtained.
     Each Test Case should include at least one Test Step and have a clear and concise
        purpose
     Each Test Step of the Test Case should have an expected result
     Each Test Step of the Test Case should be marked with a Pass/Fail status
     Test results must be stored in Test Director
     Test result documentation must provide proof of the steps of the test passed
     Each Test Director Requirement must be linked to at least one Test Case




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                                                                           13
                                  Test Director User Guide


       Each Test Case should include a description of the method used to validate actual vs‟
        expected results

The following content should be included in the Test Director Test Cases:

    Test Case Content                                   Test Case Step Guidelines

    • Description Information                           Each Test Script (Test Case within Test Director) will be
      – Prerequisites                                     made up of at least one Test Step. Each step
      – Balancing Procedures (if applicable)              should be created with the following guidelines in
                                                          mind:
      – Upstream or downstream impact (if applicable)
    • Test Case Steps with Expected Results             • Use the default step name within Test Director so
    • Other Required Fields within Test Director          steps can be renumbered easily if needed
      determined by your project                        • Each step must be reportable with a status of pass or
                                                          fail
    • Actual results




4.3 Test Plan Tab
4.3.1 Test Plan Tab - Overview

Most of the test design activity takes place at the test level. When a test within a subject folder is
selected in the test plan tree view, the following test planning tabs are displayed:
         Details – Provides the context for the test through detailed description and
            identification information, such as test priority and the name of the designer or
            reviewer. Best place for providing test setup information.
         Design Steps – Defines all the steps in the test along with expected results and
            supporting detail for the test step. Each step becomes a test case.
         Test Script – Provides launch pad for executing test scripts. (Only used if test will be
            automated)
         Attachments – Provides ability to add or remove attachments at the test level.
         Reqs Coverage – Provides ability to associate a test with a requirement for the
            purpose of ensuring requirements coverage. All tests must be associated with a
            requirement.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                                      14
                                 Test Director User Guide




4.3.2 Test Plan Tab – Creating a Test Case

There are three required fields when creating a new Test Case within Test Director
        Assoc High-Lvl Task: – The task on the Test Execution schedule that the Test Case is
           associated with
        Owning Group/App: – The business group or application that will be responsible for
           managing the test case
        Project: – The primary Project that the Test Case is associated with




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                            15
                                 Test Director User Guide




There are additional fields that were added but are not required by the system when creating a
Test Case. Many of these values are used only within specific LOB‟s. Please refer to your test
coordinator if any of these fields should be used:
        X-Impact #1 & X-Impact #2: – Identifies cross-impacted LOB/applications
        Test ID#: – Display Only – Unique system generated numeric ID for a test case
        Batch: – Identifies is the test case is batch or not
        System Interface: - Identifies any interfacing systems for the test case execution
        Account Information: - Text field to enter any account information specific to the test
           case
        Business Function/Product: - Identifies the primary business function or product that
           the test case is associated with
        Disabled Function?: - Identifies any Retail CA disabled functions
        Automated Script Type: - Additional detail for QuickTest Pro and WinRunner
           automated test scripts



Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                              16
                                 Test Director User Guide


           Development Team: - Identifies the development team that the test case is associated
            with
           Profile: - Identifies the user profile of the executor of the test case
           Site: - Identifies the test site who will be responsible for the execution of the test case
           Test Paper Required: - Identifies is physical test paper will be required during the
            execution of the test case
           Tran Type: - Identifies the type of transaction being performed in the test case


4.3.3 Test Plan Tab – Associating a Test Case with a Requirement

All Test Cases need to be associated with at least one Requirement from the Requirements Tab
to ensure proper Requirements coverage. Press the „Select Req‟ button to associate one to many
requirements with a Test Case.




          The corresponding Requirement List pops up when
         the “Select Req” button is clicked. Drag and drop the
         associated Requirement from the list to the main body
         of the Requirement section.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                     17
                                  Test Director User Guide


4.3.4 Test Plan Tab – Creating a Design Step

Click on the Design Steps tab on the main Test Plan tab to view and create Design Steps for a
Test Case.




 o 1 – To create a Design Step, click on the above
 button




 o 2 – After the Design Step create button has
 been pressed, the Design Step Editor pop-up box     o 3 – Press the button above each time you
 appears                                             need to create a design step within a Test Case




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                         18
                                 Test Director User Guide




5.0 Test Lab
5.1 Test Set Organization
Test Sets should be organized, through the use of naming conventions, to easily identify what
type of scripts are included in the Test Set.

5.1.1 Naming conventions

A naming convention should be used for the test sets that indicates what level of test is being
executed within the test set and provides a brief description of the function being tested.
        o    ABBBCD + Short Description
            (Example: CACHT1 Balancing = Commercial ACH TST Cycle 1 Balancing)
                 A = Workgroup (O, C, F, I, S, N, R, L, G, B, P)
                        O = I&O
                        C = Commercial
                        F = Finance
                        I = IMG
                        S = Card Services
                        N = NEO
                        R = Retail
                        L = Retail Lending
                        G= CIG
                        B = B1B
                        P = PCS
                 BBB = Application or Business Function (Example: ACH).
                 C = Test Phase (T, I, U, D, P)
                        A = Unit Test
                        T = System Test
                        I = IST
                        U = UAT
                        C = Certification
                        D = Disaster Recovery Testing
                        P = Performance Testing
                 D = Cycle (1, 2, etc)
                        1 = Cycle 1
                        2 = Cycle 2
                        3 = Cycle 3
                 Short Description
                        Short Description of what is included in Test Set. (Example: Balancing)




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                    19
                                 Test Director User Guide


5.1.2 Test Set Guidelines

   Tests Sets are designed to be an execution run of several Tests Cases.

   The Test Cases should be related to each other in order to achieve a testing objective.

   Each Test Case in the Test Set should be testing an objective that can result in a pass or fail
    status.

   Currently there is no folder structure for Test Sets
       - The capability to group Test Sets into Folders will be available in Test Director 8.0.

   Group Test Sets by:
       - Test Phase;
       - Related functionality;
       - Related screens;
       - Types of Test:
                 Tests for correct navigation;
                 Tests for data verification;
                 Tests for display properties.

5.1.3 Content guidelines

If a test set is to be executed during a specific phase of testing, then it should only contain Test
Cases pertinent to that level. For example, including Unit tests in an Integration test set does not
make sense.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                   20
                                 Test Director User Guide



5.2 Test Lab Tab
5.2.1 Test Lab Tab - Overview

The Test Lab tab allows for the creation of test sets, scheduling of test runs, running tests
manually or automatically, and analyzing the test results. There are three sub-tabs within the
Test Lab Tab:
        Execution Flow – Displays a pictorial representation of the execution flow of the tests
          within the selected test set
        Test Set Properties – Allows you to set properties for the test set such as how
          notifications will be handled, what do on failure, etc.
        Execution Grid – Select the „Run‟ button or „Run Test Set‟ on the Test Lab tab.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                              21
                                 Test Director User Guide



5.2.2 Test Lab Tab – Creating a Test Set

There is one required field when creating a new Test Case within Test Director:
        Planned P/C: –The planned phase and cycle in which the test will be executed




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                          22
                                 Test Director User Guide



5.2.3 Test Lab Tab – Adding Test Cases to a Test Set

Test Sets are a grouping of related Test Cases. After creating a Test Set, Test Cases must be
added.

                                                      1 -To add a Test Case to a Test Set, click on the
                                                      „Select Test Tasks‟ button. The Test Plan Tree
                                                      will pop up with the Test Case list.




                                 2 – To add a Test Case to a Test Set from the
                                 Test Case list, click and drag the appropriate Test
                                 Case




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                            23
                                      Test Director User Guide



5.2.4 Test Lab Tab – Running Test Sets

Running Test Sets during actual Test Execution will walk the tester through each of the required
steps to compare actual and expected results. Select the „Run‟ button on the Test Lab Tab.




 The Manual Runner Test Set: <…..> pop-up box
 will appear. Choose the Exec Steps box to start
 executing the Test Set.




Once the „Exec Steps‟ button is selected, the following screen will display.




   Click on the „Bug‟ button to
   create a defect due to a
   failed test.




                                            After the test execution is complete, press the
                                           above button and you will be returned to the
                                           Execution Grid portion of the Test Lab tab.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                24
                                 Test Director User Guide



5.2.5 Valid Status Options

Valid status options for a test step include the following:

Passed – the actual results of the step match the expected results.

Failed – the actual results of the step do not match the expected results.

Failed – Pending Review – the actual results do not match the expected results and the tester is
requesting that the actual results be reviewed/verified by the Test Lead.

No Run – the step has not been executed yet.

Not Completed – the test was started, but the results could not be verified at the time of the run.

Not Executable – the step could not be run due to a variety of causes including but not limited
to: system unavailable, test data not ready, third party system not available.

Pending Batch – the step can‟t be verified completely until the batch cycle is completed.

Pending Validation – the step can‟t be verified completely until the validation is completed.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                  25
                                                                      Test Director User Guide




6.0 Defect Management
6.1 Defect Management Process Flow

                                                 D e fe ct M a n a g e m e n t P ro ce ss F lo w
       ROLE                                                                                                                 A C T IV IT IE S



                                                                                                                                                                                                                    C o n firm R e te s t b y D e fe c t
                                     Id e n tify a n d O p e n D e fe c t                                                                                                                                              O rig in a to r a n d C lo s e
                                                                                                                                                                                                                                 D e fe c t
     TESTER /
     DEFECT
   S U B M IT T E R              A ll d e fe c ts w ill b e d o c u m e n te d                                                                                                                           D e fe c ts th a t a re n o t s u c c e s s fu lly re te s te d
                                  w ith in th e to o l p rio r to b e in g                                                                                                                                 w ill b e re tu rn e d to "R e -O p e n e d " s ta tu s
                                  a s s ig n e d fo r w o rk                                                                                                                                               a n d re a s s ig n e d fo r fix
                                                                                                                                                                                                          C lo s e d d e fe c ts w ill b e u p d a te d to s ta tu s o f
                                                                                                                                                                                                           "C lo s e d " w ith th e a p p ro p ria te c lo s u re c o d e


                                    R e vie w O p e n D e fe c ts a n d
                                      P rio ritize a n d A s s ig n to
                                                D e ve lo p e r
    D E V /T E S T
       LEAD
                           D e fe c ts th a t d e te rm in e d to b e lo w p rio rity o r
                            d e fe rre d w ill b e u p d a te d in th e to o l b y T e s t L e a d
                           A s s ig n e d d e fe c ts w ill b e u p d a te d to s ta tu s
                            "A s s ig n e d /In -P ro g re s s " w ith a p la n n e d fix d a te




                                                                                                     R e c e ive A s s ig n e d D e fe c t
                                                                                                                                                               C o m p le te D e fe c t F ix
                                                                                                              a n d B e g in F ix

  DEVELO PER
                                                                                           D e ve lo p e r w ill b e g in a n la ys is a n d           D e ve lo p e r w ill fix a n d c o m p le te
                                                                                            fix                                                          in itia l te s tin g a n d u p d a te s ta tu s
                                                                                                                                                         to "C o d e C o m p le te " p rio r to
                                                                                                                                                         tu rn in g it to th e E n viro n m e n t
                                                                                                                                                         M ig ra tio n te a m


                                                                                                      D e m o te F ix to R e q u ire d
                                                                                                                                                            P ro m o te F ix to R e q u ire d
                                                                                                     D e ve lo p m e n t E n viro n m e n t
                                                                                                                                                           T e s t E n viro n m e n t to M a k e
                                                                                                         to M a k e A va ila b le fo r
                                                                                                                                                                A va ila b le fo r R e te s t
                                                                                                                  U p d a te
 E N V IR O N M E N T
  / M IG R A T IO N
                                                                                            D e fe c t w ill b e d e m o te d to th e                 D e fe c t w ill b e u p d a te d to s ta tu s o f
                                                                                             re q u ire d fix e n viro n m e n t b y th e               "R e a d y fo r R e -T e s t" a fte r a ll c o d e
                                                                                             E n viro n m e n t/M ig ra tio n te a m o r                c h a n g e s h a ve b e e n re p ro m o te d to th e
                                                                                             th e d e ve lo p e r d e p e n d in g o n th e             re q u ire d te s t e n viro n m e n t (if
                                                                                             p h a s e o f te s tin g (if a p p lic a b le )            a p p lic a b le )




6.2 Defect Management Overview

A defect should ALWAYS be opened when a test is failed. If a defect is found and there is
not a corresponding test to associate with the defect, create a new test to associate with the
defect. Associating a failed test with a defect provides assurance that the new and existing
software is being effectively tested. It also ensures that the tests created are being properly
tracked and managed. Please be sure to always reference the following related information in the
description portion of the defect:
             Test Set
             Test
             Run
             Step, Description
             Expected Result
             Actual Result.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                                                                                                                                                                                               26
                                 Test Director User Guide



6.3 Defect Workflow
6.3.1 Defect Status Options

    Status                       Description
    New or Open                  Defect has been identified and created but has not yet been
                                 evaluated for assignment
    Assigned/In-                 Defect has been evaluated and assigned to a developer for
    Progress                     resolution
    Code Complete                Defect has been fixed and retested by developer and is ready
                                 to be migrated to appropriate test environment
    Ready for Re-Test            Defect fix has been migrated to appropriate test environment
                                 and is ready to be retested by defect submitter
    Closed                       Defect has been fixed and successfully retested by the
                                 original defect submitter.
    Re-Opened                    Defect has been previously closed in the current level of
                                 testing but the same problem has appeared again




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                  27
                                    Test Director User Guide




                                              Test Fails



                                              New/Open
                                                                  Assign to IT Resource


                                        Assigned/In-Progress
                                                                   IT Resource Begins Fix


                                                                                          Re-Opened


                                                                  IT Resource Completes Fix

                                         Code Complete


                                                                  Migration Complete

                                              Ready for
                                               Re-Test
                                                                  Originator re-executes
                      If fixed                                       If not fixed



                       Closed

                                                           If problem is rediscovered within
                                                                the same level of testing
    Defects that are determined to be
‘Deferred’ due to low priority, ‘No Issue’
  due to working as designed or training
issue, or ‘Duplicate’ will be ‘Closed’ with
       the appropriate closure code


   Created By: Kotti RavindraNath
   Date Last Revised: 09/07/04
   Version #: 1.0                                                                              28
                                 Test Director User Guide



6.4 Defect Classification
6.4.1 Severity Options

      Severity levels are focused on the impact of the defect that has been found from an
      application functionality perspective.

        Severity Level            Description
        Red                       Software is not functional. Critical software error
                                  Significant customer impact.
                                      Fatal application error occurs
                                      Significant functionality not available

                                  Defect Response Required Immediately
        Yellow                    Subset of the software is not functional but can be bypassed
                                  Moderate customer impact.
                                      Data displayed is incorrect
                                      Response time unacceptable

                                  Defect Response Required within 1 Day
        Green                     Software is fully functional.
                                  Minor customer impact.
                                      Minor functionality errors not impacting overall
                                         usability
                                      Minor presentation defects

                                  Defect Response Required within 2 Days




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                   29
                                 Test Director User Guide



6.5 Defect Tab
6.5.1 Defect Tab – Creating a Defect

The Defect tab allows for the creation and management of all defects and requests for
enhancement found throughout the testing life cycle:




        Note: If the defect is created from a failed Test, the description field of the defect is
        automatically populated with information from the Test Set.




Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                      30
                                 Test Director User Guide



6.5.2 Defect Tab – Required Fields

There are several required fields when creating a new Defect within Test Director:
        Summary: – A short description of the defect which will be used as the Defect Title
        Defect Type: - The type of defect that this item should be categorized as
        Detected Phase/Cycle: – The phase and cycle in which the defect was discovered
        Finding Group/App: – The business group or application that discovered the defect
        Owning Group/App: – The business group or application which currently owns the
           defect
        Project: - The primary Project that this defect is associated with
        Severity: – The current severity level of the defect (see Section 6.4.1)
        Status: – The current status of the defect (see Section 6.3.1)

6.5.3 Defect Tab – Additional Customized Fields

There are additional customized fields that were added but are not required by the system when
creating a defect. Many of these values are used only within specific LOB‟s. Please refer to
your test coordinator if any of these fields should be used:
         Account #: - Account # information associated with the defect
         Change Request #: - Change request ID from ECM (if applicable)
         Closure Code: - Reason for closure when closing a defect
         Planned Closing Phase/Cycle: – The phase and cycle in which the defect is targeted
            to be closed
         Environment: - The physical environment in which the defect was found
         Fixing Group/App: The business group or application that is actually conducting the
            fix
         Package #: Change Man, Harvest, PVCS or other package number associated with
            the defect fix
         Resolved By: - The actual user who resolved the defect
         Root Cause: - The original root cause of the defect
         Root Phase: - Identifies the Delivery Process phase that is the root cause of the defect
         Business Function/Product: - Identifies in business terms what product has a defect
            logged against it given that multiple projects could be impacting a single product
         Business Impact: - Business impact of the defect
         Description of Fix: - Description of the fix that resolved the defect
         Planned Closing Date: - The targeted planned closing date for the defect
         Script Name: - The script name for defects that weren‟t created directly from a defect
         Migration Date: - The date when the fix associated with the defect will be migrated to
            the test environment to be tested



Created By: Kotti RavindraNath
Date Last Revised: 09/07/04
Version #: 1.0                                                                                31

								
To top