Docstoc

Applying QPM Techniques for effective Test Management of ERP

Document Sample
Applying QPM Techniques for effective Test Management of ERP Powered By Docstoc
					                     Balamurali L               Arun Kumar V
                 Senior SQA Manager           Senior Engineer QA
             balamurali.l@nestgroup.net   arunkumar.v@nestgroup.net

                   Diana Ambrose                Geethu K Nair
                   Senior Lead SQA               Engineer QA
            diana.ambrose@nestgroup.net   geethu.nair@nestgroup.net




5/15/2012                                                             1
 Software Systems – A Part of Daily Life
                                            Banking
                                          applications



          Cars and other                                                         Home
             vehicles                                                          appliances
                                          Software is
                                             now
                                         omnipresent


               Communicati-                                              Educational
                 on Media                                                applications



15 May 2012                ISQT’s STEP-AUTO 2011: Conference on ERP Testing                 2
                                                                     Role of Testing



•   Reduction of errors
•   Increasing confidence of end users
•   Ensuring safe operations
•   Contributes to quality
•   Reduces cost of defect fixing




5/15/2012         ISQT’s STEP-AUTO 2011: Conference on ERP Testing              3
              Testing: Perspective shift…
• Establish confidence that a program does what it is
  supposed to do (Hetzel, 1973)




15 May 2012         ISQT’s STEP-AUTO 2011: Conference on ERP Testing   4
              Testing: Perspective shift…
• The process of executing a program or system with the
  intent of finding errors (Myers, 1979)




15 May 2012         ISQT’s STEP-AUTO 2011: Conference on ERP Testing   5
              Testing: Perspective shift…
• Verifying that a system satisfies its specified
  requirements and identifying the differences between
  expected and actual results




15 May 2012         ISQT’s STEP-AUTO 2011: Conference on ERP Testing   6
                                                                              The Lacuna




 The Planned

            Requirement Design                         Code              Testing



  The Actual
            Requirement     Design                        Code              Testing




5/15/2012             ISQT’s STEP-AUTO 2011: Conference on ERP Testing                7
                 Where we stand now…




            You can define, what you understand.


                   You can measure, what you can define


                      You can manage, what you can measure


5/15/2012              SQT’s STEP-AUTO 2011: Conference on ERP Testing   8
       bug prediction and tracking




5/15/2012     ISQT’s STEP-AUTO 2011: Conference on ERP Testing   9
                                                                        Introduction

      • Uses baseline data of the organization and size of the
        current project
      • Predicts the expected number of bugs and the testing
        schedule for the current project
      • Cumulative bug trend curve resembles the curve of a
        first order response system
      • The actual cumulative bug trend is tracked against the
        estimated
      • Any notable deviation from the bug prediction curve,
        analysis will be triggered and corrective actions taken
        accordingly.


5/15/2012            ISQT’s STEP-AUTO 2011: Conference on ERP Testing           10
                                                            Cumulative Bug Trend Curve

 •   A curve with:
      – The      days     of
                                                            CumulativeBug-TestDay Chart
         testing marked on
         X-Axis
      – The      cumulative



                                    Cumulative Bugs
                                                      400
                                                      350
         number of bugs                               300
                                                      250
                                                      200
         caught on Y-Axis                             150
                                                      100
                                                       50
                                                        0

                                                        24-Jan   29-Jan   3-Feb   8-Feb   13-Feb    18-Feb   23-Feb   28-Feb   5-Mar   10-Mar
 •   The slope of the curve
     gives the rate of bug                                                                    Day


     capture




5/15/2012                 ISQT’s STEP-AUTO 2011: Conference on ERP Testing                                                             11
                                                                          Process In Testing - Monitoring
                                                                                                                                       No. Testing
                                                                                                            Schedule                      Days
                   • Monitoring
                      – Bug Prediction Curve                                                                     ODB
                                                                                                                                        Defect
                                                                                                                                        Density           Expected
                      – Weekly Status Report                                                                                                              Defects &
                                                                                                                                                          Schedule
                      – % Schedule Completed                                                                    Project
                                                                                                                 Team
                                                                                                                                          Size

                      – Resource Utilization                                                                                          Defect
                                                                                                                                    Prevention
                                                                                                                                     measures



                        CumulativeBug-TestDay Chart                                                                    Est vs Act Total Bugs Curve




                                                                                    Cumulative No. Bugs
                  400                                                                                     200
Cumulative Bugs




                                                                                                                                                                  Est
                                                                                                                                                                  Defect
                  300                                                                                     150                                                     Act
                                                                                                                                                                  Defects
                                                                                                                                                                  ULEst
                  200                                                                                     100
                                                                                                                                                                  LLEst
                  100                                                                                     50
                   0                                                                                       0
                   24-Jan     3-Feb   13-Feb   23-Feb    5-Mar   15-Mar                                         0         2        4             6   8   10
                                           Day                                                                                No. Days of Testing




                  5/15/2012                             ISQT’s STEP-AUTO 2011: Conference on ERP Testing                                                          12
                                                                                  Bug Prediction Curve

                                        Helps in tracking the test execution
            Cumulative number of bugs




                                                   Number of days


5/15/2012                                  ISQT’s STEP-AUTO 2011: Conference on ERP Testing       13
                                                                               Bug Prediction Curve


                                              Bug Trend Curve

                            200
      Cumulative No. Bugs




                                                                                              Est Defect
                            150
                                                                                              ULEst
                            100
                             50                                                               LLEst


                             0
                                  0     2            4            6           8          10
                                            No. Days of Testing


5/15/2012                             ISQT’s STEP-AUTO 2011: Conference on ERP Testing                     14
                                                                 Possible Outcomes


• The actual curve - within expected trend
• The actual curve - outside expected trend
   – The number of bugs in the initial quarters is much less
     than estimated
   – The number of bugs in the initial quarters is much more
     than estimated




5/15/2012         ISQT’s STEP-AUTO 2011: Conference on ERP Testing            15
                                                                 The Scenario Diagram

                                                  Situations




              Bugs in initial stage :                                         Bugs in initial stage :
               Less than what is                                               More than what is
                   estimated                                                       estimated




                                                                                    Unstable
                                   Inadequate                                       Product
     Stable Product
                                     Testing




5/15/2012                  ISQT’s STEP-AUTO 2011: Conference on ERP Testing                             16
                                                                                     Inadequate Testing


                                        Est vs Act Total Bugs Curve

                                                                                                  Est Defect
                              200
        Cumulative No. Bugs




                              150                                                                 Act Defects

                              100
                                                                                                  ULEst
                               50
                                                                                                  LLEst
                               0
                                    0        2           4          6          8             10
                                                 No. Days of Testing




5/15/2012                                 ISQT’s STEP-AUTO 2011: Conference on ERP Testing                      17
                                                                                           Unstable Product


                                       Est vs Act Total Bugs Curve

                                                                                                 Est Defect
                             200
       Cumulative No. Bugs




                             150                                                                 Act Defects

                             100                                                                 ULEst

                              50                                                                 LLEst
                              0
                                   0                         5                              10
                                              No. Days of Testing




5/15/2012                               ISQT’s STEP-AUTO 2011: Conference on ERP Testing                       18
                                                   Challenges in ERP Testing

       • Wide range of stakeholders that use the system
       • Impact of changes across various modules and higher risk
         of issues to be found at integration
       • Inherent slippages that is accumulated over various
         phases of the development lifecycle
       • Tracking and Managing multiple test iterations in parallel
       • High pressure to ensure delivery dates as well as the
         quality of delivery




5/15/2012             ISQT’s STEP-AUTO 2011: Conference on ERP Testing   19
                                                                 In the ERP context


• Bug trend curve can be calculated for different iterations
• Specification limits can be defined for each iterations
• Actual bug trend curve can be plotted and monitored for each
  iterations




5/15/2012         ISQT’s STEP-AUTO 2011: Conference on ERP Testing             20
            Bug Trend Curve for multiple iterations




5/15/2012     ISQT’s STEP-AUTO 2011: Conference on ERP Testing   21
                Test Status Indicator




5/15/2012   ISQT’s STEP-AUTO 2011: Conference on ERP Testing   22
                                                      Test Status Indicator


• Root mean square of the variation between the
  estimated and actual bug trend curve
• The variation between the planned and actual
  value of the bugs can be calculated on a daily
  basis.
• The overall TSI values of the total projects in an
  organization can be plotted in a control chart for
  monitoring.
• The specification limits can be defined and from
  the control chart, the points which fall outside the
  limits can be analyzed by the top management.

5/15/2012       ISQT’s STEP-AUTO 2011: Conference on ERP Testing       23
                                                               SPC for TSI




5/15/2012   ISQT’s STEP-AUTO 2011: Conference on ERP Testing         24
                       SPC for TSI in different iterations




5/15/2012   ISQT’s STEP-AUTO 2011: Conference on ERP Testing   25
                                            Advantages




5/15/2012   ISQT’s STEP-AUTO 2011: Conference on ERP Testing   26
                                                                                              Case Study



                                                                                • Reduction in   % effort
Project      Language   Plat form   Start Date   Cost of a Bug % Testing
                                                 (Hrs/Bug)     Effort           for testing
                    Before implementation of the method

Project 1      VC        Win NT     13-Mar-11        1.11          11%
                                                                                • Implementation of
                                                                                Bug Prediction and
Project 2      VC        Win NT     21-Dec-10        1.28          37%
                                                                                Tracking method
Project 3      C#        Win NT     13-Mar-11        1.86          25%
                                                                                ensures quantitative
                    After implementation of the method
                                                                                monitoring and control
Project 4      VC        Win NT     16-May-11        0.68          13%

Project 5      VC        Win NT     11-Jun-11        0.94          7%           • Critical phases tracked
                                                                                in an efficiently and
Project 6      C#          .net     29-Apr-11        0.63          7%
                                                                                effectively

 5/15/2012                           ISQT’s STEP-AUTO 2011: Conference on ERP Testing                  27
            Questions


5/15/2012   ISQT’s STEP-AUTO 2011: Conference on ERP Testing   28
            Thank you



5/15/2012   ISQT’s STEP-AUTO 2011: Conference on ERP Testing   29

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:2
posted:5/16/2012
language:
pages:29