PrimerOnOutsourcedTesting by usvoruganti


									Outsourced Software Testing 101
A guide for Business Heads and Project Managers

Business Analysis & Research Group
Thinksoft Global Services


   Copy Right Notice: All content of this document is copyrighted by Thinksoft Global Services. All
   rights reserved. No portion of the content may be directly or indirectly copied, published,
   reproduced, modified, performed, displayed, sold, transmitted, published, broadcast, rewritten for
   broadcast or publication or redistributed in any medium without express permission from Thinksoft
   Global Services. Readers of this document are deemed to have read and accepted the conditions
   of this copyright notice.

© 2003 Thinksoft Global Services                             Page 1
Outsourced Software Testing 101
A guide for Business Heads and Project Managers

Some of the queries that this Primer will attempt to answer:

    Why should I outsource application software testing in my organization?

    How do I start an outsourced testing program in my organization?

    What is the safest route to trying out the benefits of outsourced testing?

    I already have several global development vendors; can I manage an independent testing
    service provider relationship as well?

    How do other Global 500 companies do this?

    What should I look for in an outsourced testing services vendor?

Introduction: The last few years have seen an increase in the number, size and complexity of
application software deployed at large global financial corporations. These applications have
tended to be increasingly centralized with larger numbers of users across multiple locations.
The design, development, testing and deployment of these large applications have thus called
for specialised skills. Driven by a range of factors from skills and resource availability to time
and cost pressures, Global 500 corporations have successfully looked to outsourcing as a means
to bring such applications to deployment on time and within budget.

Testing is one of the most critical processes involved in the development of software
applications. Over the life of a software product, it also takes up the most effort. Yet enough
attention has perhaps not been paid to the efficient outsourcing of this component of the
product cycle. How outsourced testing by a specialist independent test service provider can
greatly mitigate application deployment risks, especially in the context of the banking and
financial services domain, is the subject of a separate White Paper on this web site (Refer to
Domain focused independent testing: Why CEOs of Global 500 financial majors use this tool to
reduce application deployment risks…and costs).

Executive Summary: This Primer attempts to serve as an introductory how-to guide on
Outsourced Software Testing for both business heads as well as software project managers:
CEOs as well as CIOs. It first places testing within the context of the total product life cycle
(TPLC), indicating the reasons for considering outsourced-testing. It then describes the
sequential stages within a testing program, and how these stages may run in parallel with the
other development stages. The Primer then sets out the progressive steps by which an
organization can implement an outsourced software-testing regime. It then presents a
suggested project interaction model for outsourced testing, and then finally, suggests a list of
selection criteria for assessing test-outsourcing partners.

© 2003 Thinksoft Global Services                      Page 2
1. The Total Product Life Cycle (TPLC)

   Figure 1: Components of effort in the Total Product Life Cycle

       Development Phase                         Maintenance Phase

          Requirements                             New requirements

         Functional Specs.                                &
           Development                              Enhancements


                                                   Error corrections

                 33%                                      67%

      Source: Barry Boehm

The software product life cycle begins with a business need articulated by the user group. This
may be for a new application, a newer generation consolidation of a set of existing applications
or for a large-scale redesign and migration of an already operational system to a newer
technology environment. The product cycle consists of two parts, the software development
phase and the maintenance phase, the relative efforts for which are normally found to be in
the ratio 1:2. The initial development phase has distinct specification, development and
testing stages to be executed before the application can be deployed.

The maintenance phase starts once a software application is deployed. It then enters into
useful ‘production’ and simultaneously enters into a cycle of additional releases driven by
defect corrections, functional enhancements and other user-driven changes. Each of these
releases has to follow a development, testing and deployment path similar to the original
development cycle: a typical application will go through many releases – often 4 to 6 every
year – before it is completely replaced by a new software application.

The product cycle is similar for both custom development as well as for packaged software
implementation. In the latter case, the additional development will be driven by customization
as well as interface needs: the whole configuration will then need to be integrated and tested.

© 2003 Thinksoft Global Services                     Page 3
Outsourcing different components within the TPLC

In recent years, the software development phase has increasingly been outsourced to
application development vendors for the following reasons:

        Specialist resources being available with external organizations

        Potential cost benefits from locating the development efforts offsite or offshore

        Advances in both technology and infrastructure enabling such offshore migration of
        development activities.

While the benefits of outsourcing have been quite well appreciated and exploited in the case
of the software development phase, the potential benefits of outsourcing the application
testing component across the total product life cycle are only now being seen and exploited by
Global 500 Financial Giants.

Outsourced testing has a positive impact on quality, costs and time across the software product
life cycle due to:

        The relative significance of testing and defect corrections as a component of effort in
        the overall context of the TPLC

        The numerous value creators underlying outsourced software testing (set out in detail
        in the accompanying White Paper)

        The “parallelism” inherent in the TPLC that enables the testing track to run
        concurrently with the development track across the whole cycle. This is illustrated in
        the following section.

And this is why you too should be looking at outsourced software testing: huge savings on life
cycle costs with lower risk to the business.

© 2003 Thinksoft Global Services                     Page 4
2. How testing stages can run parallel with application development

      Figure 2: UAT track in parallel with development

   Development       Business        Functional              Design,            Integrated
   Track           Requirements        Specs.               Development          System
                      (BR)                                     &                   Test
                                                              Unit                (IST)

   Testing         Test Strategy           Test                Test             User           Release
   Track           Preparation            Ware                Ware            Acceptance        For
                                          Design              Design            Test           Impln.

      Figure 3: Total product life cycle testing (across multiple releases)

   Development                Release 1                                Release 2
                       Business        System                   Change           System
   track             Requirements    Development              Requirements       Testing
                        (BR)          & Testing

   Testing         Test Planning     Test Execution             Test               Release
                                         (UAT)               Maintenance            Test
   track                                                                           (UAT)

                                          Release 1                             Release 2    Release n

© 2003 Thinksoft Global Services                                   Page 5
Irrespective of the type of testing to be executed, viz., integrated functionality testing,
integrated system test (IST) or user acceptance testing (UAT), all testing phases within the
TPLC consist of five distinct components set out in Table 1 below.

Figure 2 above sets out the sequencing of testing activities vis-à-vis the development cycle in a
typical User Acceptance Testing (UAT) context: the testing track here runs in parallel to the
development track, saving time to deployment.

Figure 3 then demonstrates a typical multiple-release development cycle, where test ware
maintenance is an important component in ongoing release testing and management.

Table 1: Components/stages of testing
Test component Activities                                                      Deliverable(s)
/ stage
Preparing the test         Scope and types of testing to be conducted          Test Strategy
                           Organization & resources management plan
                           Effort estimations
                           Scheduling guidelines
                           Defect management plan

Designing the test         Functions and Conditions to be tested               Test Ware
                           Test cases and data guidelines
                           Test scenarios and detailed test scripts
                           Conditions and trace-ability         matrix    to
                           demonstrate test completeness
                           Test methodologies – iterations & their
                           Defect reporting framework

Drawing up test            Decide on number of cycles per round, based         Test Run Plan
schedules                  on the functionality requirement
                           Prepare a detailed Run Plan, which specifies
                           for each day
                               o   The scripts to be executed, taking
                                   into account dependencies
                               o   The manpower allocation for each
                                   testing cycle
                               o   The pre and post run activities to be
                               o   The test environment in which the
                                   testing will be carried out
                           Based     on    the    Run    Plan,   the
                           bottlenecks/areas of clashes between test
                           types can be identified. De-bottlenecking

© 2003 Thinksoft Global Services                          Page 6
                           should be done.
                           Based on the above activity, the detailed
                           Run Plan should be prepared
                           The overall schedules, manpower allocation
                           and expected daily outputs can then be
                           finalized and circulated to the testing team.

Test execution             Management and test governance                    Defect Analysis
                           Testing operations,    defect   logging     and
Test maintenance           Test ware maintenance and incremental test        Incremental        /
                           ware design in order to reflect change            modified Test Ware
                           requests triggered by
                           o   Software defect removal
                           o   Product enhancements
                           o   Business Process changes

Offshore vs. on-site stages

Any of the stages listed above may be executed at a range of locations with progressive cost

        Off-shore at an independent test service provider (ITSP) site in a different, lower cost
        geographic location

        On-shore at an ITSP site

        On-site at the client site

© 2003 Thinksoft Global Services                        Page 7
3. A roadmap to outsourced software testing

Progressive Engagement Models: Four steps to extracting the value of outsourced

An independent test service provider (ITSP) can be engaged in a number of ways so as to
extract the numerous risk-mitigation benefits possible. This section will briefly describe four of
them and illustrate the benefits flowing from them.

Figure 4 graphically depicts the four steps and the component services that constitute each of
them. The following table then summarizes the activities and deliverables of each of these

 Figure 4: Four steps to outsourced testing
                                                                 Test ware
                                                                 design &              Maintenance
                                                                           Test           Test
                                                      Test ware          Execution      Execution
                                      Test strategy                                     Test Ware
                                                          Test Ware      Test Ware
                                      consulting           Design         Design         Design

                 testing (by          Test         Test      Test      Test
                 user organization lStrategy r inStrategy
5. What should a developers)        ook fo        a test-outsourcing partner?
                                                           Strategy  Strategy

                                              Higher value-impact:
                                                better quality
                                                lower costs and time spent

© 2003 Thinksoft Global Services                            Page 8
Table 2: Steps to outsourced testing

Outsourced           Description          ITSP role &         Deliverables       Organisational
Testing                                   engagement                             process
Engagement                                window                                 maturity
Model                                                                            required
Test strategy        Test vendor to       Start point:        Comprehensive      Access to
consulting           undertake only       Immediately         Test Strategy      relevant
                     Test Strategy        after Business      Document           documents;
                     preparation:         Requirement                            Review and
                     remaining stages     (BR) is frozen                         acceptance
                     of testing to be     End point:                             processes and
                     undertaken by        Acceptance of                          skills
                     developers/users     Test Strategy

Test ware design     Test vendor to       Start point:        Comprehensive      In addition to
outsourcing          undertake            Immediately         Test Strategy      above,
                     complete test        after Business      Document +         Access to
                     strategy and         Requirement         Comprehensive      operating
                     planning,            (BR) is frozen      Test Ware          environment and
                     including            End point:                             platforms;
                     comprehensive        Acceptance of                          Clarifications
                     Test Ware            Test Ware Design                       from & access to
                     design.              Document                               business users

Test ware design     Test vendor to       Start point:        Comprehensive      Additionally,
& execution          undertake both       Immediately         Test Strategy      Close integration
outsourcing          Test Planning        after Business      Document +         on project plans;
                     and Test             Requirement         Comprehensive      Closely coupled
                     Execution Stages     (BR) is frozen      Test Ware +        development
                     in full              End point:          Defect Analysis    processes
                                          Completion of all   Report
                                          iterations of
                                          testing as per
                                          Test Plan
Full-cycle test      Test vendor to       Start point:        Comprehensive      Additionally,
outsourcing          undertake all        Immediately         Test Strategy      Closely coupled
                     testing activities   after Business      Document +         change
                     spanning             Requirement         Comprehensive      management
                     multiple releases    (BR) is frozen      Test Ware +        processes;
                     over the life of     End point:          Defect Analysis    Close integration
                     the product          Ongoing             Report +           on release
                                          engagement          Productivity       management;
                                          spanning            gains in testing   Collaborative
                                          multiple releases   across TPLC        mode on TPLC
                                          of product                             processes

© 2003 Thinksoft Global Services                       Page 9
4. Typical project interaction model

        Independent Test               Client                  Development
        Vendor                                                 Vendor

        Test Manager                                           Development
                                   Client                      Project Manager
        Test Lead                  Coordinator                 Project Lead

        Testers                                                Design & technical
        Subject Matter
                                                               Database Designers
        Expert                     Business Users/IS
        Business Analyst                                       Programmers

© 2003 Thinksoft Global Services                     Page 10
5. What to look for in an outsourced testing vendor

 Table 3: Vendor selection criteria

 Category                                Criterion

 Delivering on Commitments               Quality

 (Getting what you want)                 Schedule


 Engagement Interface                    Project initiation

 (Spending less effort to get it)        Planning and tracking

                                         Communication processes

                                         Metrics reporting



 Internal strengths                      Domain focus

 (Your vendor having what it takes to    Offshore infrastructure
                                         Onsite – offshore delivery model

                                         Resource scalability

                                         Test processes and methodologies

                                         Familiarity and competence with test automation

                                         Productivity focus

                                         Training and skills building

The above is a sample of what you could use in evaluating your outsourced-testing vendors: you
would need to tailor it to suit your particular circumstances by modifying the criteria and/or
applying weightings to each of them.

© 2003 Thinksoft Global Services                      Page 11
6. Summary

    You should be considering outsourced application software testing in your organization for
    the huge savings possible on total product life cycle costs, with lower risks to the business.

    Use the Four-Step model outlined above: Start with a consulting engagement; analyse its
    impact on your project quality, cost and time; if positive proceed to further steps.

    The safe route: Pick a small project as a proof of concept; pick the right vendor; get the
    outsourced-testing processes right; move on to bigger projects

    Process maturity: Most of the processes you already have in place for outsourced
    development would apply to testing as well; PM will need to interact with testing vendor as
    well; suggested interaction model may be adapted to suit your organization.

    Vendor selection: The suggested list may be modified and weighted to suit your needs.

    Other Global 500 financial majors have used similar approaches to the one outlined in this

About Thinksoft Global Services

Thinksoft Global Services is India’s largest independent financial software testing services

Thinksoft helps Global 500 Banks, financial and insurance organizations

        Benefit from the objectivity and rigor of Independent Outsourced software testing
        Unearth critical software defects early on using a domain focused methodology
        Compress time-to-market by outsourcing parallel activities in the product life cycle
        Extract costs by deploying a cost-effective onsite - offshore delivery model

Thinksoft’s Global 500 client list includes Citi Group, Deutsche Bank, Morgan Stanley, GE
Capital, Standard Chartered and ABN Amro among others.

Thinksoft is an ISO 9001:2000 certified company.

                 For more details, please contact

                                   Or visit us at

© 2003 Thinksoft Global Services                       Page 12

To top