How to do Test Estimation

Document Sample
How to do Test Estimation Powered By Docstoc
					How to perform good Test Estimation

Problem Statement:

In this document, I outline some guidelines about performing a Test Estimation exercise. For the purpose of this
document, I mean “Test Development Cost” when I say “Test Estimation”. In other words, the document is focused
on estimating Test Development and not Test Execution.

Test Estimation is not a perfect science and there are many factors that affect this. There are no silver bullets to a
successful estimation. In most cases, understanding and thinking through the factors (parameters) that I mention
below significantly enhances the possibility of a successful estimation.

Goals and Non Goals:

Goals:

   To understand various factors that play a vital role in Test Estimation Process.
   To be able to understand how these factors can be measured.
   To discuss about some possible pitfalls in the process and some tricks of the trade suggestions.

Non Goals:

   This document is limited to test development estimation and does not talk about test execution.
   As I mentioned above, Test Estimation is not a perfect science and hence this document does not intend to
    provide silver bullets. Instead, this focuses more on a generic methodology and suggestive guidelines.

Theory:

First and foremost, Test Estimation should always be “realistic” and should be “backed by information and
description of the factors that influence the estimate.”

The key is to understand the factors those affect estimation. I categorize these factors into 4 buckets:

Source                       Factor Names
Engineering and               clearly defined hand-offs between testing and the rest of the organization
processes                     timely and reliable bug fixes
                              timely arrival of high-quality dev code
                              proper execution of early test phases (unit, component, and integration)
                              Clarification of ownership between the between the various features being tested
                                 so that there is a clear demarcation to make sure nothing is left to
                                 interpretation.(UI testing/API testing, ownership of the different integration
                                 points)

Tools at hand, Resources        Existing vs new development of a test case
                                Quality of the test system which includes the test tools, test environment and so
                                 forth
                                 An adequate dedicated and secure test environment
                                 Goal of Automation – how much is planned automation vs manual
                                 Running the test pass with code coverage offers a good understanding of the state
                                  of testing
                                 available, high-quality (clear, concise, accurate, etc.) project documentation like
                                  requirements, designs, plans, and so forth
                                 Clarity of understandin of functional specs and Dev Design.
                                 reusable test systems and documentation from previous, similar release
                                 the similarity of the project and the testing to be performed to previous endeavors
                                 Not in schedule Quick fixes and Hotfixes and kind of testing performed for them

People and Team                  honesty, commitment, transparency, and open, shared agendas among the
                                  individual contributors, the managers, and the project stakeholders
                                 realistic expectations across all participants, including the individual contributors,
                                  the managers, and the project stakeholders
                                 stability of the project team, especially the absence of turnover
                                 competent, responsive test environment support
                                 use of skilled contractors and consultants to fill gaps
                                 Outsourcing should account for ramp up time, possible geographical remoteness,
                                  6 hour work week and leveraging on their experience for previous releases
                                 the need to ramp up, train, and orient a growing test or project team
                                 Required and sufficient information is supplied by the PM,Dev and leaders of the
                                  project

Other Complication               high complexity of the process, project, technology, organization, or test
                                  environment
                                 Rapidly changing PM spec and/or dev code. In those cases the start date and end
                                  dates change based on the churn
                                 Additions of new DCR’s
                                 the need to assimilate or develop new tools, techniques, or technologies at the
                                  testing or project levels
                                 any requirement for new test systems, especially automated test ware (harness
                                  and methodologies), as part of the testing effort
                                 any requirement to develop highly detailed, unambiguous test cases, especially to
                                  an unfamiliar or incomplete standard of documentation
                                 tricky timing of component arrival, especially for integration testing and test
                                  development
                                 fragile test data, for example, data that is time sensitive
                                 Buffer time incorporated in the test schedule after discussion with lead/manager


Generically speaking, prioritizing these factors is a difficult task because:

    (a) Prioritization depends and varies case to case, feature to feature, product to product
    (b) Factors have some dependencies with each other.

However, I would suggest the following as the FIVE most important factors for Estimation:

    (a) Specs: The Functional specs, Dev Design need to be frozen before a successful Test Estimation can
        happen. Understanding these specs is also a key to successful test estimation as that ensures nothing’s
        falling through the cracks and chances are nothing’s going to be discovered late.
    (b) Freezing the Test Spec: Trying to estimate test automation without scoping and freezing the test
        documentation doesn’t yield a lot of value.
    (c) State of Test Tools and Test Environment: Reliable, Performant and Robust Test Tools (including harness)
        and Test Environment (for example: Test Clusters, Shared Test Code, Test Library etc.) go in a long way to
        positively effect Test Estimation.
    (d) Automation vs Manual Trade-Off: In general, More automation means more development cost. However,
        test automation significantly reduces test execution cost
    (e) Re-Use: If existing test tools, cases and code can be re-used, that helps significantly reduce the test
        development cost.


Methodology:

Step 1. First of all, focus on the top 5 factors I mention above. Also, look at the list above to pick out the most
important factors that affect test estimation in your case. List down the current state and risks involved for these
factors.

Step 2: Analyze the throughput / rate of your test development. In other words, how long does it take for the
concerned Test Developer to develop one unit of test code. This has the potential to become a significant problem
when the person estimating and the person actually developing are different people or geographically distant.

Step 3: Start with T-Shirt size estimation for testing. In other words, divide the estimate into one of “S” (let’s say <
10 man days), “M” (let’s say 10 – 20 man days), “L” (20 – 32 man days) and “XL” ( > 32 man days).

Step 4: Now, break the total work into work items (tasks). Each task should typically be less than 1 man week (i.e. <
5 man days). In other words, making the tasks granular enough so that they can fit in a week.

Step 5: Add up all the work items (tasks). Add 30-50% buffer to that schedule (depending on your team abilities
and experience). Account for Vacation days.

Step 6: Set up a good Tracking Mechanism to track progress week-over-week. Product Studio, xls Worksheet or
Project can be used to set this up.

Step 7: Keep updating your estimation week-over-week.

				
DOCUMENT INFO
Shared By:
Tags: Testing
Stats:
views:9
posted:7/12/2012
language:English
pages:3
Tan Jag Tan Jag Planner http://
About