Testing in an Agile Environment by vijayvarma

VIEWS: 1,155 PAGES: 4

More Info
									White Paper

Testing in an Agile Environment
Last Updated: 15th January, 2009

The principal indicators of a growing business are a widening customer base and rising return on investment (ROI). Various parameters steer these indicators in a positive direction, chiefly cost, time and quality. Balancing these parameters is of paramount significance for business growth. Companies need to deliver, cost effectively, the right product, in the right way, to meet customer requirements with assured quality. They must do this first time every time to ensure that they can bring new products and services to market ahead of their competition. Companies also need to extensively consider market dynamics including technological developments, changing customer preferences, evolving global standards and regulation. These market developments add further complexity and force companies to enhance the frequency of their product updates and accelerate delivery schedules. In software product development, an Agile methodology is one strategy adopted while addressing the challenges of market dynamics. As customer satisfaction and ROI are linked to product quality and cost of quality, QA and testing services have a critical bearing on the success of any Agile project. This white paper explores QA and testing activities in an Agile environment and the critical characteristics of the testing team that ensure the success of development projects in an Agile environment. As there is broad-based acceptance of agile practices, an introduction to these practices and agile development methodologies is not provided in this paper. The testing agenda during the development phase is very different to the traditional approaches that support the Waterfall or V-Model development methodologies because Agile uses a Test Driven Development (TDD) model. However, many of the requirements of the extended testing model will still apply.

Project Initiation Phase
For an Agile project to succeed, it requires effective planning and management. As part of the project team the Test Manager is responsible for establishing the quality processes, identifying test resources and delivering the test strategy. The test strategy will include details of the Agile development process being used. It will also include details of test phases not directly related to the Agile development for example, Operational Acceptance or Performance Testing.

Development Phase
True Agile development uses a “Test Driven Development” (TDD) approach. The testers, developers, business analysts and project stakeholders all contribute to kick-off meetings where the “user stories” are selected for the next sprint. Sprints are usually of fixed duration and selecting the right number of stories and estimating the time to deliver them is the key to success. The estimation process can often be inaccurate at the beginning of an Agile project but improves as the team’s experience grows within the specific implementation they are working on. Once the user stories are selected for the sprint they are used as the basis for a set of tests. The testers create test scenarios which are presented to the business analysts and project stakeholders for their approval and signoff. These test scenarios are then broken down to test cases that offer adequate test coverage for the given functionality. The developers then write code that will pass the tests. In this approach the development and testing take place continuously throughout the sprint – there is no separate testing phase. In the TDD approach the features are tested throughout the sprint and the results presented to the stakeholders for immediate feedback. The test scenarios defined are not limited to functional testing but can include other types of testing including performance and integration testing when the product is mature enough. While the development is underway the user stories for the next sprint are written. These include the stories specified

Testing Agenda in Agile Projects
Agile methodologies are designed to break the software down into manageable parts that can be delivered earlier to the customer. The aim of any Agile project is to deliver a basic working product as quickly as possible and then to go through a process of continual improvement. An Agile project is characterized by having a large number of short delivery cycles (sprints) and priority is given to the feedback-loops from one cycle to the next. The feedbackloops drive continuous improvement and allow the issues that inevitably occur (including the way requirements have been defined or the quality of the solution provided) to be dealt with much earlier in the development life cycle. To achieve this, companies need to re-think their approach to delivery and have their previously independent contributors (Business Analysts, Developers, Testers, End Users etc.) work together in teams.

App_WhitePaper_Testing_In_Agile_Environment_1v00 Page 2 © 2009 AppLabs

in the delivery plan but will also include additional stories required to cover any issues that have been identified as part of the feedback process from previous sprints. Sprints in an Agile project can extend to multiple levels in a complex system. A sprint might not lead to a product release if it does not add enough functionality to the product being developed. The stakeholders take a decision on when the application should be moved to the release phase depending on the market need or the level of key functionality being added to the system. While multiple iterations may be required to release a product, there may also be cases where releases are more regular owing to the additional value delivered at each iteration level. Whichever release approach is adopted, the testing team’s goal is to have a release available with minimal defects and low implementation risk at the end of the Sprint. As functionality grows with each iteration, regression testing must be performed to ensure that existing functionality has not been impacted by the introduction of new functionality in each iteration cycle. Defect fixes should also be followed by extensive regression testing. The scale of the regression testing grows with each sprint and to ensure that this remains a manageable task the test team should use test automation for the regression suite and focus their manual testing effort towards locating new defects during the build phase.

Resource Management
The Agile approach required a mixture of test skills – usually held across one team. Test resource will be required to define the scenarios and test cases, conduct manual testing alongside the developers, write automated regression tests and execute the automated regression packs. As the project progresses, specialist skills will be required to cover further test areas that might include integration and performance testing. The availability of a pool of professional test resources offers the scope for effective resource management across the agile project life cycle. There should be an appropriate mix of domain specialists who plan and gather requirements in addition to test engineers who will have multiple skill sets and own the test execution.

The benefits of independent testing will not be realized unless good communication exists between the testers, developers, business analysts and project stakeholders. The iterative model of development and the frequency of releases mandate that all teams have a common understanding of the user requirements. The testing teams should be skilled in the use of change management tools. The communication model used by the testing team itself must enable both regular and ad-hoc project updates from various parties engaged in the project. The testing team should adopt the most efficient and effective methods of conveying information to the developers, project stakeholders, domain specialists using a combination of face-to-face conversation, meetings and workshops, phone calls, email and WebEx meetings.

Release Phase
The aim of any Agile project is to deliver a basic working product as quickly as possible and then to go through a process of continual improvement. This means that there is no single release phase for a product. The product will move into production when enough of the functionality is working and the quality of the product is acceptable to the stakeholders. Prior to release, a final acceptance test is performed before transitioning the application into production. The testing activities listed above are not exhaustive but broadly cover the areas which the testing team contributes to the Agile approach.

Another key success factor for Agile Development is the implementation of quality governance processes such as configuration management. Typically, an organization that has no processes in place will be in chaos. As formal processes are implemented the organization will be able to follow a repeatable and consistent delivery method. Organizations considering the use of Agile should ensure that configuration management, change management, project management and release management are in place. Testing teams which bring with them best practices and are accredited with globally recognized certifications (e.g. TMMi, CMMi, ISO etc.) will be able to help organizations accelerate testing and enable lower cost of quality to be achieved.

Getting Equipped for Agile Testing
Agile projects present their own challenges to the testing team. Unclear project scope, multiple iterations, minimal documentation, early and frequent testing needs and active stakeholder involvement all demand special and diverse skills from the testing team. Some of the essential skills are illustrated here:

App_WhitePaper_Testing_In_Agile_Environment_1v00 Page  © 2009 AppLabs

Requirements Elicitation
Agile projects do not necessarily have comprehensive documentation. Product requirements in Agile projects are often captured in the form of user stories. The testers should be experienced and smart in analyzing and defining the testing requirements in the absence of any product documentation. The ability to work side-by-side with the developers, analysts and customers is an important part in quickly understanding the test requirements. The testers should also be equipped with alternative ways of deriving test requirements. A few such approaches include:
 Getting access to all the material that can provide input

Companies which adopt Agile projects should note the importance of engaging test teams at project initiation. This will ensure an accelerated delivery of working software. If Agile projects are to achieve customer satisfaction and ROI then time, cost and quality must be controlled and balanced. To ensure accelerated delivery of working software which conforms to the desired quality, the testing team should be involved from the beginning of every iterative development cycle and not just after the first couple of sprints. The testing team must develop the necessary mindset for an Agile project. Their own agility and flexibility is essentially the key to their success in the project.

for the intended product behavior. This can include input provided to the developer to build the application, feature list, brief write-up on application capabilities and partially written use cases etc.
 Knowledge transfer from product owners and subject

matter experts (SME). Direct communication will be the best practice here as time spent with the SME helps to elicit more product knowledge than is expressed in written documents.
 Gaining insights into the capabilities needed from

the product under test by exploration of comparable products, reading user manuals and user documentation, exploring help text via user interface and taking an application tour through the GUI.
 The testing team performing exploratory testing at

various stages of the life cycle. The testing team should be well trained to scope the work, ask the right questions, and deliver a valuable output within a few hours. Exploratory test engineers should be able to analyze a product, think critically to evaluate risks and craft test cases that systematically explore the product.
 The frequency of testing is greater in an Agile project

than in a traditional development model. The testing team should be thought leaders in test automation. The testing team should, in conjunction with the development team, develop an automation framework that fits the unique requirements of the Agile project. In addition, the testing team should be efficient in identifying the candidates for automation and build a suite of automated tests around the application which enable rapid iterations. Specialist testers from the test team should have these and many more skills which are attained through experience as testing professionals. Testing teams delivering services in an Agile environment need to develop and demonstrate these skills to contribute to the success of Agile projects.

App_WhitePaper_Testing_In_Agile_Environment_1v00 Page  © 2009 AppLabs

To top