Optimizing Your Agile Testing Processes
September 23, 2008
www.uTest.com
1
Agile Manifesto
• • • • Individuals and interactions Working software Customer collaboration Responding to change over over over over processes and tools comprehensive documentation contract negotiation following a plan
Standard S/W Theory
Requirements Analysis System Design Coding QA & Bug Fixes Release!
3
The Reality…
Requirements Analysis System Design Coding
New Requirements
Boss’s Pet Feature
Redesign Coding
Redesign Coding
QA
Release!
4
QA is a defensive stance… “Protect the user” mentality
5
Agile Development
Iterative Releases
-2-4 weeks apart -Unique value in each release
Short Development Cycles (aka Iteration, Sprint, etc.)
-Don’t over design: Only what you need for this sprint -Testing integral to process -Focus on software, not documentation -Design / Test / Code as a team -Business User is part of cycle
Is Agile New?
7
Move the deadline up! Kill all documentation! Keep coding till the day of release!
This is NOT Agile!
8
Agile is VERY orderly
9
Write Unit Tests
Pass
Run Tests
Fail
Write Some Code
Test Driven Development
Fail
Run Tests
Pass
Check-in
More Features?
No
Yes
Sample JUnit Test Case
@Test
The JUnit Test Case that we are building
public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); }
Method in JUnit that: •Checks if the two arguments are equal •Displays message if assertion fails
The functionality we are about to develop, which needs to be tested
11
Sample Unit Test Code
@Test public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals( 3 , myMathClass.add(3,0) ); }
12
Sample Unit Test Code
@Test public void testAddition() { assertEquals( 12 , myMathClass.add(7,5) ); assertEquals( 12 , myMathClass.add(5,7) ); assertEquals( 3 , myMathClass.add(3,0) ); assertEquals( 2 , myMathClass.add(-5,7) ); assertEquals( -2 , myMathClass.add(5,-7) ); }
13
Sample Unit Test Code
@Test public void testAddition() { assertEquals( assertEquals( assertEquals( assertEquals( assertEquals( assertEquals( } 12 , myMathClass.add(7,5) ); 12 , myMathClass.add(5,7) ); 3 , myMathClass.add(3,0) ); 2 , myMathClass.add(-5,7) ); -2 , myMathClass.add(5,-7) ); 133332 , myMathClass.add(55555,77777) );
14
Sample Unit Test Code
@Test public void testAddition() { assertEquals( assertEquals( assertEquals( assertEquals( assertEquals( assertEquals( assertEquals( } 12 , myMathClass.add(7,5) ); 12 , myMathClass.add(5,7) ); 3 , myMathClass.add(3,0) ); 2 , myMathClass.add(-5,7) ); -2 , myMathClass.add(5,-7) ); 133332 , myMathClass.add(55555,77777) ); 6 , myMathClass.add(1,2,3) );
15
Don’t worry about that Failure…
Don’t worry about that Failure…
It’s a Known Bug
The Feature Is Not For This Release
16
Testing Phases
17
Compliance Matrix Complexity
Low Complexity:
Browser Version
High Complexity:
OS, Browser, Browser Settings/Plug-ins, Firewall / Anti-Virus / Filtering, Connection Speed, Language/Region
Medium Complexity:
OS, Hardware, Device Drivers
1997
Native-Code
1999
2001
2003
2005
2007
Dynamic Web Apps
2009
HTML Server-side
• Testing Compliance Matrix keeps getting bigger • For web apps, who is the customer in the team?
18
The Tester’s Role
A. Unit Tests = Specs
• QA and PM meet early • No more SRS TP Alchemy
B. Testing is part of development process
• Team development calls for testing frame of mind • In a programming team / pair, not everyone is homogenous
C. Acceptance Testing by Customer
• Coordinating (and participating in) customer testing / community testing
Case Study: Testing in Short Iterations
2 week development cycle
Fri 5PM - Mon 9AM Testing Simultaneous worldwide testing by uTest Community
Mon - Fri Development
Develop features for current Sprint cycle
Mon 9AM Version Release Fri 5PM Release To Testing
Post the Release Candidate to Testing arena Showstopper fixes, release decision and post to production site
Fri 5PM Release To Testing
Post the Sprint version and test plans to Testing arena
Mon - Fri Bug Fixing
Stabilization and Bug Fixing. No new development
Mon 9AM Complete Bug Report
R&D Reviews Bugs Identified in Sprint version
Fri 5PM - Mon 9AM Testing Simultaneous worldwide testing by uTest Community
Scoping each Sprint
Fri 3PM Sprint Decision Closure
List of candidate features for upcoming Sprint cycle is finalized
Fri - Mon Testing Mon - Fri Development
Fri 10 AM R&D Response to Biz Team
R&D analysis is presented to business team
Mon - Fri Bug Fixing Wed-Thurs Dev Speccing
R&D Team Leaders analyze candidates for code complexity, cost of development and risk factors
Fri - Mon Testing
Tues 11AM Sprint Planning Meeting
Prod Mgmt presents candidates to R&D Mgmt
Mon 3PM Business Priority Meeting
Sales, Mktg, Prod Mgmt choose se 6-8 candidate features for upcoming cycle
Testing and the End-User
23
Thank you! Q&A
For more info: http://www.utest.com/solutions_agile_testing.htm Download the “Agile Testing with uTest” Case Study: http://www.utest.com/downloadcs.php
24