Test Management and Contracts in
1 Old Forge Close
SL6 2RD UK
t: 01628 639173
Slide 1 Assurance with Intelligence
Paul is the founder and Principal of Gerrard Consulting, a services company
focused on increasing the success rate of IT-based projects for clients. He has
conducted assignments in all aspects of Software Testing and Quality
He has degrees from the Universities of Oxford and London, is Web Secretary
for the BCS SIG in Software Testing (SIGIST), Founding Chair of the ISEB
Tester Qualification Board and the host/organiser of the quarterly UK Test
Management Forum and annual Summit conferences. He is a regular speaker
at seminars and conferences in the UK, continental Europe and the USA.
Paul has written many papers and articles, most of which are on the
gerrardconsulting.com website. With Neil Thompson, Paul wrote 'Risk-Based E-
Business Testing' - the standard text for risk-based testing. Paul is currently
researching test methods and tools for large ERP implementation programmes
in collaboration with ERP user organizations and academia..
In 2008, Paul set up a new company, Aqastra with Susan Windsor. The aim of
Aqastra is to provide an assessment, retraining and mentoring service to
organisations wishing to transform business users into testers.
Slide 2 Assurance with Intelligence
n Agile approaches are well-established
n “Customer Collaboration over Contract
Negotiation” sounds great as a value
n But none of the Agile principles provide
guidance on how contracts are created
n Contracts are tools for promoting ‘good
behaviour’ in project participants
n Lots of advice available, but are there some
universal Axioms or principles we can apply?
n Agile Values and Contracts
n How Testing Helps Agile
n Test Axioms and Contracts
n Test Axioms Worksheet
Slide 4 Assurance with Intelligence
I am not a lawyer!
These notes are based on my own
experience in a range of projects. Use
them at your own risk.
Agile Values and Contracts
Slide 6 Assurance with Intelligence
Agile values (from agilemanifesto.org)
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
n There is value in the items on the right, but
we value the items on the left more.
Slide 7 Assurance with Intelligence
Agile principles (from agilemanifesto.org)
1. Our highest priority is to satisfy the customer 7. Working software is the primary measure of
through early and continuous delivery progress.
of valuable software. 8. Agile processes promote sustainable
2. Welcome changing requirements, even late development. The sponsors, developers,
in development. Agile processes harness and users should be able to maintain a
change for the customer's competitive constant pace indefinitely.
advantage. 9. Continuous attention to technical excellence
3. Deliver working software frequently, from a and good design enhances agility.
couple of weeks to a couple of months, with 10. Simplicity--the art of maximizing the amount
a preference to the shorter timescale. of work not done--is essential.
4. Business people and developers must work 11. The best architectures, requirements, and
together daily throughout the project. designs emerge from self-organizing teams.
5. Build projects around motivated individuals. 12. At regular intervals, the team reflects on how
Give them the environment and support they to become more effective, then tunes and
need, and trust them to get the job done. adjusts its behavior accordingly.
6. The most efficient and effective method of
conveying information to and within a
development team is face-to-face
Slide 8 Assurance with Intelligence
Agile v contracts
n “Customer collaboration over contract
negotiation” could mean:
- Rather than write contracts, we’ll communicate
with our customers better
- Therefore we HATE contracts!
n But this distorts the meaning of the value
n The value promotes the importance of:
Shared goals and constant
Slide 9 Assurance with Intelligence
n The Agile values discourage…
- Developer and customer taking ‘positions’
- Negotiating a commercial ‘middle ground’
- Freezing people’s attitudes towards fixed
n And encourage…
- Developer and customer are collaborative
- Goal-setting as ‘part of the process’
Slide 10 Flexibility to change is the key to success.
Assurance with Intelligence
How Testing Helps Agile
In the following slides, I use the term INTELLIGENCE to refer to the
information and evidence that is provided by testing and testers
Agile principles and testing
Principle Where testing fits in
1. Our highest priority is to satisfy the customer Early and continuous delivery of
through early and continuous delivery of INTELLIGENCE of valuable software. Testing
valuable software. must link to and demonstrate VALUE.
2. Welcome changing requirements, even late Changing requirements, however ‘welcome’ will
in development. Agile processes harness be reduced by providing EARLY test
change for the customer's competitive INTELLIGENCE for more reliable decision
3. Deliver working software frequently, from a Need to provide test INTELLIGENCE RAPIDLY
couple of weeks to a couple of months, with with flexibility.
a preference to the shorter timescale.
4. Business people and developers must Better, faster appraisal of the status of delivery
work together daily throughout the project. through ACCURATE, TIMELY INTELLIGENCE.
5. Build projects around motivated individuals. See above.
Give them the environment and support they
need, and trust them to get the job done.
6. The most efficient and effective method of withMaybe, but developers need INTELLIGENCE to
Slide 12 Assurance Intelligence
conveying information to and within a support or justify their proposed approach or
Agile principles and testing 2
Principle Where testing fits in
7. Working software is the primary measure of Delivery of test INTELLIGENCE is the only way
progress. to demonstrate progress.
8. Agile processes promote sustainable Delivery of INTELLIGENCE must be EFFICIENT.
development. The sponsors, developers, and
users should be able to maintain a constant
9. Continuous attention to technical excellence Continuous attention to EVIDENCING
and good design enhances agility. progress…
10. Simplicity--the art of maximizing the Achievement of test INTELLIGENCE is a
amount of work not done--is essential. SUFFICIENT requirement so should be
delivered as a PRIORITY.
11. The best architectures, requirements, and Part of the belief system.
designs emerge from self-organizing teams.
12. At regular intervals, the team reflects on how Only with evidence of ACHIEVEMENT can
to become more effective, then tunes and effectiveness be assessed.
adjusts its behavior accordingly. Assurance with Intelligence
Other references to Agile and
n Mary and Tom Poppendieck (http://www.poppendieck.com/)
n Jens Coldewey, cutter.com, “Outsourcing Agile Contracts”, “Contracting
n Kent Beck, Dave Cleal, “Optional Scope Contracts”,
n Nora Sleumer et al., “Pay Per Use Contracts” see Poppendieck site
n Martin Fowler, several bliki posts, www.martinfowler.com/bliki
n Alistair Cockburn, “Agile Contracts”, www.alistair.cockburn.us
n DSDM Consortium, “Sample Agile Contract”,
n Quinary, “Optional Scope Project description”, see Poppendieck site
n Mountain Goat Software, “Writing Contracts for Agile Development”,
n Lots of references to “Fixed Price Contracts and Agile” on the web. “They do
work…”, “They don’t work…” etc.
Slide 14 Assurance with Intelligence
Test Axioms and Contracts
(see Axiom Slides)
Slide 15 Assurance with Intelligence
n Each Axiom has a corollary (i.e. “If you don’t
do this or ignore the axiom…”)
n Usually six questions (you can add your
n For each question:
- What project behaviour am I expecting?
- How do I encourage that behaviour?
- How do I instill that behaviour?
- How do I enforce that behaviour?
n This should give you the text you need to
agree in a contract – just edit it together.
Slide 16 Assurance with Intelligence
So, you aren’t going to GIVE me
some text for an Agile contract?
n No – ALL testing is Context-Dependent
n You need to understand your own project, it’s
goals, team culture, constraints, the ‘art of the
possible’ etc. etc.
n Every project, goal, culture, constraints etc.
n Only YOU can evaluate these to answer the
n So let’s look as an example Axiom and
Slide 17 Assurance with Intelligence
Testing needs stakeholders
n Identify and engage the n Who are they?
people or organisations n Whose interests do they
that will use and benefit represent?
from the test evidence n What intelligence do they
you are to provide want?
n Corollary: You won’t have
n What do they need it for?
a mandate or any
n When do they want it?
authority for testing.
Reports of passes, fails n In what format?
or enquiries have no n How often?
Slide 18 Assurance with Intelligence
The Axioms Worksheet
Show the Word document…
Slide 19 Assurance with Intelligence
Agile testing and Axioms
n Contracts normally specify obligations and
encourage certain behaviours to achieve a goal
- Normally fixed costs, fixed timescales and known
n Agile contracts assume scope is variable
- Agile ‘works’ because of close collaboration,
communication and trust
n Testing is an information/intelligence provision
n Contracts need to specify obligations and
behaviours to deliver most valuable intelligence
n Axioms define a minimum and non-negotiable set
of behaviours for ANY testing context
Slide 21 Assurance with Intelligence
n So can be used as the basis of testing in ANY
Using the Axioms
n The Axioms specify WHAT should be done
n It is up to YOU to define HOW it will be done
n The HOW is a set of agreed processes, procedures,
deliverables, schedule, entry/exit criteria
n These practices are context-dependent and must
satisfy the constraints of your context
- The development process
- The skills, capability and culture of the people
- The technology available
- The time available
- The specific needs of your testing stakeholders
- Time and cost
Slide 22 Assurance with Intelligence
The three axiom groups
n Testing is an information provision service
n Work out who your testing stakeholders
- You (as a developer), Developers, Users,
Project Management, Sponsors
n What do those stakeholders want?
n How will you (and/or your supplier) deliver
n Contracts define and enforce behaviours
Slide 24 Assurance with Intelligence
n Axioms trigger the right questions to ask.
Test Management and Contracts in