The Tutorial
www.ddsteps.org
Data Driven
Tutorial
Adam Skogman
Björn Granvik
We Have a Dream
The Best Laid Plans...
Req.
Develop
Unit Test
Function Test
Acc.
Test
Live!
Then, Reality Happens...
Req.
Develop
Unit Test
Pre-Test
Function Test
Acc.
Test
Live!
Bugs? Who knew...
Req.
Develop
Unit Test
Pre-Test
Function Test
Fix
Re-Test
Fix
Re-Test
L
Go Agile!
9 weeks
Develop
Acc.
Req. Unit Test Live!
Test
Function Test
3w Develop
Acc.
Req. Unit Test Live!
Test
Function Test
Develop
Acc.
Req. Unit Test Live!
Test
Function Test
Circle of Life
Tag Update
Function
Test Implement
Deploy Run tests
Commit
Unit Test Detect
Build Changes
Our Dream
Continuous Everything
Zero Known Defects at Release
Known Project Progress
Transparent Project Status
Use Case Coverage
What's the
Problem, Anyway?
Guest Starring: Pet Clinic
www.springframework.org
It’s just Complicated
Add New Pet (OK)
1. Log in
2. Click “Find Owner”
3. Click “Find Owners”
4. Click “Charlie Brown”
5. Click “Add New Pet”
6. Validate defaults
7. Enter data
ID NAME BIRTH_DATE TYPE_ID OWNER_ID
1. “Snoopy”
42 Snoopy 1969-01-13 1 1
2. “1950-10-02”
3. “Dog”
8. Click “Add Pet”
9. Validate owner page
10. Validate database
Why is Function Testing so Hard?
Functions Combinations
testAddPet_Ok testAddPet_Ok_Harvey
testAddPet_Fail testAddPet_Ok_Hobbes
testFindOwner_Ok testAddPet_Ok_Woodstock
testFindOwner_Fail testAddPet_Ok_Snoopy
testFindVet_Ok
Hello DDSteps!
Yes, it's Open Source!
1. Pull Out the Data
Add New Pet (OK) Properties
1. Log in Owner Full Name
2. Click “Find Owner” Pet Name
3. Click “Find Owners” Pet Birthdate
4. Click “Charlie Brown”
Pet Type
5. Click “Add New Pet”
6. Validate defaults
7. Enter data
1. “Snoopy”
2. “1969-01-13”
3. “Dog”
Data reuse
8. Click “Add Pet”
9. Validate owner page
10. Validate database
Put it in a Spreadsheet
One File
=
One JUnit Test Case
Header File name
= 1:1
Property Class Name
One Row
=
One Run
Sheet Name
=
Method Name
Data Driven Test Case
Basic example
Data and Code separation
Setting JavaBeans properties
Does not use
Test Steps
Fixtures
Spring
Example: Compare strings
The Method Under Test
public class StringComparator {
public boolean compare(String one,
String two) {
...
}
}
JUnit is Everywhere
JUnit
IntelliJ Expensive
Eclipse Ant Maven
IDEA Test Tool
Cruise
AntHill Gump Continuum
Control
The Old Way
public void testCompare() {...}
public void testCompareDifferentOrder() {...}
public void testCompareSwedishCharacters() {...}
public void testCompareSmallCharacters() {...}
public void testCompareCaseInsensitive() {...}
public void testCompareStrangeCharacters() {...}
public void testCompareNull() {...}
public void testCompare...() {...}
...
The Data Driven Way
Strings to
compare
Colors, just for
our own
pleasure.
Delimit
Data using
Data and
"---"
Properties
using
"---"
The Test
public class DdStringComparatorTest extends DdTestCase {
private String one;
private String two;
private boolean result;
public void setOne(String one) { … }
public void testCompare() {
StringComparator tested = new StringComparator();
boolean actual = tested.compare(one, two)
assertEquals(result, actual);
}
protected DataLoader createDataLoader() {
return CachingExcelDataLoader.getInstance();
}
(Backup) Keep the matrix green!
(Backup) The Bug
public class StringComparator {
...
public boolean compare(String one, String two) {
return one.equals(two);
}
}
Let's fix it!
(Backup) Fixed code
public boolean compare(String one, String two) {
if (one == null) {
return false;
}
return one.equalsIgnoreCase(two);
}
(Backup) Result
(Backup) ÅÄÖ?
Still fine
Summary
Test Driven
Too Easy?
Can Do a Lot
Unit Testing
Integration Testing
Loads of Opportunities
Testing Web
Applications
PetClinic from Above
JWebUnit
HTTP
JMeter
HTTP
Some
Tool
HTTP
API
TestCase JWebUnit HttpUnit
jwebunit.sourceforge.net
httpunit.sourceforge.net
JWebUnit API
Navigate
webBrowser.clickLinkWithText("Find owner");
webBrowser.clickLinkWithImage("...next.png");
Execute
webBrowser.setFormElement("name", "Snoopy");
webBrowser.selectOption("type", "Dog");
webBrowser.checkCheckbox("fictional");
webBrowser.clickButton("Add");
webBrowser.submit("Finish");
Validate
webBrowser.assertTitleEquals("Login Page");
webBrowser.assertFormElementEquals("name","Snoopy");
webBrowser.assertTextInElement(
"owner","Charlie Brown");
JWebUnit – Start Here
Code
WebBrowser webBrowser = new WebBrowser();
webBrowser.setUrl(“http://localhost:8080”);
webBrowser.beginAt("/");
webBrowser.clickLinkWithText("Find owner");
JWebUnit - Choose a Form
HTML
...
Code
setWorkingForm(owner);
submit();
Data
owner = "Charlie Brown"
JWebUnit – Click "Add New Pet"
Same, same
JWebUnit - Entering Data
HTML
...
Name:
...
Code
setFormElement("name", name);
setFormElement("birthDate", birthdate);
Data
name = "Snoopy"
birthdate = "1969-01-13"
JWebUnit - Drop Box
HTML
Bear
Bird
Dog
Hamster
Tiger
Code
selectOption("typeId", type);
Data
type = "Dog"
JWebUnit - Validating
HTML
NameSnoopy
Birth Date1969-01-13
TypeDog
Code
assertTextPresent("Owner Information");
assertTextPresent("Pets and Visits");
assertTablePresent(name);
assertTextInTable(name, name);
assertTextInTable(name, birthdate);
assertTextInTable(name, type);
Data
name = "Snoopy"
birthdate = "1969-01-13"
type = "Dog"
Database
Testing
PetClinic from Below
DbUnit and Spring JDBC
API
TestCase DbUnit JDBC
Spring
API
TestCase JDBC
JDBC
www.dbunit.org
www.springframework.org
Spring JDBC Validator
ID NAME BIRTH_DATE TYPE_ID OWNER_ID
Table 42 Snoopy 1969-01-13 1 1
Code (part 1)
public class ValidatePetRow extends SpringJdbcValidator {
public ValidatePetRow(DataSource ds) {
super(ds,
"select * from PETS where NAME = ? and OWNER_ID = ?");
declareParameter(new SqlParameter(Types.VARCHAR));
declareParameter(new SqlParameter(Types.INTEGER));
compile();
}
Spring JDBC Validator
protected Object[] assembleParameters() {
return new Object[] { name, new Integer(ownerId) };
}
protected void validateRow(ResultSet rs, int rowNum) … {
…
DateAssert.assertDatesEqual("Wrong birthdate. ",
birthdate,
rs.getDate("BIRTH_DATE"));
Assert.assertEquals("Wrong pet type", typeId,
rs.getInt("TYPE_ID"));
}
protected void validateRowCount(int numRows) {
Assert.assertEquals(
"There should be one and only one pet row.",
1, numRows);
}
DbUnit Fixture Data
First Row
=
Column Name
Excel
Ranges
Sheet Name
=
Table Name
TestSteps
Or, what's up with
that name, anyway?
Test Steps
Add New Pet (OK) Navigators
1. Log in Gets you there
2. Click “Find Owner”
Composites
3. Enter Owner Lastname
4. Click “Find Owners” Executors
5. Click Owner Enter input
6. Click “Add New Pet” Validators
7. Validate defaults
Check output
8. Enter data
1. Pet Name
2. Pet Birthdate
3. Pet Type
9. Click “Add Pet”
10. Validate owner page
11. Validate database
Tests from Test Steps
PetFTest.testAddPet_Ok()
NavigateToAddPet NavigateToOwner
ValidatePetForm
ExecuteAddPet
ValidatePetOnOwnerPage
NavigateToFindOwner
ValidatePetRow
ExecuteFindOwner
PetFTest.testAddPet_Fail()
NavigateToAddPet
ValidatePetForm
ExecuteAddPet
ValidatePetErrorMsg
ValidateNoPetRow
JavaBeans
JWebUnitTestStep
DdSpringTestCase
SpringJdbcValidator
PetFTest ValidatePetRow
# nav : NavigateToAddPet # name : String
# valForm : ValidatePetForm # birthdate : java.sql.Date
# exeAddPet : ExecuteAddPet # typeId : int
# valRow : ValidatePetRow # ownerId : int
+ testAddPet_Ok() + runStep()
+ getNav() : NavigateToAddPet + setName(String)
+ getValForm() : ValidatePetForm + setBirthdate(Date)
+ getExeAddPet() : ExecuteAddPet + setTypeId(int)
+ getValRow() : ValidatePetRow + setOwnerId(int)
Column Header: valRow.name
valRow.birthdate
valRow.typeId
DDSteps
Springing It Together
The Big Picture
Spring Context
DDSpring
DataLoader
TestCase
FixtureLoader
ValidatePetRow DataSource
PetFTest
NavigateToAddPet
ValidatePetForm
ExecuteAddPet
ValidatePetOn WebBrowser
OwnerPage
DDSpringTestCase Flow
1. Spring Dependency Injection
2. Call setUpMethod()
3. Run "Excel Row 8"
1. Call setUpBeforeData()
2. DDSteps Property Injection
3. Call setUpAfterData()
4. Call testAddPet_Ok()
5. Call tearDownBeforeData()
4. Run "Excel Row 9"
5. …
6. Call tearDownMethod()
The TestCase
public class PetFTest extends DdSpringTestCase {
protected NavigateToAddPet nav;
protected ValidatePetForm valForm;
protected ExecuteAddPet exeAddPet;
protected ValidatePetOnOwnerPage valOwnerPage;
protected ValidatePetRow valRow;
public void setNav(…) {…}
public void testAddPet_Ok() throws Exception {
nav.runStep();
valForm.runStep();
exeAddPet.runStep();
valOwnerPage.runStep();
valRow.runStep();
}
Filling the Database
protected FixtureLoader fixtureLoader;
public void setFixtureLoader(...) {...}
protected Fixture fixture;
public void setUpMethod() throws Exception {
fixture = fixtureLoader.loadFixture(this);
fixture.setUp();
// DB is now filled with data
}
public void tearDownMethod() throws Exception {
fixture.tearDown();
fixture = null;
}
A Navigator TestStep
public class NavigateToAddPet extends JWebUnitTestStep {
protected NavigateToOwner navigateToOwner;
public NavigateToAddPet(WebBrowser webBrowser) {
super(webBrowser);
navigateToOwner = new NavigateToOwner(webBrowser);
}
public void runStep() throws Exception {
navigateToOwner.runStep();
setWorkingForm("formAddPet");
submit();
assertTextPresent("New Pet");
writeTrail("Add New Pet Form");
}
Spring Context
Spring Context
DDSpring
DataLoader
TestCase
PetFTest
Data Loader
ddsteps-context.xml
...
Database Stuff
Spring Context
DDSpring
DataLoader
TestCase
FixtureLoader
DataSource
PetFTest
Data Source
...
FixtureLoader
...
OWNERS, TYPES, PETS, VETS,
SPECIALTIES, VET_SPECIALTIES, VISITS
Database Stuff
Spring Context
DDSpring
DataLoader
TestCase
FixtureLoader
ValidatePetRow DataSource
PetFTest
NavigateToAddPet
ValidatePetForm
ExecuteAddPet
ValidatePetOn WebBrowser
OwnerPage
TestSteps
...
Adapt to your Environment
ddsteps-context.properties
jdbc.driverClassName=org.hsqldb.jdbcDriver
jdbc.url=jdbc:hsqldb:hsql://localhost:9001
jdbc.username=sa
jdbc.password=
url=http://localhost:8080/petclinic
trails=C:\trails
Best Practices
Make sure you can Reuse
Executors don't assert
Validators assert
Write trails
Data is Cheap
Happy flow
Bad data
Empty data
Comment data rows
One database per developer
Gotchas
Excel Column ≠ JavaBean Property
Bad Column Header?
Missing Property?
'F5' in Eclipse to refresh workspace
Everyday Life
...and how DDSteps will affect it
Cruise Control
Tag Update
Function
Test Implement
Deploy Run tests
Commit
Unit Test Detect
Build Changes
Working With Data Driven Testing
• Domain:
• Test Cases
• Test Steps
• Data
• Refactored Test Steps
• Technical:
• Test Cases
Tester • Test Steps Programmer
• Data
• Test Report
• Project Status
Project Leader
Real Regression Testing
Project
Development Test Staging Production
Environment
Community
www.ddsteps.org
jira.ddsteps.org
DDSteps Roadmap
Release 1.0 1.x Series
Out in September Smart Clients?
JMS?
Release 1.1 LDAP?
Community AJAX Support?
Documentation HtmlUnit?
Courtesy Classes OpenOffice?
2.x Series
JUnit 4?
TestNG?
Q&A
www.ddsteps.org
Lab!
Let's try DDSteps
My First DDSteps TestCase
public class StringComparator {
public boolean compare(String one,
String two) {
...
}
}
Brainstorm!