A Parameterized Cost Model to
Order Classes for Class-based
Testing of C++ Applications
B. A. Malloy, P. J. Clarke,
and E. L. Lloyd
outline
Part I: ISSRE paper – obtaining a class
order for testing
Part II: Using keystone to analyze C++
programs
2
When to test?
Can find faults earlier if we test classes
rather than the system
But classes interact with other classes
A A
B C B C
3
When to test?
Can find faults earlier if we can test at
the class level rather than system level
But classes interact with other classes
Stub
A A For A
B C B C
Goal: find an order for testing classes; minimize stubs 4
Construct graph or ORD
No cycles: reverse topological order
Cycles: remove an edge based on cost
model:
Weight edge based on edge type
Combine edges (and weights) if source and
destination coincide
5
overview
6
Edge types – based on UML
Our ORD has 6 edges:
Inheritance
Polymorphic
OwnedElement
Composition
Association
Dependency
7
Edge types
class Shape { };
class Circle : public Shape {};
class Triangle : public Shape {};
D
class Figure { Figure Shape
A
public:
C I I
void print(Shape *) {}
P
private: Circle Triangle
Shape * y;
Circle x; P
};
previous research:
(1) every cycle has association edge
(2) remove association edge to break cycle
(3) never remove inheritance edge 8
A template class is not a class
class A {};
template
class S{
B S
T x; C
}; C
Class B { A
S m;
};
9
Myth: every cycle has association edge
class A;
class B {
A * x; A
}; B A
C
class A {
B y;
};
10
Curiously recursive template pattern
template
class ObjectCounter {
public:
static size_t live() { return count; }
protected: ObjectCounter
ObjectCounter() { ++count; }
ObjectCounter(ObjectCounter const&) {
++count; I D
}
~ObjectCounter() { --count; }
void fun(T t) { t.print(); }
private: MyString
static size_t count;
};
template
class MyString : public ObjectCounter > {
};
Compliments of Vandevoorde & Josuttis
11
Cost model
ORD G = (V, E), E = V x V
C =
W = {w1, w2, w3, w4, w5, w6}
f : E -> W
mx,y = {(x,y) in E}
W(mx,y) = S f(e)
e e mx,y
12
COS Algorithm
1. Build ORD G
- assign weights to edges
- merge edges
2. Assign reverse DF numbering
3. Build GR
4. Partition GR into SCC
5. Break cycles:
- choose a SCC, c, with more than 1 node
- remove edge in c with smallest weight
- repeat step 2 to find SCC in c
- repeat step 3 until all SCC have 1 node
6. Convert GR into G
4. Find reverse topological sort of reduced G
13
The Class Ordering System - COS
14
Test suite
15
How many SCCs
16
Nodes and edges in SCCs
17
18
Threats to validity
Test cases are not large
Only one test case used templates
Weights chosen arbitrarily
Markov distribution
Metric + edge type
19
Contributions
It’s false that every cycle has association edge
It’s false that by removing only association edges you
will break cycles
Our work is first to accommodate C++ templates
We show that programs with hundreds of classes can
have thousands of edges
We question a widely held assumption: never remove
inheritance edge
Removing inheritance edges may reduce total edges
(stubs) by factor of 2 or more
20
Using keystone for analysis
T. Gibbs, J. Power and B. Malloy
Compact parser and front-end
With API and ASG
(fits on a floppy)
21
Overview of keystone
Parser & front-end for C++
New approach to disambiguation
Uses flex, btyacc and token-decorated
parsing
Current state of keystone:
Parser& AST can handle all of ISO C++
ASG not finished for templates
22
Traditional approaches to
disambiguation
Rewrite the grammar
Use parser directives, such as
precedence or associativity in
yacc/bison
Yacc/bison: make the default action the
right one!
23
Token Decorated Parsing
Use semantic information, acquired from
symbol table, to give more info to parser
Hints in Brown Parser
James Power and I came up with the
idea, design and initial implementation
Dr. Gibbs made parser work for ISO C++
24
Modern Languages
Larger scale
Intricate constructs
Difficult to parse
typedef int f; void f(float x);
float x;
f ( x ); f ( x );
25
For most languages:
Two Grammars
ANSI or ISO committee grammars are
Easy to read
Correct
Difficult to parse
Traditional disambiguation requires
grammar modification
Java and C have 2 grammars
Dialects
26
One grammar for keystone
Using token decorated parsing: ISO C++
grammar
Only one change to the grammar
27
Problems Testing & Validating
a parser or front-end
Grammar and actions are separate
Very little to no intermediate output makes
visual validation difficult (Oracle)
Circular references are hard to validate
28
Token decorated parsing
29
Classic Parsing
getToken()
Parser Scanner
30
Token Decorated Parsing
getToken() getToken()
Token
Parser Scanner
Decorator
token>
31
Token-decorated Parsing
Name lookup facilitates grammar
recognition
Augment lexical token with semantic info
Must be able to do name lookup
Typedef int f; Template
f ( x ); class A {};
A C
32
What token gets decorated:
IDENTIFIER
ID_typedef_name
ID_original_namespace_name
ID_namespace_alias
ID_enum_name
ID_class_name
ID_template_name
33
Many productions use IDENTIFIER
type_name type_name
: class_name : class_name
| enum_name | enum_name
| typedef_name | typedef_name
; ;
type_def_name type_def_name
: IDENTIFIER : ID_typedef_name
; ;
34
Current Keystone Design
35
Clouseau
An API for easy access to keystone
symbol table
Info about names in C++ programs
Implemented as a unix shared object
Containers for info are vectors
36
C++ programs
Composed of namespaces that can
contain:
Namespaces
Classes
Variables
Functions
Declarations & includes (files)
37
Keystone symbol table
The keystone symbol table is a
collection of names (NameDeclaration)
and scopes (Scope)
The Clouseau API is symmetric with the
keystone symbol table
38
Scopes
39
Interface for Clouseau
The outer level of the API is a class,
Interface, consisting of a vector of
namespaces
Each element in the vector is an
instance of NameSpace_Info
40
NameSpaces
Scope
string getName()
string name;
NameSpace_Info
Vector nameSpaceDefs()
Vector classDefs()
Vector freeFunDefs()
Vector variables()
41
Classes
Scope
string getName()
string name;
Class_Info
Vector funDefsInClass()
Vector classDefsInClass()
Vector parents()
AccessSpecifier visibility()
AccessSpecifier classDerivedAccess(Class_Info)
42
Variables
Scope
string getName()
string name;
Variable_Info
AccessSpecifier visibility()
DataType getType()
43
Type information
DataType
string getName()
Type_Info getCategory()
bool isUserDefined()
44
SWIG
Simplified Wrapper and Interface Generator
Subliminal message
On title bar of
David M. Beazley Home page:
www.swig.org
University of Utah
45
What is it?
Easily & quickly build scripting language
interfaces to C, C++ or Objective-C
Takes C/C++ programs and wraps them
for scripting languages: Java, Python,
Perl, …
Requires no modification to existing code
46
SWIG enables
Construct interface to C programs
Rapid prototyping
Adding a GUI to C programs
Use scripts to test C (cpptest, pytest,
Jtext)
Make C programming more enjoyable
47
C/C++ program Interface File
.c .i
SWIG Parser
Code Documentation
Generator Generator
Tcl CHICKEN Java Latex
Perl C# Ocaml HTML
Python Ruby Tcl/Tk ASCII
Guile PhP MzScheme 48
Some stats about keystone
49
Scale of Keystone
50
The Parser:
Conformance to ISO standard
Compiler 3 4 5 6 7 8 9 10 11 12 13 14 15 Failures % Passed
keystone 0 0 0 0 0 0 0 0 0 0 0 3 0 3 99.3
gcc 3.2 6 0 0 0 1 4 1 0 2 4 0 9 0 27 94.0
gcc 3.0 6 0 0 0 1 4 1 0 2 4 0 9 0 27 94.0
MIPS 6 0 0 1 1 3 2 0 0 5 1 12 0 31 93.0
Pro
gcc 2.95 6 0 0 0 4 4 1 0 1 4 0 12 1 33 92.6
VC++ 14 0 3 2 3 11 6 6 3 11 15 49 0 123 72.7
6.0
51
The ASG:
Standard Conformance
Compiler 3 4 5 6 7 8 9 10 11 12 13 14 15 Failures % Passed
keystone 0 0 1 0 4 2 2 0 0 3 1 19 0 32 92.9
gcc 3.2 6 0 0 0 1 4 1 0 2 4 0 9 0 27 94.0
gcc 3.0 6 0 0 0 1 4 1 0 2 4 0 9 0 27 94.0
MIPS 6 0 0 1 1 3 2 0 0 5 1 12 0 31 93.0
Pro
gcc 2.95 6 0 0 0 4 4 1 0 1 4 0 12 1 33 92.6
VC++ 14 0 3 2 3 11 6 6 3 11 15 49 0 123 72.7
6.0
52
Keystone ASG
Purpose
Allow user-friendly access to detailed keystone
information
Usability of the parser
Design Goals
Modularized
User friendly
53
ASG Nodes: Datrix
54
ASG Visitors
55