Docstoc

google - Download as PowerPoint

Document Sample
google - Download as PowerPoint Powered By Docstoc
					How Many Paths
     must a
Test Walk Down?


      Willem Visser
<wvisser@email.arc.nasa.gov>


          with
    Corina Pasareanu
<pcorina@email.arc.nasa.gov>

 NASA Ames Research Center

                               1
          What is Symbolic Execution?
✦ Let’s Ask You know who…




                                Oops, I guess I should know!




                                                               2
                     What is Symbolic Execution?
✦ Static Analysis Technique
   • Executes code in a non-standard way
✦ Instead of concrete inputs, symbolic values are manipulated
✦ At each program location, the state of the system is defined by
   • The current assignments to the symbolic inputs and local variables
   • A path condition that must hold for the execution to reach this location
✦ At each branch in the code, both paths must be followed
   • On the true branch: the condition is added to the path condition
   • On the false branch: the negation of the condition is added to the path condition
   • If a branch is infeasible, then execution along that branch is terminated




                                                                                         3
                          Symbolic Execution
                       Walking Many Paths at Once


                   [pres = 460;pres_min = 640;pres_max = 960]
                      if( (pres < pres_min) || (pres > pres_max)) {
                           …
                      } else {
                         …
                      }



       [pres = X;pres_min = MIN;pres_max = MAX] [PC: TRUE]
if ((pres < pres_min) ||          if ((pres < pres_min)) ||           if ((pres < pres_min) ||
    (pres > pres_max)) {              (pres > pres_max)) {                (pres > pres_max)) {
       …
    [PC: X< MIN]                         …
                                    [PC: X > MAX]                             …
} else {                         } else {                         } else {
      …                                …                                [PC:
                                                                        … X >= MIN &&
}                                }                                }         X <= MAX
                                                                                                 4
Concrete Execution Path (example)




                                    5
Symbolic Execution Tree (example)




                                    6
                 For this we believe…



“… symbolic execution for testing programs is a more exploitable
technique in the short term than the more general one of program
verification”

                                                  James King
                                                  CACM 19:7, 1976




                                                                    7
                Symbolic Execution Timeline




Khurshid, Pasareanu,Visser,
TACAS, 2003




James King,
CACM, 1976




                                              8
                    Generalized Symbolic Execution
                             void foo(Node n1, Node n2) {
                              if (n1 != null && n2 != null) {
                                n1.x = 2;
                                n2.x = 3;
Will this always hold?          assert n1.x == 2 && n2.x == 3;
                             }}


✦ Deals with Object-oriented programs
    • Fields
    • Inheritance
    • Aliasing!
✦ Uses lazy initialization of Objects
    • Only creates objects when they are required
    • No bound on the number of objects
✦ Concurrency


                                                                 9
          Observation from King Paper


       “The symbolic execution of IF statements requires
theorem proving which, even for modest programming languages,
                 is mechanically impossible.”


    Although true, it didn’t account for two things:
    • Moore’s Law
    • Improvements in decision procedure efficiency
       • Driven by the great strides made by SAT solvers




                                                                10
                  Decision Procedures Everywhere
✦ Satisfiability Modulo Theories Competition (SMT-COMP)
   • Competition between DPs for all kinds of theories (integer, float, bitvectors etc.)
✦ Participants in 2006:
   •   Ario 1.2 website, description, download
   •   Barcelogic 1.1 website, description, download
   •   CVC website, description, download
   •   CVC3 website, description, download
   •   ExtSAT website, description, download
   •   HTP website, description, download
   •   Jat description download
   •   MathSAT 3.4 website, description, download
   •   NuSMV website, description, download
   •   Sateen description, download
   •   STP description, download
   •   Yices 1.0 website, description, download
✦ Doesn’t even include commercial bigshots such as Prover Technology


                                                                                           11
                     Symbolic Execution Everywhere

✦ Static Analysis
    • Prefix
        ‣ Intrinsa, now Microsoft

✦ Testing
    • DART and SMART
        ‣ Lucent
    • JTest
        ‣ Parasoft

✦ Model Checking
    • XRT
        ‣ Microsoft
    • JPF
        ‣ NASA

✦ Only listed industry and this is a woefully inadequate list at best!




                                                                         12
    Example using Java PathFinder
public class SymExample {

 // Node has an “int x” field and a “Node next” field
 public static void foo(Node n1, Node n2) {
    if (n1 != null && n2 != null) {
      n1.x = 2;
      n2.x = 3;
      assert n1.x == 2 && n2.x == 3;
 }}

 public static void main(String[] args) {
  Node n1 = Node._get_Node(); // get Symbolic Node
  Node n2 = Node._get_Node(); // get Symbolic Node
  try {
     foo(n1, n2);
    System.out.println("No violation for n1 = " + n1 + " and n2 = " + n2);
  } catch (AssertionError ae) {
    System.out.println("Found violation for n1 = " + n1 + " and n2 = " + n2);
}}}




                                                                                13
                   Example Output

No violation for n1 = null and n2 = null
No violation for n1 = null and n2 = Node_43844 = (x = *, next = *)
No violation for n1 = Node_43844 = (x = *, next = *) and n2 = null

No violation for n1 = Node_43844 = (x = *, next = *) and
                 n2 = Node_43791 = (x = *, next = *)

Found violation for n1 = Node_43844 = (x = *, next = *) and
                    n2 = Node_43844 = (x = *, next = *)



          if (n1 != null && n2 != null) {
                n1.x = 2;
                n2.x = 3;
                assert n1.x == 2 && n2.x == 3;
          }


                                                                     14
                 What is Model Checking?




Oh no, we got downgraded




                                           15
                           Analogy
         The Web                                 Java Program

                                        void foo(Node n1, Node n2) {
                                         if (n1 != null && n2 != null) {
                                           n1.x = 2;
                                           n2.x = 3;
                                           assert n1.x == 2 && n2.x == 3;
                                        }}
                                                                            Path to
                                                                            Error
                                  Model Checking
                                searching reachable
                    Requested   states for violations
Web Search          Page

                                                                        Error
                                                         Property
                                                        (no deadlock)

       User Query                                                                 16
                     What is Java PathFinder (1)
✦ explicit state model checker for Java bytecode
✦ focus is on finding bugs in Java programs
   • concurrency related: deadlocks, (races), missed signals etc.
   • Java runtime related: unhandled exceptions, heap usage, (cycle budgets)
   • but also: complex application specific assertions




                                                                               17
                     What makes JPF interesting?
✦ It can search the behaviors of a Java program in an efficient fashion
   • Records all visited states and stops traversing a path when it revisits a state
   • Uses various search heuristics
✦ It treats non-deterministic behavior
   • Scheduling is non-deterministic in many Java settings
   • The precise inputs to a program can be non-deterministic
✦ Data non-determinism is user definable
   • User can define the environment the program executes in
✦ However, to make JPF scale we also need symbolic input data
   • Hence the extensions to do symbolic execution
   • Without symbolic execution the tool enumerates all input values, which is often too large

✦ One of the key issue is configurable extensibility: overcome scalability
  constraints with suitable customization (using heuristics)



                                                                                                 18
                                     JPF Status
✦ developed at the Robust Software Engineering Group at NASA Ames
  Research Center
✦ currently in it’s fourth development cycle
   •   v1: Spin/Promela translator - 1999
   •   v2: backtrackable, state matching JVM - 2000
   •   v3: extension infrastructure (listeners, MJI) - 2004
   •   v4: symbolic execution, choice generators - 4Q 2005
✦ open sourced since 04/2005 under NOSA 1.3 license:
  <javapathfinder.sourceforge.net>
✦ it’s a first: no NASA system development hosted on public site before
✦ 11100 downloads since publication 04/2005




                                                                          19
Implementation via Instrumentation




                                     20
Handling Aliasing (illustration)




                                   21
                   Symbolic Execution in Testing
✦ Symbolic execution gives constraints on the inputs to reach a specific
   •   Statement
   •   Branch
   •   Condition
   •   Etc.
✦ Using an instrumented program that will save the test inputs whenever
  the testing criterion is reached will allow automatic test generation




                                                                           22
                                 ISSTA 2006
✦ Generate test inputs for Java container classes using sequences of API
  calls
   • Random, Model Checking, Symbolic Execution, etc.
✦ Objective was to generate test cases to cover basic blocks and predicates
                                          Branch Number      Predicate Values
             Unique ID for the test

      Test case number 77 for '15,L+R+P-REDroot':
      put(0);put(4);put(5);put(1);put(2);put(3);remove(4);

 Test-case to achieve above coverage

                           Test case number 7 for '32,L-R-P+RED':
             Concrete
                           put(1);put(0);remove(0);
            Symbolic       Test case number 7 for '32,L-R-P+RED':
       Path Condition
                           X2(0) == X1(0) &&
       with solutions      X2(0) < X0(1) &&
                           X1(0) < X0(1)
    Symbolic TC            put(X0);put(X1);remove(X2);                          28
           Statement coverage is harder than you think
✦ One of the basic blocks in the Binomial Heap implementation required a
  minimum sequence of 13 API calls to be covered
private void merge(BinomialHeapNode binHeap) {
  BinomialHeapNode temp1 = Nodes, temp2 = binHeap;
  while ((temp1 != null) && (temp2 != null)) {          X4(1) >= X8(1) && X10(2) > X8(1) &&
    if (temp1.degree == temp2.degree) {                 X10(2) <= X11(2) && X11(2) > 0 &&
       BinomialHeapNode tmp = temp2;                    X10(2) > 0 && X8(1) <= X9(1) &&
       temp2 = temp2.sibling;                           X9(1) > 0 && X8(1) > 0 && X4(1) <= X2(1) &&
       tmp.sibling = temp1.sibling;                     X6(2) > X4(1) && X6(2) <= X7(2) &&
       temp1.sibling = tmp;                             X7(2) > 0 && X6(2) > 0 && X4(1) <= X5(1) &&
       temp1 = tmp.sibling;                             X5(1) > 0 && X4(1) > 0 && X2(1) <= X0(1) &&
    } else {                                            X2(1) <= X3(1) && X3(1) > 0 && X2(1) > 0 &&
      if (temp1.degree < temp2.degree) {                X0(1) <= X1(1) && X1(1) > 0 && X0(1) > 0
         if ((temp1.sibling == null) ||                 insert(X0);insert(X1);insert(X2);insert(X3);insert(X4);
             (temp1.sibling.degree > temp2.degree)) {   insert(X5);insert(X6);insert(X7);insert(X8);insert(X9);
            // HERE!                                    insert(X10);insert(X11);extractMin();
            …




                                                                                                                  29
                                     Orion Onboard Abort Executive
✦ During ascent the OAE decides if an abort should occur
         • It encodes 27 flight rules for when an abort should occur, and if one gets triggered
         • It decided which of 7 aborts it should pick
✦ We applied JPF’s symbolic execution to a prototype
         • Within 1 min it finds 23 tests that cover all the flight rules and all 7 of the aborts
         • In addition it found a case in which an abort should have occurred but none was selected
              See constraint below to get to this scenario!
inputs_fr.navIN_fr.geod_alt(300000) >= 300000 && inputs_fr.navIN_fr.geod_alt(300000) >= 120000 && inputs_fr.navIN_fr.geod_alt(300000) >= 38000 && inputs_fr.navIN_fr.geod_alt(300000) >=
10000 && inputs_fr.sysIN_fr.cev_cm_cabin_pres_rate(-1) >= -1 && inputs_fr.sysIN_fr.cev_cm_cabin_pres_rate(-1) >= -2 && inputs_fr.navIN_fr.las_jettison_cmd(-1000000) != 1 &&
inputs_fr.navIN_fr.roll_rate(40) <= 50 && inputs_fr.navIN_fr.yaw_rate(31) <= 41 && inputs_fr.navIN_fr.pitch_rate(70) <= 100 && inputs_fr.navIN_fr.las_jettison_cmd(-1000000) != 0 &&
inputs_fr.sysIN_fr.stage2_apu_volt(22) <= 33 && inputs_fr.sysIN_fr.stage2_apu_volt(22) >= 22 && inputs_fr.navIN_fr.vmissmag(5) <= 10 && inputs_fr.sysIN_fr.stage2_thrust(221922) <= 332883
&& inputs_fr.sysIN_fr.stage2_thrust(221922) >= 221922 && inputs_fr.sysIN_fr.stage2_helium_tnk_pres(560) <= 840 && inputs_fr.sysIN_fr.stage2_helium_tnk_pres(560) >= 560 &&
inputs_fr.sysIN_fr.stage2_lh2_tnk_pres(26) <= 40 && inputs_fr.sysIN_fr.stage2_lh2_tnk_pres(26) >= 26 && inputs_fr.sysIN_fr.stage2_lox_tnk_pres(17) <= 25 &&
inputs_fr.sysIN_fr.stage2_lox_tnk_pres(17) >= 17 && inputs_fr.sysIN_fr.stage2_hpft_speed(28288) <= 42432 && inputs_fr.sysIN_fr.stage2_hpft_speed(28288) >= 28288 &&
inputs_fr.sysIN_fr.stage2_lpft_speed(12948) <= 19422 && inputs_fr.sysIN_fr.stage2_lpft_speed(12948) >= 12948 && inputs_fr.sysIN_fr.stage2_hpot_speed(22496) <= 33744 &&
inputs_fr.sysIN_fr.stage2_hpot_speed(22496) >= 22496 && inputs_fr.sysIN_fr.stage2_lpot_speed(4120) <= 6180 && inputs_fr.sysIN_fr.stage2_lpot_speed(4120) >= 4120 &&
inputs_fr.sysIN_fr.stage2_efi_pres(4800) <= 7200 && inputs_fr.sysIN_fr.stage2_efi_pres(4800) >= 4800 && ((inputs_fr.sysIN_fr.stage1_tvc_actual(0) MINUS
inputs_fr.sysIN_fr.stage1_tvc_commanded(0)) MULT 10) <= (inputs_fr.sysIN_fr.stage1_tvc_commanded(0) MULT 3) && inputs_fr.sysIN_fr.stage1_chmbr_pres(640) <= 960 &&
inputs_fr.sysIN_fr.stage1_chmbr_pres(640) >= 640 && theQueue.q[7].spred.outputs_p.vgo_Mag(115) <= 515 && theQueue.q[7].spred.outputs_p.vgo_Mag(115) >= 115 &&
theQueue.q[0].spred.outputs_p.vmissmag_eo_now(410) <= 610 && theQueue.q[0].spred.outputs_p.vmissmag_eo_now(410) >= 410 && inputs_fr.navIN_fr.yaw_rate(31) <= 51 &&
inputs_fr.navIN_fr.yaw_rate(31) >= 31 && inputs_fr.navIN_fr.yaw(100) <= 140 && inputs_fr.navIN_fr.yaw(100) >= 100 && inputs_fr.navIN_fr.roll_rate(40) <= 60 &&
inputs_fr.navIN_fr.roll_rate(40) >= 40 && inputs_fr.navIN_fr.roll(100) <= 140 && inputs_fr.navIN_fr.roll(100) >= 100 && inputs_fr.navIN_fr.pitch_rate(70) <= 130 &&
inputs_fr.navIN_fr.pitch_rate(70) >= 70 && inputs_fr.navIN_fr.pitch(45) <= 65 && inputs_fr.navIN_fr.pitch(45) >= 45 && inputs_fr.navIN_fr.vmissmag(5) <= 15 &&
inputs_fr.navIN_fr.vmissmag(5) >= 5 && inputs_fr.navIN_fr.inert_vel_mag(22000) <= 26000 && inputs_fr.navIN_fr.inert_vel_mag(22000) >= 22000 && inputs_fr.navIN_fr.geod_alt(300000) <=
310000 && inputs_fr.navIN_fr.geod_alt(300000) >= 0 && inputs_fr.sysIN_fr.stage2_thrust(221922) <= 342883 && inputs_fr.sysIN_fr.stage2_thrust(221922) >= 211922 &&
inputs_fr.sysIN_fr.stage2_lpot_speed(4120) <= 6680 && inputs_fr.sysIN_fr.stage2_lpot_speed(4120) >= 3620 && inputs_fr.sysIN_fr.stage2_lpft_speed(12948) <= 20422 &&
inputs_fr.sysIN_fr.stage2_lpft_speed(12948) >= 11948 && inputs_fr.sysIN_fr.stage2_lox_tnk_pres(17) <= 30 && inputs_fr.sysIN_fr.stage2_lox_tnk_pres(17) >= 12 &&
inputs_fr.sysIN_fr.stage2_lh2_tnk_pres(26) <= 45 && inputs_fr.sysIN_fr.stage2_lh2_tnk_pres(26) >= 21 && inputs_fr.sysIN_fr.stage2_hpot_speed(22496) <= 35744 &&
inputs_fr.sysIN_fr.stage2_hpot_speed(22496) >= 20496 && inputs_fr.sysIN_fr.stage2_hpft_speed(28288) <= 30288 && inputs_fr.sysIN_fr.stage2_hpft_speed(28288) >= 26288 &&
inputs_fr.sysIN_fr.stage2_helium_tnk_pres(560) <= 1040 && inputs_fr.sysIN_fr.stage2_helium_tnk_pres(560) >= 360 && inputs_fr.sysIN_fr.stage2_efi_pres(4800) <= 7700 &&
inputs_fr.sysIN_fr.stage2_efi_pres(4800) >= 4300 && inputs_fr.sysIN_fr.stage2_apu_volt(22) <= 48 && inputs_fr.sysIN_fr.stage2_apu_volt(22) >= 7 &&
inputs_fr.sysIN_fr.stage1_tvc_commanded(0) <= 10 && inputs_fr.sysIN_fr.stage1_tvc_commanded(0) >= 0 && inputs_fr.sysIN_fr.stage1_tvc_actual(0) <= 10 &&
inputs_fr.sysIN_fr.stage1_tvc_actual(0) >= 0 && inputs_fr.sysIN_fr.stage1_chmbr_pres(640) <= 1160 && inputs_fr.sysIN_fr.stage1_chmbr_pres(640) >= 440 &&
inputs_fr.sysIN_fr.cev_cm_cabin_pres_rate(-1) <= 0 && inputs_fr.sysIN_fr.cev_cm_cabin_pres_rate(-1) >= -3 &&
                                                                                                                                                                                             30
                      Did someone say “Loops”?
✦ How do we terminate our symbolic analysis?
✦ We use the following termination conditions
   • Limit the depth that the model checker searches to
   • Limit the number of calls made from the environment
   • Limit the size of structures required
✦ Model checkers work from one core idea, called state matching
   • Whenever you see a state you have visited before, you stop traversing that path
✦ However, doing state matching over symbolic states that include a path
  condition is very hard
   • We implemented a subsumption check for symbolic states over container structures
        ‣ “Test Input Generation for Java Containers using State Matching”
          W. Visser, C. Pasareanu, R. Pelanek, ISSTA 2006
   • We also developed a heuristic for “widening” to compute loop invariants
        ‣ “Verification of Java Programs Using Symbolic Execution and Invariant Generation”
          C. Pasareanu and W. Visser, SPIN 2004




                                                                                              31
           Decision Procedure Support


                             JPF

                Formula                satisfiable/unsatisfiable



              Generic Decision Procedure Interface




Omega         CVCLite                       STP                    Yices
Maryland      Stanford                    Stanford                  SRI
                                     A New Idea
✦ Path-sensitive Static Analysis tools for finding runtime errors are now
  very popular
    • Coverity, KlocWork, etc.
✦ This new generation of tools can miss bugs
    • Path sensitive analyses tend to have this issue
✦ But it is much better at false positive warnings than the old abstract
  interpretation style defect detectors
    • That will never miss a bug, but it floods you with warnings that the user must classify as bugs
      or not
✦ However, wouldn’t it be nice to have actual test inputs to tell you if
  something is a bug or not.
✦ We combined symbolic execution and test input generation to do this
    • Joint work with Aaron Tomb from UCSC




                                                                                                    33
                 Codename Jitterbug

        Java            ✦ Starts symbolic analysis at each method
       classes            in the class file(s)


Symbolic Execution
                        ✦ Symbolic execution detects a possible
 [SOOT + CVCL]            error; passes it and the symbolic state to
Warnings
                          the test generator


 Test Generation        ✦ From the current state and the path
                          condition generate a test to try and cover
    [POOC]                the error
Test Cases
                        ✦ Execute the test and check if the
                          expected error is triggered
    Test Cases



                                                                       34
               Symbolic Execution Benefit


public class Example {
  public String hexAbs(int x) {
    String result = null;
    if(x > 0)                              Dataflow Analysis
      result = Integer.toHexString(x);    Warning: possible null
    else if(x < 0)                         dereference on line 8
      result = Integer.toHexString(-x);
    return result.toUpperCase();
  }                                        Symbolic Execution
}                                         Error: null dereference
                                             on line 8 if x = 0




                                                                    35
                              Small Example
public class ArrayBound {
 public void f(int n, int m) {
   int[] array = new int[m];
                                   WARNING: possible array upper bound violation (f:5)
   for(int i = 0; i < n; i++) {
                                   Symbolic state at time of warning:
     array[i] = 0;                   Method: <ArrayBound: void f(int,int)>
}}}                                  Instruction: array[i] = 0
                                     Line number: 5
                                     Depth: instruction = 6, branch = 1, pc = 4
                                     Path condition: [U1 >= 0, 0 < U0, U1 = len(A),
                                                       0 >= len(A0), len(A0) >= 0]
                                     Parameter values: [U0, U1]
                                     This object: o0
                                     Local vars: [i=0, m=U1, n=U0, this=o0, array=A0]
                                   Solution (1): this = o0, param0 = 1, param1 = 0
Running 5 test(s)...
2) Solution (1)
Testing ArrayBound.f
  REAL? Caught expected exception: java.lang.ArrayIndexOutOfBoundsException: 0
 Occurred at ArrayBound.f:5
                                                                                     36
                            Variably Interprocedural
 ✦ Experience with runtime error detection tools suggest that a very large
   percentage of errors are intraprocedural
 ✦ However, for object-oriented programs it is common to access instance
   fields through accessor methods
 ✦ We made the level of interprocedural analysis an input to the tool
                    public void foo(int m) {
                       m = answer(m);
                       m = m/(1-m);
                    }
                    private int answer(int v) { return v == 42 ? 1 : 0; }
DEPTH 0                                              DEPTH 1
WARNING: possible division by zero (foo:6)           ERROR: certain division by zero (foo:6)
Symbolic state at time of warning:                   Symbolic state at time of error:
  Instruction: m = m / $i0                             Instruction: m = m / $i0
  Path condition: [1 - U1 = 0]                         Path condition: [U0 = 42]
  Parameter values: [U0]                               Parameter values: [U0]
   No vars: [m=U0, – pick randomly
  Local Constraint m=U1, $i0=1 - U1, this=o0]          Local vars: [m=U0, m=1, $i0=0, this=o0]

Solution (0): param0 = U0, this = o0                 Solution (0): this = o0, param0 = 42        37
                 Deals with Objects and Fields

public class Node {                 WARNING: possible null dereference (swap:8)
 public int value;                  Symbolic state at time of warning:
 public Node next;                    Method: <NodeSwap: NodeSwap swap()>
                                      Instruction: $i1 = $r0.<NodeSwap: int value>
                                      Line number: 8
    public Node swap() {              Depth: instruction = 3, branch = 0, pc = 1
      if (next != null)               Path condition: [U1 = null]
        if (value > next.value) {     Parameter values: []
          Node t= next;               This object: o0
          next = t.next;              Local vars: [this=o0, $i0=U0, $r0=U1]
          t.next = this;              Initial field values: [o0.value=U0, o0.next=U1]
                                      Current field values: [o0.value=U0, o0.next=U1]
          return t;
        }                           Solution (0):
      return this;                  this.value = -1000000
    }                               this.next = null
}                                   this = o0


                                                                                   38
                  Deals with Objects and Fields (2)

public class Node {                 WARNING: possible division by zero (swap:12)
                                    Symbolic state at time of warning:
 public int value;                    Method: <NodeSwap: NodeSwap swap()>
 public Node next;                    Instruction: x = 10 / $i1
                                      Line number: 12
                                      Depth: instruction = 13, branch = 2, pc = 3
    public Node swap() {              Path condition: [U0 /= null, U1 > U2, 5 - U1 = 0]
      if (next != null)               Parameter values: []
                                      This object: o0
        if (value > next.value) {     Local vars: [$i1=5 - U1, $r0=U0, this=o0, $i0=U1, $i2=U1,
          Node t= next;                              $r1=U3, t=U0]
          next = t.next;              Initial field values: [o1.value=U2, o0.next=U0, o0.value=U1,
                                                             o1.next=U3]
          t.next = this;              Current field values: [o1.value=U2, o0.next=U3, o0.value=U1,
          int x = 10/(5-value);                                o1.next=o0]
          return t;                   Unknown value mappings: [U0=o1]
        }                           Solution (1):
      return this;                  this.next.next = U3
    }                               this.next.value = -1000000
                                    this.value = 5
}                                   this.next = o1
                                    this = o0
                                                                                                     39
                  Doesn’t do Full Aliasing yet
                       void foo(Node n1, Node n2) {
                         if (n1 != null && n2 != null) {
                           n1.x = 2;
                           n2.x = 3;
                           assert n1.x == 2 && n2.x == 3;
                        }
                       }

✦ Cannot find the bug here, since it will not consider the case where
  n1 = n2
✦ To handle this we need special paths that consider possible aliasing
  as we did in the JPF case
✦ This will impact the performance
✦ We need to study the trade-offs


                                                                         40
                        Efficient Array Bounds Checking
✦ Here everything is concrete in the symbolic execution, hence it cannot find the
  violation in the code
✦ We add an acceleration heuristic to find the bug here
       • For conditions such as “i < n” we add a new assignment to set “i = n-1” to accelerate the loop to its boundary
         condition
       • Side-effect is to make “i” symbolic

                                                     WARNING: possible array upper bound violation (k:6)
                                                     Symbolic state at time of warning:
public void k(int n) {
                                                       Method: <ArrayBound2: void k(int)>
   int[] array = new int[100];                         Instruction: l2[l3] = 0
                                                       Line number: 6
                                                       Depth: instruction = 4, branch = 1, pc = 4
       for (int i = 0; i < n; i++) {                   Boundary hack: true branch (false), false branch (true)
           array[i] = 0;                               Path condition: [0 < U0, U0 - 1 >= len(A0), len(A0) >= 0]
       }                                               Parameter values: [U0]
                                                       This object: o0
   }                                                   Local vars: [l3=U0 - 1, l0=o0, l2=A0, l1=U0]

                                                     Solution (0):
                                                     this = o0
                                                     param0 = 101
                                                                                                                          41
                          Termination Revisited
✦ This is the #1 issue in path sensitive analyses
✦ Set the termination criteria too weak then you miss bugs
✦ Set it too precise then the analysis runs too long
✦ At first we counted the number of times an instruction got executed
  across all paths
   • This criteria was too weak and it missed many bugs
✦ We currently use size of the path condition
   • This works well, but it can lead to many warnings for the same error
✦ We plan to use abstraction based termination conditions, such as
  predicate abstractions, as proposed in
   • “Concrete Model Checking with Abstract Matching and Refinement”
     Corina Pasareanu, Radek Pelanek and Willem Visser
✦ The exact trade-offs for the termination conditions must still be studied


                                                                              42
                                           Current State
✦ Absolutely zero effort has gone into making the tool fast
✦ Finds all the same errors as Check’n’Crash and a few more
    • Christoph Csallner (Georgia Tech, and Google Summer intern this past year)
✦ Ran it on some large NASA source bases
    • Found numerous errors
    • Quick study showed some are real
✦ Note not all errors reported are reachable
    • Not considering pre-conditions that are not explicit in the code
✦ Not all warnings will have tests to show the error
    • Path conditions only constrain explicit inputs and the values of other variables might be important too, e.g. in the
       code below there is a bug only if the number of files in the directory is greater than 10 – but the test case has no
       control over the number of files:
                      File dataDir = new File(dataDirName);
                      File[] conflicts = dataDir.listFiles();
                      TesterThread[] threadList = new TesterThread[10];
                      for(int i=0;i<threadList.length;++i) {
                       FileInputStream fis = new FileInputStream(conflicts[i]);
✦ In general we use a random test generator to try and create valid data if we have no
  constraints on some variable
                                                                                                                              43
                                Future Work



             Call Depth                 Aliasing     Termination

✦   Turn the dials to find the sweet spot between accuracy and speed
✦   Address the uncontrolled environment problem
✦   Improve efficiency
✦   Integrate with Eclipse and JUnit
✦   Compare with commercial tools
    • Both KlocWork and Coverity supports Java now




                                                                       44
                                     Conclusions
✦ Symbolic Execution is a powerful technique for doing advanced testing
✦ Showed two different tools supporting symbolic exeution
   • Java PathFinder
        ‣ Whole program analysis
        ‣ Every error discovered is a real error
        ‣ Stops when it runs into a constraint none of its decision procedures can deal with
   • Codename JitterBug
        ‣ Variably interprocedural
        ‣ Not every warning is a real bug, not even every failed test indicates a real bug
        ‣ Can find errors even when the decision procedure fails because it assumes feasibility

✦ Main research goal is to combine symbolic execution with concurrency
  analysis
   • Concurrency errors are hard to catch
        ‣ Model checkers are about as good as it gets in this domain
   • Model checking by itself cannot deal with all types of programs
   • Static analysis based techniques will be required




                                                                                                  45

				
DOCUMENT INFO