Symbolic Execution and Software Testing - NASA_2_

Document Sample
Symbolic Execution and Software Testing - NASA_2_ Powered By Docstoc
					  Symbolic Execution with Mixed
   Concrete-Symbolic Solving


Corina Pasareanu1, Neha Rungta2 and Willem Visser3
      1Carnegie Mellon, 2SGT Inc./NASA Ames
            3University of Stellenbosch
                                Symbolic Execution
—   Program analysis technique
    —   King [Comm. ACM 1976] , Clarke [IEEE TSE 1976]
    —   Executes a program on symbolic inputs
    —   Maintains path condition (PC) – checked for satisfiablity with decision procedures

—   Received renewed interest in recent years due to
    —   Algorithmic advances
    —   Increased availability of computational power and decision procedures

—   Applications:
    —   Test-case generation, error detection, …

—   Tools, many open-source
    —   UIUC: CUTE, jCUTE, Stanford: EXE, KLEE, UC Berkeley: CREST, BitBlaze
    —   Microsoft’s Pex, SAGE, YOGI, PREfix
    —   NASA’s Symbolic (Java) Pathfinder
    —   IBM’s Apollo, Parasoft’s testing tools etc.
void test(int x, int y) {
  if (x > 0) {              S0, S1, S3, S4 =
    if (y == hash(x))       statements we wish to cover
      S0;
    else                    Symbolic Execution
      S1;
    if (x > 3 && y > 10)
      S3;
    else
      S4;
  }
}
  void test(int x, int y) {
    if (x > 0) {                      S0, S1, S3, S4 =
      if (y == hash(x))               statements we wish to cover
        S0;
      else                            Symbolic Execution
        S1;                           Can not handle it!
      if (x > 3 && y > 10)
        S3;                           Solution:
      else                            Mixed concrete-symbolic
        S4;                           solving
    }
  }
Assume hash is native or can not be
handled by decision procedure
            Mixed Concrete-Symbolic Solving

                          //hash(x)=10*x




            Example                        Mixed concrete-symbolic solving: all paths covered




                                                                                                Predicted path “S0;S4”
EXE results: stmt “S3” not covered                DART results: path “S0;S4” not covered        != path taken “S1;S4”
          Mixed Concrete-Symbolic Solving


— Use un-interpreted functions for external library calls

— Split path condition PC into:
  — simplePC – solvable constraints
  — complexPC – non-linear constraints with un-interpreted
    functions

— Solve simplePC
  — Use obtained solutions to simplify complexPC

— Check the result again for satisfiability
      Mixed Concrete-Symbolic Solving


Assume hash(x) = 10 *x:
PC: X>3 ∧ Y>10 ∧ Y=hash(X)

     simplePC      complexPC

Solve simplePC
Use solution X=4 to compute h(4)=40
Simplify complexPC: Y=40
Solve again:
    simplified PC: X>3 ∧ Y>10 ∧ Y=40 Satisfiable!
Symbolic Execution

void test(int x, int y) {
   if (x > 0) {                                                 PC: true
     if (y == hash(x))
       S0;
     else                                               PC: X>0        PC: X<=0
                                     Solve X>0
       S1;                           hash(1)=10                          …
     if (x > 3 && y > 10)            Check X>0 & PC: X>0 &
       S3;                                  Y=10     Y=hash(X) S0
     else
       S4;
   }                      Solve X>3 & Y>10 PC: X>3 & Y>10 &     PC: X>0 & X<=3 &
}                         hash(4)=40            Y=hash(X) S3       Y=hash(X) S4
                          Check X>3 & Y>10
int hash(x) {                    & Y=40
  if (0<=x<=10)
      return x*10;
  else return 0;
}
Potential for Unsoundness

  test (int x, int y) {
    if (x>=0 && x>y && y == x*x)
      S0;         Not Reachable
    else
      S1;
  }

           PC: X>=0 & X > Y & Y = X*X S0


simplePC X>=0 & X>Y        complexPC Y = X*X     Must add constraints
                                                 on the solutions back into
                                                 simplified PC
           X=0, Y=-1                   Y=0*0=0
                                                 X>=0 & X>Y & Y=0 & X=0
   simplified PC   X>=0 & X>Y & Y=0                               Not SAT!
                     Is SAT which implies
                     S0 is Reachable!                  DART/Concolic
                                                       will diverge instead
Directed Automated Random Testing (DART)
Godefroid, Klarlund and Sen 2005

or Concolic Execution


•   Collects path conditions along concrete executions
•   Negates constraints on the PC after a run and
•   Executes again with the newly found solutions
•   Can overcome the weaknesses of classic symbolic
    execution
                                                         X>0 & Y!=10 & X>3

                                  test(1,0)                   test(4,0)
void test(int x, int y) {
  if (x > 0) {                      X>0                        X>0
    if (y == hash(x))
      S0;                   X > 0 & Y != 10 S1          X > 0 & Y != 40 S1
    else
      S1;
    if (x > 3 && y > 10)
      S3;
                          X>0 & Y!=10 & X<=3 S4   X>0 & Y!=40 & X>3 & Y<= 10 S4
    else
      S4;
  }
}

native int hash(x) {
  if (0<=x<=10)
      return x*10;
  else return 0;
}

                                            DART/Concolic Execution
                       X>0 & Y!=40 & X>3 & Y>10     X>0 & Y=40 & X>3 & Y>10


                                 test(4,11)                    test(4,40)
void test(int x, int y) {
  if (x > 0) {                     X>0                          X>0
    if (y == hash(x))
      S0;
                            X > 0 & Y != 40 S1            X > 0 & Y = 40 S0
    else
      S1;
    if (x > 3 && y > 10)
      S3;             X>0 & Y!=40 & X>3 & Y>10 S3   X>0 & Y=40 & X>3 & Y>10 S3
    else
      S4;
  }
}

native int hash(x) {
  if (0<=x<=10)
      return x*10;
  else return 0;
}

                                           DART/Concolic Execution
                            X>0 & Y=40 & X<=3 & Y>10


                                  test(1,40)
void test(int x, int y) {
  if (x > 0) {                        X>0
    if (y == hash(x))
      S0;                      X > 0 & Y != 10 S1
    else
      S1;
                                                       Divergence!
    if (x > 3 && y > 10)
      S3;                    X>0 & Y!=10 & X<=3 S4     Aimed to get S0;S4
    else                                               But reached S1;S4
      S4;
  }
}

native int hash(x) {
  if (0<=x<=10)
      return x*10;
  else return 0;
}

                                          DART/Concolic Execution
         Mixed Concrete-Symbolic Solving
                    vs DART

— Both incomplete

— Incomparable in power (see paper)

— Mixed concrete-symbolic solving can handle only “pure”,
  side-effect free functions
  — DART does not have the limitation; will likely diverge
 Addressing Incompleteness: 3 Heuristics


Incremental Solving


                User Annotations


                                Random Solving
Incremental Solving

void test(int x, int y) {
  if (x > 0) {                                                      PC: true
    if (y == hash(x))
      S0;
    else                                                  PC: X>0          PC: X<=0
                                      Solve X>0
      S1;                             hash(1)=10                               …
    if (y > 10)                       Check X>0 & PC: X>0 &
      S3;                                    Y=10    Y=hash(X) S0
    else
      S4;
  }                         Solve X>0 & Y>10 PC: X>0 & Y>10 &       PC: X>0 & X<=3 &
}                           Solution: X=1        Y=hash(X) S3          Y=hash(X) S4
                            hash(1)=10
int hash(x) {               Check X>0 & Y>10
  if (0<=x<=10)                    & Y=10 Not SAT!
      return x*10;
  else return 0;                         Get another solution: Solution: X=2
}                                                              hash(2)=20
                                                               Check X>0 & Y>10
                                                                      & Y=20 SAT!
User Annotations
@Partition({“x>3”,”x<=3”})
void test(int x, int y) {
   if (x > 0) {                                                      PC: true
     if (y == hash(x))
       S0;
     else                                                    PC: X>0        PC: X<=0
                                      Solve X>0
       S1;                            hash(1)=10                              …
     if (y > 10)                      Check X>0 & PC: X>0 &
       S3;                                     Y=10       Y=hash(X) S0
     else
       S4;
   }                 Solve X>0 & Y>10 & X>3 PC: X>0 & Y>10 &         PC: X>0 & X<=3 &
}                    Hash(4)=40                   Y=hash(X) S3          Y=hash(X) S4
                     Check X>0 & Y>10
int hash(x) {               & Y=40 SAT!
  if (0<=x<=10)
                           Add user partitions one at a time
      return x*10;
  else return 0;
}
             Random Solving

• Pick solutions randomly from the solution
  space

• Current implementation only picks
  randomly if the solution space is
  completely unconstrained
                    o n
                  ti
             n ta
        e                Mixed Concrete-        Custom Listeners on SPF
       m                 Symbolic Solving


  p le
Im
                       Symbolic PathFinder      Symbolic Execution
                              SPF               Extension for JPF (jpf-symbc)



                                                Model Checker for Java
 Java PathFinder                                Open Source
                                                http://babelfish.arc.nasa.gov/trac/jpf


 Experience
     —   TSAFE (Tactical Separation Assisted Flight Environment)
     —   Apollo Lunar Pilot
     —   Example PC: 37 constraints in simplePC and 6 in complexPC
                Related Work
— Tools that perform mixture of concrete and symbolic
  execution
  — EXE, DART, CUTE, PEX, SAGE, …

— “Higher order test generation” – P. Godefroid [PLDI’11]
  — Uses combination of validity checking and un-interpreted
    functions
  — Generates tests from validity proofs
  — Implementation challenge
      Conclusions and Future Work
— Mixed concrete-symbolic solving to address problems with
  classic symbolic execution
  — Handling native libraries
  — Incomplete decision procedures

— Open source implementation for Java
— Future Work
  — More experiments
  — More heuristics
  — Handle data structures executed outside symbolic execution
    — Use JPF’s serialization
Thank you!

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:1
posted:6/27/2013
language:English
pages:22