mtm by fanzhongqing


									Modular Shape Analysis for View-Serializable Libraries

      N. Rinetzky1 , A. Bouajjani2 , G. Ramalingam3, M. Sagiv1 , and E. Yahav4
                     Tel Aviv University {maon,msagiv}
              LIAFA, CNRS & University of Paris 7
                     Microsoft Research India
                  IBM T.J. Watson Research Center

       Abstract. We present novel modular static shape analysis algorithms for con-
       current libraries. Our analyses conservatively verify the absence of certain mem-
       ory and concurrency errors, verify a certain class of program assertions, and infer
       shape (heap) module invariants. The key idea is to focus on a class of concur-
       rent programs that follow certain standard locking policies which ensure view-
       serializability. This allows our analyses not to consider interleaving between the
       low-level instructions implementing high-level operations on thread-shared data
       structures. Technically, we employ existing sequential shape analysis algorithms
       to perform interprocedural and modular analyses that can establish sufficient con-
       ditions for view-serializability for concurrent libraries.

1 Introduction

The verification of concurrent heap-manipulating programs is a challenging problem: it
requires reasoning about interleaved executions of threads manipulating an unbounded
shared state. Modular verification (see, e.g., [1]) is even more challenging because it
attempts to break the program into parts and verify each program part separately. Still,
modular verification is desirable for achieving scalability and reuse, both of which are
particulary hard to obtain for verification of concurrent programs (see, e.g., [2]). In
this paper, we present a novel approach for modular verification of concurrent heap-
manipulating programs.
    One of the main challenges in developing and verifying concurrent programs is the
need to consider all possible interleavings of operations performed by different threads.
To simplify this kind of reasoning, some programming paradigms restrict the possible
interleavings of a program (see, e.g., [3]). In this paper, we are interested programming
paradigms which ensure that a program is view-serializable.
    View-serializability (see, e.g., [4]) is one of the main correctness conditions in con-
current programming. It ensures that every execution has a corresponding serial execu-
tion which is equivalent to it with respect to the thread-local views of the program state.
We leverage the view-serializability of a program to simplify its verification: we verify
certain interesting local-view properties (described in Sec. 3.2) of a view-serializable
program by exploring only its serial executions. The class of local-view properties in-
cludes simpler properties such as the absence of null dereferences as well as more com-
plex module invariants, as illustrated later.
     class Acc {
       int checking=0; int saving=0;   int total() {                    void save(int s) {      void transfer(Acc to, int t) {
       Object Lc = new Object();         this.Lc.lock();                  this.Lc.lock();         this.Lc.lock();
       Object Ls = new Object();         int cur = this.checking;         this.checking -= s;     this.checking -= t;
        void deposit(int d) {            this.Ls.lock();                  this.Ls.lock();         to.Lc.lock();
         this.Lc.lock();                 this.Lc.unlock();                this.Lc.unlock();       to.checking += t;
         int old = this.checking;        cur += this.saving;              this.saving += s;       this.Lc.unlock();
         this.checking += d;             this.Ls.unlock();                this.Ls.unlock();       to.Lc.unlock();
         this.Lc.unlock();               return cur;                    }                       }
         return old;                   }                            }

Fig. 1. Bank account modules using two-phase locking. Module ACC contains class ACC. Module
ACC P LUS contains class AccPlus which extends class Acc with procedure transfer. (For
space reasons, we do not show class AccPlus).

Motivating Example Fig. 1 shows Java-like pseudo code of two modules ACC and AC -
C P LUS implementing a bank account data structure.5 In the simpler ACC module, every
bank account contains the balance of a saving account and the balance of a checking
account. Bank accounts can be (directly) manipulated only by procedures deposit,
which adds a given amount to the checking account; total, which returns the total bal-
ance in the bank account; save, which transfers a specified amount from the checking
account to the saving account of the same bank account. Our goal is to automatically
verify that (I) module Acc is view-serializable and (II) after every concurrent execution
of any client using Acc, the total amount in every bank account is equal to the sum of
all the deposits made into that account. Module ACC P LUS extends the ACC module
by adding a transfer method that transfers a specified amount from one checking
account to another and will be used later to illustrate certain aspects of our algorithm.
     A key point to note is the invariant (II) above is reasonably easy to check in the
sequential case (i.e., if a single client (thread) uses this module). It is well-known that
in a concurrent setting all kinds of natural invariants may fail to hold if the program
fails to use locking properly.
Our Approach We do not assume that a program is view-serializable; we verify it.
More specifically, we (conservatively) determine that a program is view-serializable by
verifying that it conforms to one of several standard locking policies which ensure view-
serializability. These policies have the interesting property that if they are respected by
all the serial executions then they are respected by all the concurrent executions [5].
Our analyses leverage this property: They conservatively verify the conformance by
exploring only the serial executions of the program, i.e., our analyses are sequential.
    Technically, we develop static analyses that verify conformance with the strict two-
phase locking (S2PL) policy [6], the two-phase locking (2PL) policy [7], and the tree-
locking (TL) policy [8–10]. For example, as we explain in Sec. 3, module Acc adheres
to the 2PL policy. This allows us to verify properties (I) and (II) using a sequential
    Our analyses are modular: We assume the program is partitioned into a set of mod-
ules, where each module is a collection of types and procedures. Our analyses analyze
every program module separately. Technically, we obtain a modular analysis for con-
     Adapted from standard examples used in transaction processing. See, e.g., [4, §§1].

current programs by placing certain syntactic and semantic restrictions on the allowed
behavior of modules and on the properties that we aim to verify. These restrictions al-
low us to provide a fixpoint characterization of all possible inputs for procedures of the
analyzed module, and thus to obtain a modular analysis.
    Note that our analyses verify view-serializability in a modular fashion. This result
might seem surprising: serializability is known to be a non-compositional property.
The reason that we achieve modularity is that instead of checking directly for view-
serializability, we establish a modularly checkable program-invariant which ensures
Main Results The main results of this paper can be summarized as follows:
1. We identify an interesting class of local-view invariants. In serializable programs,
   a local-view invariant holds in all the serial executions only if it holds in all execu-
2. We develop novel conservative shape analyses for concurrent programs capable of
   (i) showing conformance with certain locking policies which ensure view-serializability;6
   (ii) establishing interesting program properties such as the absence of memory er-
   rors, e.g., the absence of null-valued pointer dereferences and of memory leaks;
   (iii) verifying the correctness of local-view program assertions.
3. Our analyses utilize the results of [5] to explore only serial executions.
4. We employ existing sequential interprocedural and modular shape analysis algo-
   rithms in a concurrent setting.
Paper Outline Due to space restrictions, we concentrate on providing an extended
informal overview of our work using two extended examples. Formal details can be
found in [11].

2 Programming Model and Verification Goal

Program Model We analyze imperative concurrent object-based (i.e., without subtyp-
ing) programs. A program consists of a collection of modules.
Modules and Classes. Every module consists of a collection of classes. Every class type
consists of the collection of fields and a collection of procedures. A field defined in
class type can be directly manipulated only by one of its procedures. Class-level fields, a
la Java’s static fields, are not allowed. Procedures are allowed to have formal pass-
by-value parameters. (Global variables are not allowed). We refer to the first parameter
to a procedure call as the target of the call, and use Java’s syntax x0 .p(x1 ,. . .,xk
to denote an invocation of a procedure p on x0 . We say that a module m1 depends on
(another) module m2 if it uses a class that belongs to m2 as a type of one of the fields
that belong to m1 or as a variable of a procedure that belongs to m1 .
Concurrency. Threads can be dynamically created and launched to execute a given pro-
cedure p. Synchronization is achieved via locks. (We follow Java’s idiom, and use
Objects for locks [3].) We assume non-reentrant locks, i.e., a thread gets blocked if
it tries to acquire a lock it already has. We discuss this choice in Sec. 5.
     Conformance to the tree-locking policy also ensures deadlock freedom. See, e.g., [4, §§3].

System Model We assume that the module dependency relation is a directed acyclic
graph with a single root. We refer to the root module as the client module and to all other
modules as library modules. The client module contains the program’s main procedure
and is the only module which can create threads.
Operations. We refer to a procedure of a library module that can be invoked from an-
other module as a library module interface procedure. We refer an invocation of a li-
brary module interface procedure as an operation.
Verification Goal Our goal is to modularly verify the view-serializability of the library
modules and to discover their properties. Note that the serializability of a library module
is independent of its client. This is inline with the standard definition of serializability
in databases where the database is required to be serializable for any client.

3 Verifying Programs Employing Two-Phase Locking
In this section, we present an informal overview of our approach for verifying programs
employing two-phase locking, by illustrating how it works for the example of Fig. 1.

3.1 Modular Verification For Sequential Clients
We first consider the simpler problem of verification in a sequential setting. Specifically,
we would like to verify that the module ACC satisfies the desired invariants for any
possible sequential client of the module. We do this by first constructing a most-general-
client, whose behavior subsumes the behavior of any possible single-threaded client.
The bank accounts most-general-client is a (sequential) program that allocates a bank
account and invokes an arbitrary nondeterministic sequence of operations on it.
    We then apply abstract interpretation to the program consisting of ACC together
with the most-general-client. In general, we can utilize any suitable abstract interpreta-
tion as appropriate for the properties we wish to prove. Thus, to establish shape invari-
ants and the absence of memory errors, we can use any interprocedural shape analysis,
e.g., [12–15]. (The ACC example is, however, simple enough that the full power of
shape analysis is not needed.) Abstract numeric domains such as those that track linear
equalities between variables or the Octahedron domain [16] suffice to establish verifi-
cation goal (II).
    Note: The most-general-client described above is correct when all module opera-
tions involve a single module object (such as a single bank account in the ACC exam-
ple). If there are module operations that involve multiple module objects (e.g., such as
the transfer operation of the ACC P LUS module), the most-general-client is a bit more
complicated. The most-general-client for ACC P LUS must essentially simulate arbitrary
sequences of operations on an arbitrary number of objects (e.g., bank accounts). This
also means that the abstract interpretation we use must be able to handle an unbounded
number of objects, much like shape analyses.

3.2 Modular Verification For Concurrent Clients
We now consider the verification problem in a concurrent setting. We would like to
verify that the module ACC satisfies the desired invariants even when multiple arbitrary

clients invoke operations on the same ACC objects concurrently (or, equivalently, for an
arbitrary multi-threaded client).
Exploiting Serializability Let π be an execution sequence of (low-level) instructions
produced by the concurrent (i.e., potentially interleaved) execution of a set S of high-
level operations on (the data structures implemented by) a module M . π is said to be
view-serializable if there exists a sequence πs produced by some serial execution of the
same set S of high-level operations that is semantically equivalent to π: πs corresponds
to a permutation of π that produces the same end state as π; furthermore, every low-
level statement in π reads and writes the same values as its matching statement in πs
(see, e.g., [4]). Informally, view-serializability means that from the local viewpoint of
every thread π and πs are indistinguishable.
    We say that a module M is view-serializable if any execution sequence produced
by any set of concurrent operations on M is view-serializable.
    Let us now consider module assertions: these are assertions embedded in the im-
plementation of module operations and, hence, are considered part of the execution
sequence produced by the execution of the module operations. It follows from the defi-
nition of view-serializability that:

Observation 1 (Main observation) If M is a view-serializable module, a module as-
sertion is an invariant in all executions of M’s operations iff it is an invariant in all
serial executions of M’s operations.

   Also note that:

Observation 2 A module assertion is an invariant in all serial executions of M’s oper-
ations iff it is an invariant for the most-general sequential client.

    As a result of the above observations, it follows that we can meet our goal (of verifi-
cation for concurrent clients) if we can establish that module ACC satisfies serializabil-
ity and verify that the module assertions hold for the most-general-sequential-client.
Verifying Serializability A technique commonly used to ensure that an implementa-
tion is serializable is to follow certain concurrency control mechanisms that guarantee
serializability. We follow a similar approach and conservatively check if a module is
serializable by checking if it follows one of several locking protocols that guarantee
serializability. For the running example, our analysis is able to establish serializability
by verifying that the component adheres to the 2PL policy.
    The 2PL policy dictates that (i) Every part s of the thread-shared state is owned
by a lock ls which protects all accesses to s. Specifically, a thread can access (i.e.,
read or write) s only when it holds lock ls . At every point during the execution, all
the threads have consistent views of the lock ownership map, i.e., they agree on the
state-lock ownership relation. (ii) Every operation is comprised of two phases: In the
first phase, the growing phase, the operation is allowed to acquire locks, but may not
release any locks. In the second phase, the shrinking phase, the operation may release
locks, but may not acquire any locks. When all operations in a module follow 2PL, it is
guaranteed to be view-serializable (see, e.g., [4]).

    In the running example, the checking balance is protected by Lc and the saving
balance is protected by Ls. In our current work we assume that we are given a specifi-
cation of which lock protects which data field.
    Our analysis verifies that a module is serializable by (i) ensuring that every field is
accessed only when its protecting lock is held, and (ii) verifying that no lock is acquired
after a lock is released, and (iii) verifying that the lock fields are initialized when the
module object is created and that they are not subsequently modified.

Local-view Assertions Note that we need to ensure that the assertions we verify are
themselves serializable (to ensure that it is sufficient to verify these assertions over
serial executions). This holds true for an assertion at a program point if the assertion
refers only to fields whose protecting lock is held at that program point. We refer to
such assertions as local-view assertions.
Verifying 2PL Via Sequential Analysis We need to verify that an operation of a mod-
ule satisfies 2PL even when its execution interleaves with other concurrent operations.
An interesting aspect of our algorithm for 2PL verification is that it considers only serial
executions of the module operations. While this is an optimization similar to the one de-
scribed above for verifying module invariants, it cannot be justified the same way. This
is because until 2PL is established, we do not have the guarantee that any interleaved
execution of operations is equivalent to some serial execution of operations.
    At this point we leverage the fact that 2PL is a sequentially observable locking
policy, as shown in [5].

Definition 1 ( [5, Def. 1]). We say that a locking policy is sequentially observable when
the following holds—if every serial execution of a concurrent system follows the locking
policy, then every execution of the concurrent system follows the locking policy.

Theorem 1 ( [5, The. 1]). The 2PL policy is sequentially observable.

    As a result of the above theorem, we can integrate the 2PL verification with our
verification of module invariants and apply it to the most-general-sequential-client. This
is done by augmenting the abstraction domain to track the set of locks acquired, as
well as whether any lock has been released (which signals the beginning of the second
phase). For our running example, verification of 2PL is straightforward. (In general,
if the 2PL property depends on some module invariant, our analysis can exploit the
computed module invariants in establishing the 2PL property.)

4 Verifying Programs Employing Tree Locking
In this section, we discuss our approach for verifying programs employing the tree-
locking (TL) policy to achieve view-serializability with fine-grained synchronization.
The TL policy is significantly more complicated than the 2PL policy, and so is its ver-
ification. We make the problem even more challenging by presenting a modular ap-
proach which verifies each library module separately. Specifically, in this section we
harness the full power of shape analysis for modular analysis of concurrent hierarchi-
cal dynamically-allocated linked data structures. Our analysis verifies the absence of

memory errors; infers shape invariants; and establishes that the module implementation
respects the TL policy. We note that the TL policy not only ensures view-serializability
and race-freedom, but also guarantee the absence of deadlocks (see, e.g., [4]).
Tree Locking The tree-locking policy (TL) is a locking policy which is specialized for
programs which manipulate hierarchical data structures. Informally, it assumes that the
thread-shared state can be decomposed into a tree. Every node of the tree is protected by
a lock. (I.e., the program uses an unbounded number of fine-grained locks). Whenever a
thread wants to access a piece of the shared state, it acquires the lock protecting the root
of the tree, and start traversing the tree edges. The traversal is done using a technique
called lock coupling [8]: After the thread releases a lock of a node, it is not allowed to
access it again during its current traversal. (This form of traversal is also known as a
‘‘hand-over-hand” traversal). For a further explanation of the TL policy, see Sec. C.
     Lock coupling is a popular technique in the implementation of concurrent search
trees. (It is used, in particular, in the implementation of concurrent B-trees, which are
at the heart of many data base implementations [4].)

4.1 Illustrative Example

We illustrate our ideas by applying our approach to analyze the modules R EQPOOL and
R EQ shown in Fig. 2. Module R EQPOOL implements a request pool data structure. A
request pool manages a collection of requests, implemented as a singly-linked list. Ev-
ery list node contains three fields: a successor link (n), a pointer to a request (r), and a
flag (handled) which indicates whether the request referenced by the node has been
(or is currently being) handled. Request pools can be (directly) manipulated only by
R EQPOOL’s procedures. Procedure crtReqInPool allocates a request and links it at
the tail of the pool’s list. It locks the request pool throughout its execution. Procedure
processPool processes requests. It obtains the request using a variant of lock cou-
pling: it first locks the pool, then searches for an unhandled request; if such a request
is found, it locks the request, unlocks the pool, and processes the request. Procedure
donePool checks whether all requests in the pool have been processed by iterating
over all requests; locking every request in the list; and invoking procedure endedReq
on it, before it continues. Note that donePool locks the pool throughout its execution.
    Module R EQ implements a request data structure. A request is comprised of a Req
object pointing to a Data object. A request can be (directly) manipulated only by proce-
dure processReq. The latter uses the the boolean field ended to record that the request
has been processed.
    When verifying the request pool module, our goal is to automatically verify that
(I) module R EQPOOL is view-serializable, (II) R EQPOOL’s procedures neither leak
memory nor dereference null-valued pointers, and (III) R EQPOOL’s list is acyclic.7

     Our approach can also establish the partial correctness of procedure donePool. Specifically,
     an extension of our analysis, which, for space reason, is described only in [11], can show that
     the unhandled requests follow the handled ones.

                                                                                                      (a) Module REQPOOL                                                                                                                                  (b) Module REQ
 class ReqPool {                                                                                                                                                                                class Node {                                          class Req {
   Node hd = null;                                                                                                                                                                                Node n = null;                                        boolean done = false;
                                                                     void handleReqInPool() {                                     boolean donePool() {
   void crtReqInPool() {                                                                                                                                                                          Req r = null;                                         Data d;
                                                                       this.lock();                                                 this.lock();
     this.lock();                                                                                                                                                                                 boolean handled = false;
                                                                       Node t = this.hd;                                            Node t = this.hd;
     Node t = this.hd;                                                                                                                                                                            Node() {                                                void handleReq() {
                                                                       while (t != null && t.handled)                               boolean dn = true;
     while (t != null && t.handled)                                                                                                                                                                 this.r = new Req();                                     ...
                                                                         t = t.n;                                                   while (t != null) {
       t = t.n;                                                                                                                                                                                   }                                                         this.done = true;
                                                                       if (t != null) {                                               Req req = t.r;
     if (t != null)                                                                                                                                                                             }                                                         }
                                                                         t.handled = true;                                            req.lock();
       t.n = new Node ();                                                                                                             dn = dn && req.doneReq();
                                                                         Req req = t.r;                                                                                                                                                                   boolean doneReq() {
     else                                                                                                                             req.unlock();
                                                                         req.lock();                                                                                                                                                                        return this.done;
       this.hd = new Node ();                                                                                                         t = t.n;
                                                                         this.unlock();                                                                                                                                                                   }
     this.unlock();                                                                                                                 }
                                                                         req.handleReq();                                                                                                                                                             }
   }                                                                                                                                this.unlock();
                                                                       }                                                            return dn;
                                                                                                                                                                                                                                                      class Data {
                                                                       else this.unlock();                                        }
                                                                     }                                                        }

                                                                                          Fig. 2. (a) the request pool module. (b) the request module.

tid=0           this   tid=3       t
pc=…                   pc=…                                                                                                            t                                                                                              t
                                                                                                                  this    pc=…                                                                                  this       pc=…

   pool                  held by
                                                                                                                              held by                                                                                        held by

                   hd                   n                 n         n           n
                             h                    h                                                                      hd                n         n       n               n                                         hd                 n           n          n       n
                                                                                                                                  h             h                                                                                h                h
          p                    r                  r            r           r                  r
 tid=4                                                                                                                             r            r        r          r                   r                                         r               r          r       r          r
                         s                    s       e
                                                                                                                              s e              s e
    this                       d                  d            d           d                  d                                                                                                                              s e              s e
                                                                                                                                   d            d        d              d               d
        tid=1                          this                                                                                                                                                                                       d               d          d       d          d
                  held by                tid=2            held by

              (a) Concurrent memory state.                                                                           (b) Sequential memory state.                                                                (c) Component decomposition.

                                                                                          t                                                                                                 t
                                                                               pc=…                                                                                              pc=…
                                                                    this                                                                                         this
                                                                                 held by                                                                                           held by
                                                                                                                                                                                                 n                                            n
                                                                           hd                     n       n          n            n                                         hd                       n                 n
                                                                                    h                 h                                                                                 h
                                                                                      r               r       r           r                r                                                         r      r                r

                                                              (d) Trimmed sequential memory state.                                                   (e) Bounded trimmed sequential memory state.

                Fig. 3. (a,b,d,e) Representative memory states. (c) Component decomposition of the sequential memory state shown in (b).
4.2 Modular Separate Verification For Sequential Clients
In the spirit of Sec. 3, we first consider the simpler problem of verification in a se-
quential setting. We adapt the sequential modular shape analysis of [17] which treats
each module separately to also verify conformance with the TL policy. We present an
analysis by applying it to verify module R EQPOOL. For simplicity, we assume that only
intermodule procedure calls are allowed in library modules.
Heap Modular Abstract Interpretation Our analysis simultaneously verifies and ex-
ploits the (expected) tree decomposition of the memory to analyze every module using
only the part of the memory state manipulated by that module.
Componentized Heaps. The analysis represents the program heap as an evolving collec-
tion of heap-components. Every heap-component is comprised of objects whose types
are defined in the same module. More specifically, every heap-component is a maximal
weakly connected component (in the graph-theoretic sense) of objects whose types are
defined in the same module. (We say that a heap-component belongs to that module.)
Note that multiple heap-components belonging to the same module may co-exist. Refer-
ences between heap-components belonging to different modules are allowed. However,
these reference are is expected (and verified) to adhere to the following constraints:
(i) The component graph induced by the inter-heap-component references structure to
form a tree. (ii) All inter-heap-component references pointing to a heap-component
must have the same target object. (We call this object the component’s header.)
Example 1. Fig. 3(b) presents a memory state that can arise during an execution of a sequential
client which uses a request pool. The (single) thread is depicted as an hexagon. The REQPOOL
object is depicted as a square, Node objects are depicted as circles, Req objects are depicted as
pentagons, and Data objects are depicted as rounded rectangles. A reference field f between
heap-allocated objects is depicted as an f -labeled arrow. The value of a boolean field b at an ob-
ject o is depicted by a label b inside objects inwhich b has a true value. The value of a reference
variable x is depicted as an x-labeled arrow leaving the thread’s hexagon. The held by labeled
arrow pointing from the square depicting the request pool to the hexagon depicting the thread
indicates that the thread holds the lock of the request pool. Fig. 3(c) depicts the decomposition
of the memory state into heap-components. Every heap component is shown inside a rectangular
frame. Every header object is pointed to by a wide arrow.
Trimming Abstraction. We refer to the union of the heap-components pointed to by the
the local variables of the current procedure as the active-component. The trimming ab-
straction abstracts away all parts of the memory except the active component. (Loosely
speaking, only the heap structure of the current component, and the aliasing relation-
ships between intermodule references leaving the active component, are tracked.)
Example 2. Fig. 3(d) presents the trimmed memory state resulting from the application of the
trimming abstraction to the memory state shown in Fig. 3(b).

Bounded Abstraction. Our bounded shape abstraction is obtained using a 2-step suc-
cessive abstraction. We first apply the aforementioned trimming abstraction. We then
apply a bounded conservative abstraction of trimmed memory states. Rather than pro-
viding a new intraprocedural abstraction, we lift existing intraprocedural shape analy-
ses, e.g., [18–20], to obtain a modular shape abstraction (see [17, §4]). The main idea
is that the abstraction is extended to include (i) references pointing to headers of inter-
heap-component references leaving the active component and (ii) the held by edges

which record the current set of acquired locks. (Note that, similar to [17], our analy-
sis is parametric in the bounded shape abstraction. Specifically, we can use different
bounded abstractions when analyzing different modules.)
Intraprocedural Analysis. Our program model ensures that the internal structure of a
component can be accessed or modified only by the (procedures in the) module to which
it belongs.8 Specifically, intraprocedural statements cannot traverse inter-component
heap references. Thus, we use the underlying intraprocedural shape to analyze these
Separate Modular Analysis of Library Modules Our static analysis is conducted
in an assume-guarantee manner allowing each module to be analyzed separately. The
analysis, computes a conservative representation of every possible heap-components of
the analyzed module in programs that adhere to our restrictions. This process, in effect,
identifies structural invariants of the heap-components of the analyzed module, i.e., it
infers module invariants for these programs.
     Technically, the module is analyzed together with its most-general-client (see Sec. 3.2)
using an adaptation of the underlying shape analysis, as discussed above. (A more de-
tailed discussion about the required adaptation can be found in [11] and in [17].)
     For simplicity, we assume that all module operations involve a single data structure
object i.e., either a single request pool or a single request. We note that our analysis can
be extended to support multiple object parameters and changing component trees using
the dynamic encapsulation techniques of [4]. This extension would require, in partic-
ular, using the more complicated most-general-sequential-client discussed in Sec. 3.1.
This extension, however, is outside the scope of this paper. For details, see [11].

4.3 Modular Separate Verification For Concurrent Clients
We now consider the verification problem in a concurrent setting.
Exploiting Tree Locking The tree locking policy guarantees view-serializability. Thus,
we can use the most-general-sequential-client to verify that module assertions are in-
deed module invariants for concurrent clients. (See Sec. 3.2). However, in case the
module is shown to respect the TL protocol by our analysis, we can give a syntactical
characterization for a set of local-view assertions: We say that an access path is local
to module M if it dereference only fields that belong to types defined in M . The key
property of a module assertion is that it only refers to fields whose protecting lock is
held at that program point. Our analysis verifies that when an intermodule procedure
call occurs, the lock on the active component is held. Thus, we know a priori, by the
definition of a component, that any module assertions in procedure p written using only
access paths local to the module of p is guaranteed to be a local-view assertion.
Example 3. The invariant that the list starting at request pool object is acyclic is a local-view
assertion: It uses only the fields hd and n.

Verifying Serializability We conservatively verify that a module follows the TL policy,
which guarantee view-serializability, by:
     A module m can manipulate a component of a module m′ by passing it as a parameter to an
     intermodule procedure call.

  (i) Checking whether any execution of an interface module acquires the lock on its
      target (which is the only data structure parameter) before it manipulates it. If
      so, then the procedure is marked as self-locking. All the procedures of module
      R EQPOOL, for example, are self-locking. Otherwise, it is marked as externally-
      locked. When analyzing invocation of externally-locked procedures, the analysis
      needs to establish that the lock on the target of the call is held. All the procedures
      of module R EQ, for example, are externally-locked.
 (ii) Verifying that the active component is not accessed is not accessed after releasing
      it lock. Note, for example, that procedure handleReqInPool does not access the
      request pool after the return of the intermodule procedure call to module R EQ.
      Procedure donePool, on the other hand, can access the request pool fields after
      invoking module R EQ because it does not release its lock on the request pool.
Note that the requirement that the inter-heap-component references form a tree is ful-
filled by construction thanks to our single data structure parameter requirement. Also
note that we (conservatively) assume that the component header functions also as its
lock. (Both assumptions can be generalized. But this outside the scope of this paper.)

Verifying TL Via Sequential Analysis Our analysis considers only serial executions
of the module operations, as justified by the following theorem,

Theorem 2 ( [5, The. 4]). The TL policy is sequentially observable.

5 Discussion and Related Work

The main novelty of our approach is that it leverages serializability to analyze con-
current systems efficiently. Specifically, it shows that all possible program executions
satisfy certain properties by (safely) considering only their serial subset. By doing so, it
does not need to consider interleavings between the low-level statements implementing
the high-level operations on thread-shared data structures. In this section, we briefly
discuss some key aspects of our approach, its strengths, its weaknesses, and contrast
it with closely related automatic verification techniques for concurrent programs. For
additional discussion, see [11].
Partial Order Reduction. Partial order reduction (POR) techniques [21–23] combat the
state-explosion problem by exploring only a representative subset of all program exe-
cutions. In general, however, verifying that a subset of all executions is representative,
may be as hard as solving the underlying verification problem. Therefore, existing POR
techniques traditionally use static analysis to check for potential collisions between
low-level operations of different threads (see, e.g., [24]). Our approach can be seen as
a POR employing a different static analysis.
Thread-modular heap analysis. [25] presents a thread-modular analysis of heap ma-
nipulating programs. Their analysis operates on every thread in isolation, but in a non-
modular fashion. [25] does not place a-priori restrictions on the shape of the heap, but
handles a only bounded number of lock variables. In contrast, we can handle an un-
bounded number of heap-storable locks in hierarchical data structures.

Automatic Verification of Linearizability. Several works [26–28] automatically verify
linearizability [29] of concurrent data structures. Linearizability is an important cor-
rectness condition for concurrent abstract data types. However, by definition, a proce-
dure of a linearizable data type can only operate on a single instance. When a procedure
operates on multiple data structure instances, serializability is a more natural correct-
ness condition [29]. For example, our analysis can show that the transfer procedure
shown in Fig. 1 (transferring money between two account objects) is serializable.
Atomicity. Atomicity [30, 31] is an important correctness condition for concurrent pro-
grams. Atomicity, can be seen as a special case of linearizability [29], where the allowed
behaviors of a procedure in a concurrent setting is specified by its sequential behaviors.
Linearizability itself, is a special kind of strict serializability [29], where the serial exe-
cution must preserve the order between non overlapping transactions. Thus, if applica-
ble, we can use existing techniques to verify that all the procedures in the program are
atomic, and use our shape analysis to infer thread local-view program invariants.
    We note, however, that there are some interesting programming paradigms that en-
sure atomicity, but cannot be verified using existing automatic approaches. For exam-
ple, current approaches for atomicity does not validate the atomicity of programs that
manipulate hierarchical data structures using fine-grained locking according to the lock-
coupling policy [8], while our analyses can. Also, they current techniques for atomicity
cannot work when a procedure manipulates data structures which are protected by dif-
ferent locks. E.g., the transfer procedure.

We noticed that there is a nice match between the restrictions imposed by our modular
analysis and by the TL policy. We believe that this is non accident. Both restrictions aim
at limiting the interference between operations invoked on the data structure from the
(unknown) context. Specifically, both analyses use the fact that there is a single path
reaching to every tree node to ensure orderly processing of the data structure. In the
future, we plan to further investigate these relations between the two.

 1. Cousot, P., Cousot, R.: Modular static program analysis, invited paper. In: CC. (2002)
 2. Qadeer, S., Rajamani, S.K., Rehof, J.: Summarizing procedures in concurrent programs. In:
    POPL, ACM (2004)
 3. Lea, D.: Concurrent Programming in Java: Design Principles and Patterns. 2nd edn.
    Addison-Wesley (1999)
 4. Bernstein, P.A., Hadzilacos, V., Goodman, N.: Concurrency Control and Recovery in
    Database Systems. Addison-Wesley (1987)
 5. Rinetzky, N., Attiya, H.:        Sequential verification of locking policies for serial-
    izability is enough!       Technical report, Tel Aviv University (2008) available at
    “∼maon” (in preperation).
 6. Papadimitriou, C.H.: The serializability of concurrent database updates. J. ACM 26(4)
    (1979) 631–653
 7. Eswaran, K.P., Gray, J., Lorie, R.A., Traiger, I.L.: The notions of consistency and predicate
    locks in a database system. CACM 19(11) (1976) 624–633

 8. Bayer, R., Schkolnick, M.: Concurrency of operations on b-trees. Acta Inf. 9 (1977)
 9. Samadi, B.: B-trees in a system with multiple users. Information Processing Letters 5(4)
    (1976) 107–112
10. Kedem, Z.M., Silberschatz, A.: A characterization of database graphs admitting a simple
    locking protocol. Acta Inf. 16 (1981) 1–13
11. Rinetzky, N., Bouajjani, A., Ramalingam, G., Sagiv, M., Yahav, E.: Modular shape analysis
    for view-serializable programs. Technical report, Tel Aviv University (2008) available at
    “∼maon” (in preperation).
12. Jeannet, B., Loginov, A., Reps, T., Sagiv, M.: A relational approach to interprocedural shape
    analysis. In: SAS. (2004)
13. Rinetzky, N., Bauer, J., Reps, T., Sagiv, M., Wilhelm, R.: A semantics for procedure local
    heaps and its abstractions. In: POPL. (2005)
14. Hackett, B., Rugina, R.: Region-based shape analysis with tracked locations. In: POPL.
15. Gotsman, A., Berdine, J., Cook., B.: Interprocedural shape analysis with separated heap
    abstractions. In: SAS. (2006)
16. Claris´ , R., Cortadella, J.: The octahedron abstract domain. SCP 64(1) (2007) 115–139
17. Rinetzky, N., Poetzsch-Heffter, A., Ramalingam, G., Sagiv, M., Yahav, E.: Modular shape
    analysis for dynamically encapsulated programs. In: ESOP. (2007)
18. Lev-Ami, T., Immerman, N., Sagiv, M.: Abstraction for shape analysis with fast and precise
    transformers. In: CAV. (2006)
19. Manevich, R., Yahav, E., Ramalingam, G., Sagiv, M.: Predicate abstraction and canonical
    abstraction for singly-linked lists. In: VMCAI. (2005)
20. Distefano, D., O’Hearn, P.W., Yang, H.: A local shape analysis based on separation logic.
    In: TACAS. (2006)
21. Valmari, A.: Stubborn sets for reduced state space generation. In: ICATPN, Springer-Verlag
22. Peled, D.: All from one, one for all: on model checking using representatives. In: CAV.
23. Godefroid, P.: Partial-Order Methods for the Verification of Concurrent Systems: An Ap-
    proach to the State-Explosion Problem. Springer-Verlag (1996)
24. Dwyer, M.B., Hatcliff, J., Robby, Ranganath, V.P.: Exploiting object escape and locking in-
    formation in partial-order reductions for concurrent object-oriented programs. FMSD (2004)
25. Gotsman, A., Berdine, J., Cook, B., Sagiv, M.: Thread-modular shape analysis. In: PLDI,
    New York, NY, USA, ACM (2007) 266–277
26. Wang, L., Stoller, S.D.: Static analysis of atomicity for programs with non-blocking syn-
    chronization. In: PPoPP. (2005)
27. Amit, D., Rinetzky, N., Reps, T., Sagiv, M., Yahav, E.: Comparison under abstraction for
    verifying linearizability. In: CAV. (2007)
28. Manevich, R., Lev-Ami, T., Ramalingam, G., Sagiv, M., Berdine, J.: Heap decomposition for
    concurrent shape analysis. Technical Report TR-2007-11-85453, Tel Aviv University (2007)
29. Herlihy, M.P., Wing, J.M.: Linearizability: a correctness condition for concurrent objects.
    TOPLAS 12(3) (1990)
30. Flanagan, C., Qadeer, S.: A type and effect system for atomicity. In: PLDI, ACM (2003)
31. Flanagan, C.: Verifying commit-atomicity using model-checking. In: SPIN. (2004)

A    Additional Code

Fig. 4 shows the code of a concurrent (multi-threaded) client of the request pool module.

                  class ReqPoolMTClientMain {
                    static void main() {                       static void process(ReqPool p) {
                      ReqPool pool = new ReqPool();              p.handleReqInPool();
                      while (true)                             }
                        if (?)
                          fork process(pool);                  static void add(ReqPool p) {
                        else if (?)                              p.crtReqInPool();
                          fork add(pool);                      }
                        else (?)
                          fork done(pool);                     static void done(ReqPool p) {
                    }                                            p.donePool();
                  }                                            }

                     Fig. 4. A concurrent client of the REQPOOL module.

B Proofs

In this section we provide a brief sketch of the proof for [5, The. 1].
    Assume that in any serial execution of a sequence of operations on the given module,
every operation follows 2PL.
    Consider any execution trace τ produces by the interleaved execution of a set of
operations T1 , · · · , Tk . We say that a dependence exists between Th and Tj if some
instruction i1 executed by Th and some instruction i2 executed by Tj access the same
resource, and i1 precedes i2 in the execution trace τ . If this dependence relation is
acyclic, then any serial execution of the operations T1 , · · · , Tk that is in topological-
sort order with respect to these dependences is equivalent to the execution trace τ . Note
that this is valid even if some of the Ti s represent partially completed operations.
    The desired result obviously follows if the dependence relation is acyclic.
    We can establish that dependence relation is acyclic by induction on the length of
the execution trace τ . Assume that the dependence relation is acyclic for τ , and, hence,
that τ corresponds to the serial execution of a set of (possibly partially completed)
operations, say S1 , · · · , Sj . Consider the execution of any instruction i following τ .
Assume that i is generated by the execution of operation Sh . Any resource r accessed
by i must have already been locked by Sh (as Sh corresponds to a partial sequential
execution of an operation). Hence, r could not have been accessed by any operation Sm
where m > h. Hence, i can not create a dependence cycle.

C       Tree Locking: The Intuition
The idea behind the tree locking protocol may be understood by examining the exe-
cutions shown in Fig. 5.i and Fig. 5.ii. Fig. 5.i shows a serial execution of a sequence
of 3 operations on a data structure implemented following the tree-locking policy. The
effect of every operation is depicted by changing the color of the data structure . Note
that when op2 is invoked it “sees” the black data structure produced by op1 , which op-
erates on a white data structure. It cannot however, “see” the white data structure as op2
started manipulating the data structure only after op2 finished changing the color of the
tree into black. Similarly, op3 can only “see” the light gray tree.
    Consider the concurrent execution shown in Fig. 5.ii. Note that when op3 start ma-
nipulating the data structure, neither op1 nor op2 finished executing. However, the TL
policy, ensures that op3 cannot overtake op2 and that op2 cannot overtake op1 . Thus,
op3 “sees” a light gray tree and op2 light gray tree, exactly as in the serial execution.

                                           op1      op2      op3

                                   (i) Illustrated sequential execution.
                                                   op2       op2
                                          op1      op1       op1

                                  (ii) Illustrated concurrent execution.

                                      Fig. 5. Illustrated executions

D       On The Importance of Serializability
In this section, we discuss the importance of race-freedom, in general, and of serializ-
ability, in particular, using the (standard) bank account example.
Importance of Race-Freedom Module Acc-NoL, shown in Fig. 6(d), provides an im-
plementation of a bank account which does not use locks, thus multiple threads can
concurrently manipulate the same bank account. However, module ACC-NoL has some
unexpected behaviors with undesirable effects due to the absence of a synchronization
mechanism: Consider for example the multi-threaded client module, shown in Fig. 6(a),
which uses a bank account implemented by module ACC-NoL. The account is created
with an initial total balance of $0. It is then is manipulated concurrently by three threads:
one thread deposits $100; another thread saves $50; and the third thread queries about
the total amount of money in the account. the total operation might return that the
account has a total balance of $50. This unexpected result, at least from the point of
view of the holder of the bank account,9 is due to an interleaving in which the total
     She would expect the total balance to be either $0, before the deposit, or $100, after the deposit.

          operation reads both balances after the save operation removed the money from the
          saving account and before it added to the saving account.
          Importance of Serializability Fig. 6(c) shows an implementation of a bank account
          which is uses lock-splitting and, unlike module ACC-NoL, is race-free []: Every opera-
          tion on a shared data item is done under the protection of a lock. The locks, however,
          are not managed according to a specific locking policy, and in particular, there are no
          guarantees that the bank account implementation is serializable.
              Module ACC-TwoL has several undesired behaviors similar to the ones of module
          ACC-NoL. More specifically, total can obtain both locks after the save operation re-
          moved the money from the checking balance and released its lock but before it obtained
          the lock on the saving account and added to it the saved money.
              Note however, that if the bank account is implemented using (the serializable) mod-
          ule ACC-2PL, total can return either $0 or $100. Specifically, the save procedure in
          module ACC-2PL does not release its lock on the checking balance, before it acquires
          the lock of the saving balance.

       (a) module MTC-Acc                      (d) module Acc-NoL                               (c) module Acc-2L
class AccMTC {                                 class Acc {                      class Acc {
  static void main() {                           int checking=0;                  int checking=0;                 int total() {
    Acc acc = new Acc();                         int saving=0;                    int saving=0;                     this.Lc.lock();
    fork doDeposit(acc, 100);                                                     Object Lc = new Object();         int cur = this.checking;
    fork doSave(acc, 50);                          void deposit(int d) {          Object Ls = new Object();         this.Lc.unlock();
    fork doToal(acc);                                int old = this.checking;                                       this.Ls.lock();
  }                                                  this.checking += d;         void deposit(int d) {              cur += this.saving;
   static void doDeposit(Acc acc, int sum) {         return old;                   this.Lc.lock();                  this.Ls.unlock();
    acc.deposit(sum);                              }                               int old = this.checking;         return cur;
  }                                                                                this.checking += d;            }
                                                   int total() {                   this.Lc.unlock();
   static void doTotal(Acc acc) {                    int cur = this.checking;      return old;                    void save(int s) {;                                     cur += this.saving;         }                                  this.Lc.lock();
  }                                                  return cur;                                                    this.checking -= s;
   static void doSave(Acc acc, int sum) {          }                                                                this.Lc.unlock();;                                                                                                  this.Ls.lock();
  }                                                void save(int s) {                                               this.saving += s;
}                                                    this.checking -= s;                                            this.Ls.unlock();
                                                     this.saving += s;                                            }
                                                   }                                                          }

          Fig. 6. (a) multi threaded client of the bank account modules. (d) a implementation of the bank
          account data structure which has races. (c) a race-free implementation of the bank account data
          structure which is not serializable.


To top