Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

038 by xiuliliaofz

VIEWS: 3 PAGES: 6

									                              O p t i m i s t i c Parallel Discrete Relaxation*
                                        K i n s o n H o and Paul N . H i l f i n g e r
                                                Computer Science Division
                                   University of California at Berkeley, U. S. A.
                                                    Hans W . Guesgen
                                             Computer Science Department
                                        University of Auckland, New Zealand
                              {ho,hilfingr}@cs.berkeley.edu, hans@cs.auckland.ac.nz

                          Abstract                               an appropriate initial value for X, discrete relaxation re-
                                                                 peats the assignment                until a fixed point for
      Discrete relaxation is frequently used to com-             / is found. Monotonicity of / with respect to < is a suf-
      pute the fixed point of a discrete system                  ficient (but in general not necessary) condition for itera-
                      where / is monotonic with respect          tive, hill-climbing techniques such as discrete relaxation
      to some partial order          Given an appropriate        to find the fixed point of / [Parker, 1987].
      initial value for X, discrete relaxation repeats              Discrete relaxation is widely used in the solution of
      the assignment                    until a fixed point      constraint satisfaction problems (CSPs), and many par-
      for / is found. Monotonicity of / with respect             allel implementations of discrete relaxation for CSPs
      to        is a sufficient (but in general not neces-       have been reported [Kasif and Rosenfeld, 1983; Rosen-
      sary) condition for iterative, hill-climbing tech-         feld et a/., 1976]. These attempts have all focused on
      niques such as discrete relaxation to find the             CSP solvers that compute locally consistent (arc con-
      f i x e d point o f / .                                    sistent) solutions, which are relatively straightforward
       In this paper we introduce monotonic asyn-                to parallelize as the computations are inherently mono-
       chronous iteration as a novel way of imple-               tonic. On the other hand, discrete relaxation algorithms
      menting parallel discrete relaxation in prob-              used in CSP solvers that compute globally consistent so-
      lem domains for which monotonicity is a neces-             lutions are very difficult to parallelize because for this
      sary condition. This is an optimistic technique            class of problems, monotonicity is a necessary correct-
      that maintains monotonicity without limiting               ness condition that is not automatically satisfied. The
      concurrency, resulting in good parallel perfor-            need to maintain monotonicity (for correctness) often
      mance. We illustrate this technique with the               limits the amount of concurrency available in a paral-
      parallel implementation of a constraint satisfac-          lel implementation, degrading the performance signifi-
      tion system that computes globally consistent              cantly.
      solutions, and present performance numbers for                 In this paper we introduce monotonic asynchronous
      experiments on a shared-memory implementa-                  iteration as a novel way of implementing parallel dis-
      tion. The performance numbers show that it is              crete relaxation in problem domains for which mono-
      indeed possible to obtain a reasonable speedup             tonicity is a necessary condition. This is an optimistic
      when parallelizing global constraint satisfac-             technique that maintains monotonicity without limiting
      tion. We believe that monotonic asynchronous               concurrency, resulting in good parallel performance. We
      iteration is applicable to parallel discrete relax-        illustrate this technique with the parallel implementa-
      ation in general.                                          tion of CONSAT [Guesgen, 1989], a constraint satisfac-
                                                                 tion system that computes globally consistent solutions,
                                                                 and describes an experiment on a shared-memory im-
1     Introduction                                               plementation. The performance numbers show that it is
                                                                 indeed possible to obtain a reasonable speedup when par-
Discrete relaxation is frequently used to compute the
                                                                 allelizing global constraint satisfaction, and thus proving
fixed point of a discrete system            where / is
                                                                 that Kasif's conjecture is correct [1990]. We believe that
monotonic with respect to some partial order <. Given
                                                                 monotonic asynchronous iteration is applicable to paral-
   *Ho and Hilfinger are supported by NSF Grant CCR-84-          lel discrete relaxation in general.
51213. Guesgen performed part of this work while at the Ger-
man National Research Center for Computer Science ( G M D ) .
in St. Augustin, Germany, and the International Computer         2    Discrete R e l a x a t i o n
Science Institute in Berkekey, California. At the G M D he
was supported by the German Federal Ministry for Research        Consider the problem of finding the fixed point of a dis-
and Technology ( B M F T ) within the joint projects T E X - B   crete system .             For our purposes, the system
(grant ITW8506D) and TASSO (grant ITW8900A7).                    has the following properties:


268       Constraint Satisfaction Problems
Ho, Hilfinger, and Guesgen   269
                                         is considerable freedom in applying application-specific
                                         scheduling strategies for good performance.

                                         3     Example: C O N S A T
                                         CONSAT is a system for the definition and satisfaction
                                         of constraints in arbitrary finite discrete domains. A
                                         constraint consists of a set of variables and a relation
                                         among the variables, and a constraint network is a set
                                         of constraints connected by common variables. A glob-
                                          ally consistent solution of a constraint network is a tuple
                                         of values, one per variable of the network, that satisfies
                                         all the constraints of the network simultaneously. CON-
                                         SAT is a global CSP solver whose constraint satisfaction
                                         technique is based on filtering [Waltz, 1972], i.e., on the
                                         successive deletion of inconsistent values from the set
                                         of potential values for the variables. Unlike traditional
                                         filtering algorithms, CONSAT uses values that are asso-
                                         ciated with some additional information (called tags) for
                                         maintaining interrelationships among values (of different
                                         variables). In the following we give an informal explana-
                                         tion of how CONSAT modifies local constraint propaga-
                                         tion, a technique commonly used to compute locally con-
                                         sistent solutions, with tagging to compute the globally
                                         consistent solutions of a constraint network. As global
                                         constraint satisfaction problems over finite domains are
                                         generally NP-complete, the algorithm used by CONSAT
                                         is exponential in the worst case. A more formal treat-
                                         ment of CONSAT is given elsewhere [Guesgen, 1989].
                                         C o n s t r a i n t N e t w o r k E x a m p l e Figure 1 defines con-
                                         straint network A, which will be used in the examples
                                         throughout this paper. There are three variables (Va,
                                         Vb, Vc) and six constraints (C1-C6). For example, con-
                                         straint C1 restricts the possible values of Va to R or Y,
                                         and constraint C4 restricts the values of (Va,Vb) to one
                                         of the combinations (R,G) or (R,B). Va and Vb are called
                                         the adjacent variables of constraint C4. (A constraint
                                         may be adjacent to more than two variables, i.e., we do
                                         not restrict ourselves to binary constraints.) The steps
                                         of CONSAT as it computes the global solutions of con-
                                         straint network A are shown in Figure 2. Details of the
                                         algorithm used may be found elsewhere [Guesgen, 1989].




                                         Figure 1: Constraint Network A: Variables are repre-
                                         sented by circles and constraints by rectangles. An edge
                                         between a circle and a rectangle means that the corre-
                                         sponding variable belongs to the constraint represented
                                         by the rectangle.

                                         Tagged Values For any variable of a constraint net-
                                         work, the potential values are associated with additional
                                         information called tags, and tagged values (of different
                                         variables) with the same f u l l tag form a global solution
                                         of the constraint network. For a constraint network of


270   Constraint Satisfaction Problems
                                                                m constraints, each full tag is an m-tuple, where the
                                                                r t h component (the r t h subtag) indicates the tuple in
                                                                the r t h constraint that is part of the global solution.
                                                                (For example, in constraint C4 of Figure 1 the tuple
                                                                (R,B) corresponds to the subtag 2.) The special symbol
                                                                "-" is the wildcard character for subtags and indicates
                                                                that a tuple for the corresponding constraint has not
                                                                been chosen. A subtag with a tuple number is said to
                                                                be determined, while one with the special symbol "-"
                                                                is said to be undetermined. As an example, consider
                                                                the global solutions of constraint network A shown in
                                                                Figure 2. The two solutions (Va,Vb,Vc) are (R,G,B)
                                                                with full tag ( 1 , 1 , 1 , 1 , 1 , 1 ) and (R,B,G) with full tag
                                                                ( 1 , 3 , 3 , 2 , 2 , 2 ) . An intuitive explanation of the use of
                                                                tags for constraint network A is given in Figure 3.

                                                                F i l t e r i n g of Tagged Values The feasible set of a vari-
                                                                able contains its current set of potential (tagged) values,
                                                                and is initialized to the special value Unconstrained,
                                                                meaning the variable may take any value from the do-
                                                                main under consideration. The filtering function / of
                                                                the constraint network uses local propagation of tagged
                                                                values to eliminate inconsistent values from the feasi-
                                                                ble sets, and replaces undetermined subtags with deter-
                                                                mined subtags. Guesgen showed in his thesis that filter-
                                                                ing of tagged values is guaranteed to terminate with the
                                                                globally consistent solutions of a constraint network in
                                                                a finite number of steps provided that it is fair [1989].
                                                                This means each constraint is evaluated (or activated)
                                                                at least once, and if a constraint changes the feasible
                                                                set of any adjacent variable during filtering, other con-
                                                                straints adjacent to the modified variable have to be re-
                                                                activated. (This is not the usual definition of fairness
Figure 2: Activations of constraint network A: Each con-        in the parallel programming literature.) Upon termina-
straint being activated (leftmost column) is shown with         tion of local propagation, each tuple of values—one per
the feasible sets of its adjacent variables after its activa-   variable—with identical full tags form a globally consis-
tion. An " * " at the end of a variable means that it has       tent solution of the constraint network (see Figure 2). If
not been changed in the current activation. The feasible        the feasible set of any variable becomes empty, there is
set of any variable non-adjacent to the current constraint      no solution for the constraint network. Local propaga-
can be found by searching backwards from the current            tion computes the fixed point of the filtering function /
activation. (Componentwise unification of full tags is the      of the constraint network.
key idea behind the algorithm.)




                                                                   Intuitively, filtering either eliminates a tagged value if
                                                                it is inconsistent, or replaces one or more undetermined
                                                                subtags of a tagged value by determined subtags. A de-
Figure 3: Globally Consistent Solutions of Constraint           termined subtag is never replaced by an undetermined
Network A                                                       subtag or another determined subtag. A more formal ex-
                                                                planation of why filtering of tagged values is monotonic


                                                                                     Ho, Hilflnger, and Guesgen              271
 with respect to < will be given elsewhere [Ho, 1993].                ing) to line drawings in a polyhedral world of trihedral
                                                                      vertices [Horn, 1986]. The constraints restrict the la-
 C O N S A T as D i s c r e t e R e l a x a t i o n In this section
                                                                      beling of edges meeting at a vertex to be the few com-
 we show that CONSAT is a special case of discrete re-
                                                                      binations physically possible. In addition, each edge is
 laxation as defined in Section 2.
                                                                      constrained to have the same label at both ends (where
      • CONSAT computes the fixed point of the discrete               it meets other edges). The particular scene (constraint
        system X = f(X) for the non-deterministic / that              network) chosen, Stair5, has 33 variables and 56 con-
        corresponds to the filtering function of the entire           straints (36 2-variable and 20 3-variable constraints).
        constraint network. This fixed point corresponds to
        the set of all the globally consistent solutions of the
        constraint network.
      • / = { 1 , . . . , n) is the set of all the variables of the
        constraint network.
      • / may be decomposed into component functions fc
        for constraints                           is the filtering
        function of constraint c, and eliminates value com-
        binations inconsistent with c from the feasible sets
        of its adjacent variables. Without loss of generality,
        we assume that a constraint may be uniquely iden-             Table 1: Performance of CONSAT for Stair5: P is the
        tified by its set of adjacent variables. For example,         number of processors, and Seq is the sequential imple-
        /{2,3,5} is the filtering function of the constraint that     mentation. A l l times are real time in seconds, no gc is
        is adjacent to variables 2, 3 and 5. fc only depends          time without garbage collection, gc is time for garbage
        on the values of variables          and fc(X) only dif-       collection, and T is the total time. Speedup is based on
        fers from X for the same set of variables. The new            times excluding garbage collection. Both sequential and
        value of any such variable i is given by                      parallel versions are compiled with the highest optimiza-
   • For                                       Each              is   tion setting for speed. Steps is the number of discrete
     the feasible set of variable i.                                  relaxation steps, including non-monotonic ones. Abort
                                                                      is the number of non-monotonic steps (re-executed).
   • A"0 is the initial state of the feasible sets of all the
     variables. A l l its components have the special value
     Unconstrained.
                                                                         The results for Stair5 are summarized in Table 1.
   • The partial order < on D has been defined on                     Speedup ranges between 0.9 on one processor to 4.6 on
     page 4.                                                          six processors. The one processor time is within 5% of
   We have shown that CONSAT is a special case of dis-                the sequential time, showing that the parallel implemen-
crete relaxation, and have defined an application-specific            tation is reasonably efficient. The speedup is less than
monotonicity test between successive states of the relax-             linear because a small number of constraint activations
ation. Consequently, an efficient parallel implementation             (relaxation steps) have to be re-executed because they vi-
of CONSAT may be obtained by using monotonic asyn-                    olate monotonicity. Serial bottlenecks in the allocation
chronous iteration.                                                   routines of the current CLiP implementation also pre-
                                                                      vent parallel CONSAT from achieving better speedup.
 P e r f o r m a n c e We implemented parallel CONSAT us-                An earlier parallel implementation of CONSAT that
 ing CLiP [Franz Inc., 1990], a commercial imple-                     performs the update to the feasible sets (X) in a single
mentation of the multiprocessing features of SPUR                     critical section for each constraint activation has virtu-
 Lisp [Zorn et al., 1989] that currently runs on the shared           ally no speedup because contention for X serializes all
memory Sequent Symmetry multiprocessor.            To esti-           the concurrent constraint activations. The current im-
mate the performance of parallel CONSAT in the ab-                    plementation based on monotonic asynchronous itera-
sence of garbage collection—an upper bound on paral-                  tion performs significantly better.
lel performance—we define speedup as the ratio of real
time excluding garbage collection of the sequential ver-
                                                                      4    Related W o r k in Parallel R e l a x a t i o n
sion relative to the real time (excluding garbage collec-
tion) of the parallel version on n processors. This def-              The formulation of asynchronous iterations in Section 2
inition is chosen because the current CLiP implemen-                  is similar to generalized iterations defined by Pohlmann
tation uses a sequential garbage collector that stops all             in the context of parallel discrete event simulation [1991].
but one processor each time a collection occurs. We feel              Generalized iterations operate in the domain of infinite
our definition better models the performance of parallel              streams of elements, each of which corresponds to an
CONSAT in a parallel Lisp system with a more realis-                  event over time in the system being simulated.
tic garbage collector, such as the concurrent collector in               Parallel relaxation is also used by various schemes for
TOP-1 Common Lisp [Tanaka and Uzuhara, 1990].                         solving systems of equations X <— f(X) in the domain
   We measured the parallel performance of CONSAT                     of real numbers in parallel, including chaotic relaxations
for a problem in machine vision, which assigns three-                 defined by Chazan and Miranker [1969], asynchronous it-
dimensional edge labelings (convex, concave, or occlud-               erations defined by Baudet [1978] and chaotic iterations


272        Constraint Satisfaction Problems
w i t h delay defined by M i e l l o u [1975]. In these systems                          i n A r t i f i c i a l Intelligence. M o r g a n K a u f m a n n , San M a -
/ is m a d e up of c o m p o n e n t f u n c t i o n s f c , a n d t h e state           teo, C a l i f o r n i a , 1989.
X of t h e f i x e d - p o i n t c o m p u t a t i o n m a y be decomposed
                                                                                      [Ho, 1993] K i n s o n H o . High-level abstractions f o r s y m -
i n t o a set of ( p o t e n t i a l l y d i s j o i n t ) components t h a t are
                                                                                        bolic p a r a l l e l p r o g r a m m i n g . P h D thesis, C o m p u t e r
c o m p u t e d by different f c f u n c t i o n s concurrently. Each
                                                                                        Science D i v i s i o n ( E E C S ) , U n i v e r s i t y o f C a l i f o r n i a ,
f c m a y b e c o m p u t e d using m u l t i p l e previous states, w i t h
                                                                                        Berkeley, C a l i f o r n i a , 1993. To appear.
a different state for each c o m p o n e n t o f X , t o m i n i m i z e
the a m o u n t of s y n c h r o n i z a t i o n r e q u i r e d . These schemes      [ H o r n , 1986] B e r t h o l d K l a u s P a u l H o r n . Robot Vision.
are o p t i m i z e d for t h e d o m a i n of real numbers by using                      M I T Press, C a m b r i d g e , Massachusetts, 1986.
the properties of real n u m b e r s , and t h e i r convergence
c r i t e r i a are s o m e w h a t analogous to the test for m o n o -               [Kasif a n d Rosenfeld, 1983] S i m o n K a s i f and A z r i e l
t o n i c i t y used in m o n o t o n i c asynchronous i t e r a t i o n s . O u r      Rosenfeld. T h e fixed p o i n t s of images and scenes. In
f o r m u l a t i o n o f asynchronous i t e r a t i o n s for parallel dis-            Proceedings C V P R '83: I E E E C o m p u t e r Society Con-
crete r e l a x a t i o n is a d o m a i n - i n d e p e n d e n t generalization       ference on C o m p u t e r Vision a n d P a t t e r n Recognition,
of these parallel i t e r a t i o n schemes. In a d d i t i o n , the use               pages 4 5 4 - 4 5 6 , W a s h i n g t o n , D C , June 1983.
of an o p t i m i s t i c test for m o n o t o n i c i t y for i m p r o v e d par-   [Kasif, 1990] S i m o n Kasif. On t h e parallel c o m p l e x i t y of
allel p e r f o r m a n c e is u n i q u e to our approach.
                                                                                        discrete r e l a x a t i o n in c o n s t r a i n t satisfaction networks.
                                                                                        A r t i f i c i a l Intelligence, 4 5 ( 3 ) : 2 7 5 - 2 8 6 , O c t o b e r 1990.
5      Conclusion
                                                                                      [ M i e l l o u , 1975] Jean-Claude M i e l l o u . I t e r a t i o n s chao-
In t h i s paper we proposed m o n o t o n i c asynchronous i t -                        tiques a retards; etudes de la convergence dans le
e r a t i o n as a correct and efficient way of i m p l e m e n t i n g                  cas d'espaes p a r t i e l l e m e n t ordonnes ( C h a o t i c itera-
parallel discrete r e l a x a t i o n for systems for w h i c h m o n o -                tions w i t h delay; studies of convergence for the case
t o n i c i t y is a necessary correctness c o n d i t i o n .           Mono-           of p a r t i a l l y ordered spaces).         Comptes Rendus Heb-
tonic asynchronous i t e r a t i o n uses an o p t i m i s t i c scheme                   dom ad aires des Seances De L 'Academic des Sciences,
to c o m p u t e a possible next state of the system. T h i s                            280, Series A ( 4 ) : 2 3 3 ~ 2 3 6 , J a n u a r y 1975. In French.
o p t i m i s t i c scheme is h i g h l y efficient b u t is not necessar-
i l y m o n o t o n i c (i.e., correct). A n application-specific test                [Parker, 1987] D. S t o t t Parker. P a r t i a l order p r o g r a m -
for m o n o t o n i c i t y is then applied to the c o m p u t e d state.               m i n g . Technical R e p o r t CSD-870067, C o m p u t e r Sci-
If the test succeeds, the state t r a n s i t i o n is made ( a t o m -                 ence D e p a r t m e n t , U n i v e r s i t y of C a l i f o r n i a , Los Ange-
i c a l l y ) . O t h e r w i s e , the c o m p u t a t i o n is repeated using         les, C a l i f o r n i a , December 1987.
the current state.                  We have applied our technique to
                                                                                      [ P o h l m a n n , 1991] Werner P o h l m a n n . A fixed p o i n t ap-
the parallel i m p l e m e n t a t i o n of a c o n s t r a i n t satisfaction
                                                                                          proach t o parallel discrete event s i m u l a t i o n . A c t a I n -
system t h a t computes g l o b a l l y consistent solutions, for
                                                                                          f o r m a t i c a , 2 8 ( 7 ) : 6 1 1 - 6 2 9 , October 1991.
w h i c h m o n o t o n i c i t y is a necessary correctness c o n d i t i o n
t h a t is not a u t o m a t i c a l l y satisfied. We believe m o n o t o n i c      [Rosenfeld et a i , 1976] A z r i e l Rosenfeld,               R o b e r t A.
asynchronous i t e r a t i o n is applicable to parallel discrete                       H u m m e l , and Steven W. Zucker. Scene l a b e l i n g by re-
r e l a x a t i o n in general. It w i l l be interesting to see if this                l a x a t i o n o p e r a t i o n s . I E E E Transactions o n Systems,
application-specific m o n o t o n i c i t y test is easy to derive                     M a n and Cybernetics, 6 ( 6 ) : 4 2 0 - 4 3 3 , June 1976.
for other discrete r e l a x a t i o n p r o b l e m s .
                                                                                      [ S c h m i d t , 1986] D a v i d A. S c h m i d t . D e n o t a t i o n a l Se-
                                                                                          m a n t i c s : A Methodology f o r Language Development.
6       Acknowledgements
                                                                                          A l l y n and B a c o n , B o s t o n , Massachusetts, 1986.
We t h a n k Suresh K r i s h n a for t r a n s l a t i n g M i e l l o u ' s paper
                                                                                      [Stoy, 1977] Joseph E. Stoy. D e n o t a t i o n a l Semantics:
o n chaotic i t e r a t i o n s f r o m French i n t o E n g l i s h , and E d
                                                                                         The Scott-Strachey Approach to P r o g r a m m i n g L a n -
W a n g for e x p l a i n i n g the subtleties o f l a t t i c e theory. W e
                                                                                         guage Theory. M I T Press, C a m b r i d g e , Massachusetts,
also t h a n k C h u - C h e o w L i m , E d W a n g , L u i g i Semenzato
                                                                                         1977.
and K a t h y Yelick for t h e i r c o m m e n t s on various drafts
o f t h i s paper.                                                                    [Tanaka a n d U z u h a r a , 1990]
                                                                                        T o m o y u k i T a n a k a and Shigeru U z u h a r a . M u l t i p r o -
References                                                                              cessor C o m m o n Lisp o n T O P - 1 . I n Proceedings o f the
                                                                                        Second I E E E Symposium o n P a r a l l e l and D i s t r i b u t e d
[ B a u d e t , 1978] G e r a r d M. B a u d e t . Asynchronous iter-
                                                                                        Processing, D a l l a s , Texas, December 1990.
    a t i v e m e t h o d s for multiprocessors.        J o u r n a l of the
    A C M , 2 5 ( 2 ) : 2 2 6 - 2 4 4 , A p r i l 1978.                               [ W a l t z , 1972] D. L. W a l t z . G e n e r a t i n g semantic descrip-
[Chazan and M i r a n k e r , 1969] D . Chazan and W . M i -                             t i o n s f r o m d r a w i n g s of scenes w i t h shadows. Techni-
  ranker. C h a o t i c r e l a x a t i o n . L i n e a r Algebra and its                cal R e p o r t A I - T R - 2 7 1 , M I T L a b o r a t o r y for C o m p u t e r
  A p p l i c a t i o n s , 2 : 1 9 9 - 2 2 2 , 1969.                                    Science, C a m b r i d g e , Massachusetts, 1972.

[Franz I n c . , 1990] Franz Inc. Allegro C L i P M a n u a l , re-                   [Zorn et al., 1989] B e n j a m i n Z o r n , K i n s o n H o , James
   lease 3.0.3 e d i t i o n , M a r c h 1990.                                          L a r u s , L u i g i Semenzato, and Paul Hilfinger. M u l t i -
[Guesgen, 1989] Hans Werner Guesgen. C O N S A T : A                                    processing extensions i n Spur Lisp. I E E E Software,
  System f o r C o n s t r a i n t S a t i s f a c t i o n . Research Notes             6 ( 4 ) : 4 1 - 4 9 , J u l y 1989.



                                                                                                                  Ho, Hilfinger, and Guesgen                          273

								
To top