VIEWS: 3 PAGES: 6 POSTED ON: 3/1/2012
O p t i m i s t i c Parallel Discrete Relaxation* K i n s o n H o and Paul N . H i l f i n g e r Computer Science Division University of California at Berkeley, U. S. A. Hans W . Guesgen Computer Science Department University of Auckland, New Zealand {ho,hilfingr}@cs.berkeley.edu, hans@cs.auckland.ac.nz Abstract an appropriate initial value for X, discrete relaxation re- peats the assignment until a fixed point for Discrete relaxation is frequently used to com- / is found. Monotonicity of / with respect to < is a suf- pute the fixed point of a discrete system ficient (but in general not necessary) condition for itera- where / is monotonic with respect tive, hill-climbing techniques such as discrete relaxation to some partial order Given an appropriate to find the fixed point of / [Parker, 1987]. initial value for X, discrete relaxation repeats Discrete relaxation is widely used in the solution of the assignment until a fixed point constraint satisfaction problems (CSPs), and many par- for / is found. Monotonicity of / with respect allel implementations of discrete relaxation for CSPs to is a sufficient (but in general not neces- have been reported [Kasif and Rosenfeld, 1983; Rosen- sary) condition for iterative, hill-climbing tech- feld et a/., 1976]. These attempts have all focused on niques such as discrete relaxation to find the CSP solvers that compute locally consistent (arc con- f i x e d point o f / . sistent) solutions, which are relatively straightforward In this paper we introduce monotonic asyn- to parallelize as the computations are inherently mono- chronous iteration as a novel way of imple- tonic. On the other hand, discrete relaxation algorithms menting parallel discrete relaxation in prob- used in CSP solvers that compute globally consistent so- lem domains for which monotonicity is a neces- lutions are very difficult to parallelize because for this sary condition. This is an optimistic technique class of problems, monotonicity is a necessary correct- that maintains monotonicity without limiting ness condition that is not automatically satisfied. The concurrency, resulting in good parallel perfor- need to maintain monotonicity (for correctness) often mance. We illustrate this technique with the limits the amount of concurrency available in a paral- parallel implementation of a constraint satisfac- lel implementation, degrading the performance signifi- tion system that computes globally consistent cantly. solutions, and present performance numbers for In this paper we introduce monotonic asynchronous experiments on a shared-memory implementa- iteration as a novel way of implementing parallel dis- tion. The performance numbers show that it is crete relaxation in problem domains for which mono- indeed possible to obtain a reasonable speedup tonicity is a necessary condition. This is an optimistic when parallelizing global constraint satisfac- technique that maintains monotonicity without limiting tion. We believe that monotonic asynchronous concurrency, resulting in good parallel performance. We iteration is applicable to parallel discrete relax- illustrate this technique with the parallel implementa- ation in general. tion of CONSAT [Guesgen, 1989], a constraint satisfac- tion system that computes globally consistent solutions, and describes an experiment on a shared-memory im- 1 Introduction plementation. The performance numbers show that it is indeed possible to obtain a reasonable speedup when par- Discrete relaxation is frequently used to compute the allelizing global constraint satisfaction, and thus proving fixed point of a discrete system where / is that Kasif's conjecture is correct [1990]. We believe that monotonic with respect to some partial order <. Given monotonic asynchronous iteration is applicable to paral- *Ho and Hilfinger are supported by NSF Grant CCR-84- lel discrete relaxation in general. 51213. Guesgen performed part of this work while at the Ger- man National Research Center for Computer Science ( G M D ) . in St. Augustin, Germany, and the International Computer 2 Discrete R e l a x a t i o n Science Institute in Berkekey, California. At the G M D he was supported by the German Federal Ministry for Research Consider the problem of finding the fixed point of a dis- and Technology ( B M F T ) within the joint projects T E X - B crete system . For our purposes, the system (grant ITW8506D) and TASSO (grant ITW8900A7). has the following properties: 268 Constraint Satisfaction Problems Ho, Hilfinger, and Guesgen 269 is considerable freedom in applying application-specific scheduling strategies for good performance. 3 Example: C O N S A T CONSAT is a system for the definition and satisfaction of constraints in arbitrary finite discrete domains. A constraint consists of a set of variables and a relation among the variables, and a constraint network is a set of constraints connected by common variables. A glob- ally consistent solution of a constraint network is a tuple of values, one per variable of the network, that satisfies all the constraints of the network simultaneously. CON- SAT is a global CSP solver whose constraint satisfaction technique is based on filtering [Waltz, 1972], i.e., on the successive deletion of inconsistent values from the set of potential values for the variables. Unlike traditional filtering algorithms, CONSAT uses values that are asso- ciated with some additional information (called tags) for maintaining interrelationships among values (of different variables). In the following we give an informal explana- tion of how CONSAT modifies local constraint propaga- tion, a technique commonly used to compute locally con- sistent solutions, with tagging to compute the globally consistent solutions of a constraint network. As global constraint satisfaction problems over finite domains are generally NP-complete, the algorithm used by CONSAT is exponential in the worst case. A more formal treat- ment of CONSAT is given elsewhere [Guesgen, 1989]. C o n s t r a i n t N e t w o r k E x a m p l e Figure 1 defines con- straint network A, which will be used in the examples throughout this paper. There are three variables (Va, Vb, Vc) and six constraints (C1-C6). For example, con- straint C1 restricts the possible values of Va to R or Y, and constraint C4 restricts the values of (Va,Vb) to one of the combinations (R,G) or (R,B). Va and Vb are called the adjacent variables of constraint C4. (A constraint may be adjacent to more than two variables, i.e., we do not restrict ourselves to binary constraints.) The steps of CONSAT as it computes the global solutions of con- straint network A are shown in Figure 2. Details of the algorithm used may be found elsewhere [Guesgen, 1989]. Figure 1: Constraint Network A: Variables are repre- sented by circles and constraints by rectangles. An edge between a circle and a rectangle means that the corre- sponding variable belongs to the constraint represented by the rectangle. Tagged Values For any variable of a constraint net- work, the potential values are associated with additional information called tags, and tagged values (of different variables) with the same f u l l tag form a global solution of the constraint network. For a constraint network of 270 Constraint Satisfaction Problems m constraints, each full tag is an m-tuple, where the r t h component (the r t h subtag) indicates the tuple in the r t h constraint that is part of the global solution. (For example, in constraint C4 of Figure 1 the tuple (R,B) corresponds to the subtag 2.) The special symbol "-" is the wildcard character for subtags and indicates that a tuple for the corresponding constraint has not been chosen. A subtag with a tuple number is said to be determined, while one with the special symbol "-" is said to be undetermined. As an example, consider the global solutions of constraint network A shown in Figure 2. The two solutions (Va,Vb,Vc) are (R,G,B) with full tag ( 1 , 1 , 1 , 1 , 1 , 1 ) and (R,B,G) with full tag ( 1 , 3 , 3 , 2 , 2 , 2 ) . An intuitive explanation of the use of tags for constraint network A is given in Figure 3. F i l t e r i n g of Tagged Values The feasible set of a vari- able contains its current set of potential (tagged) values, and is initialized to the special value Unconstrained, meaning the variable may take any value from the do- main under consideration. The filtering function / of the constraint network uses local propagation of tagged values to eliminate inconsistent values from the feasi- ble sets, and replaces undetermined subtags with deter- mined subtags. Guesgen showed in his thesis that filter- ing of tagged values is guaranteed to terminate with the globally consistent solutions of a constraint network in a finite number of steps provided that it is fair [1989]. This means each constraint is evaluated (or activated) at least once, and if a constraint changes the feasible set of any adjacent variable during filtering, other con- straints adjacent to the modified variable have to be re- activated. (This is not the usual definition of fairness Figure 2: Activations of constraint network A: Each con- in the parallel programming literature.) Upon termina- straint being activated (leftmost column) is shown with tion of local propagation, each tuple of values—one per the feasible sets of its adjacent variables after its activa- variable—with identical full tags form a globally consis- tion. An " * " at the end of a variable means that it has tent solution of the constraint network (see Figure 2). If not been changed in the current activation. The feasible the feasible set of any variable becomes empty, there is set of any variable non-adjacent to the current constraint no solution for the constraint network. Local propaga- can be found by searching backwards from the current tion computes the fixed point of the filtering function / activation. (Componentwise unification of full tags is the of the constraint network. key idea behind the algorithm.) Intuitively, filtering either eliminates a tagged value if it is inconsistent, or replaces one or more undetermined subtags of a tagged value by determined subtags. A de- Figure 3: Globally Consistent Solutions of Constraint termined subtag is never replaced by an undetermined Network A subtag or another determined subtag. A more formal ex- planation of why filtering of tagged values is monotonic Ho, Hilflnger, and Guesgen 271 with respect to < will be given elsewhere [Ho, 1993]. ing) to line drawings in a polyhedral world of trihedral vertices [Horn, 1986]. The constraints restrict the la- C O N S A T as D i s c r e t e R e l a x a t i o n In this section beling of edges meeting at a vertex to be the few com- we show that CONSAT is a special case of discrete re- binations physically possible. In addition, each edge is laxation as defined in Section 2. constrained to have the same label at both ends (where • CONSAT computes the fixed point of the discrete it meets other edges). The particular scene (constraint system X = f(X) for the non-deterministic / that network) chosen, Stair5, has 33 variables and 56 con- corresponds to the filtering function of the entire straints (36 2-variable and 20 3-variable constraints). constraint network. This fixed point corresponds to the set of all the globally consistent solutions of the constraint network. • / = { 1 , . . . , n) is the set of all the variables of the constraint network. • / may be decomposed into component functions fc for constraints is the filtering function of constraint c, and eliminates value com- binations inconsistent with c from the feasible sets of its adjacent variables. Without loss of generality, we assume that a constraint may be uniquely iden- Table 1: Performance of CONSAT for Stair5: P is the tified by its set of adjacent variables. For example, number of processors, and Seq is the sequential imple- /{2,3,5} is the filtering function of the constraint that mentation. A l l times are real time in seconds, no gc is is adjacent to variables 2, 3 and 5. fc only depends time without garbage collection, gc is time for garbage on the values of variables and fc(X) only dif- collection, and T is the total time. Speedup is based on fers from X for the same set of variables. The new times excluding garbage collection. Both sequential and value of any such variable i is given by parallel versions are compiled with the highest optimiza- • For Each is tion setting for speed. Steps is the number of discrete the feasible set of variable i. relaxation steps, including non-monotonic ones. Abort is the number of non-monotonic steps (re-executed). • A"0 is the initial state of the feasible sets of all the variables. A l l its components have the special value Unconstrained. The results for Stair5 are summarized in Table 1. • The partial order < on D has been defined on Speedup ranges between 0.9 on one processor to 4.6 on page 4. six processors. The one processor time is within 5% of We have shown that CONSAT is a special case of dis- the sequential time, showing that the parallel implemen- crete relaxation, and have defined an application-specific tation is reasonably efficient. The speedup is less than monotonicity test between successive states of the relax- linear because a small number of constraint activations ation. Consequently, an efficient parallel implementation (relaxation steps) have to be re-executed because they vi- of CONSAT may be obtained by using monotonic asyn- olate monotonicity. Serial bottlenecks in the allocation chronous iteration. routines of the current CLiP implementation also pre- vent parallel CONSAT from achieving better speedup. P e r f o r m a n c e We implemented parallel CONSAT us- An earlier parallel implementation of CONSAT that ing CLiP [Franz Inc., 1990], a commercial imple- performs the update to the feasible sets (X) in a single mentation of the multiprocessing features of SPUR critical section for each constraint activation has virtu- Lisp [Zorn et al., 1989] that currently runs on the shared ally no speedup because contention for X serializes all memory Sequent Symmetry multiprocessor. To esti- the concurrent constraint activations. The current im- mate the performance of parallel CONSAT in the ab- plementation based on monotonic asynchronous itera- sence of garbage collection—an upper bound on paral- tion performs significantly better. lel performance—we define speedup as the ratio of real time excluding garbage collection of the sequential ver- 4 Related W o r k in Parallel R e l a x a t i o n sion relative to the real time (excluding garbage collec- tion) of the parallel version on n processors. This def- The formulation of asynchronous iterations in Section 2 inition is chosen because the current CLiP implemen- is similar to generalized iterations defined by Pohlmann tation uses a sequential garbage collector that stops all in the context of parallel discrete event simulation [1991]. but one processor each time a collection occurs. We feel Generalized iterations operate in the domain of infinite our definition better models the performance of parallel streams of elements, each of which corresponds to an CONSAT in a parallel Lisp system with a more realis- event over time in the system being simulated. tic garbage collector, such as the concurrent collector in Parallel relaxation is also used by various schemes for TOP-1 Common Lisp [Tanaka and Uzuhara, 1990]. solving systems of equations X <— f(X) in the domain We measured the parallel performance of CONSAT of real numbers in parallel, including chaotic relaxations for a problem in machine vision, which assigns three- defined by Chazan and Miranker [1969], asynchronous it- dimensional edge labelings (convex, concave, or occlud- erations defined by Baudet [1978] and chaotic iterations 272 Constraint Satisfaction Problems w i t h delay defined by M i e l l o u [1975]. In these systems i n A r t i f i c i a l Intelligence. M o r g a n K a u f m a n n , San M a - / is m a d e up of c o m p o n e n t f u n c t i o n s f c , a n d t h e state teo, C a l i f o r n i a , 1989. X of t h e f i x e d - p o i n t c o m p u t a t i o n m a y be decomposed [Ho, 1993] K i n s o n H o . High-level abstractions f o r s y m - i n t o a set of ( p o t e n t i a l l y d i s j o i n t ) components t h a t are bolic p a r a l l e l p r o g r a m m i n g . P h D thesis, C o m p u t e r c o m p u t e d by different f c f u n c t i o n s concurrently. Each Science D i v i s i o n ( E E C S ) , U n i v e r s i t y o f C a l i f o r n i a , f c m a y b e c o m p u t e d using m u l t i p l e previous states, w i t h Berkeley, C a l i f o r n i a , 1993. To appear. a different state for each c o m p o n e n t o f X , t o m i n i m i z e the a m o u n t of s y n c h r o n i z a t i o n r e q u i r e d . These schemes [ H o r n , 1986] B e r t h o l d K l a u s P a u l H o r n . Robot Vision. are o p t i m i z e d for t h e d o m a i n of real numbers by using M I T Press, C a m b r i d g e , Massachusetts, 1986. the properties of real n u m b e r s , and t h e i r convergence c r i t e r i a are s o m e w h a t analogous to the test for m o n o - [Kasif a n d Rosenfeld, 1983] S i m o n K a s i f and A z r i e l t o n i c i t y used in m o n o t o n i c asynchronous i t e r a t i o n s . O u r Rosenfeld. T h e fixed p o i n t s of images and scenes. In f o r m u l a t i o n o f asynchronous i t e r a t i o n s for parallel dis- Proceedings C V P R '83: I E E E C o m p u t e r Society Con- crete r e l a x a t i o n is a d o m a i n - i n d e p e n d e n t generalization ference on C o m p u t e r Vision a n d P a t t e r n Recognition, of these parallel i t e r a t i o n schemes. In a d d i t i o n , the use pages 4 5 4 - 4 5 6 , W a s h i n g t o n , D C , June 1983. of an o p t i m i s t i c test for m o n o t o n i c i t y for i m p r o v e d par- [Kasif, 1990] S i m o n Kasif. On t h e parallel c o m p l e x i t y of allel p e r f o r m a n c e is u n i q u e to our approach. discrete r e l a x a t i o n in c o n s t r a i n t satisfaction networks. A r t i f i c i a l Intelligence, 4 5 ( 3 ) : 2 7 5 - 2 8 6 , O c t o b e r 1990. 5 Conclusion [ M i e l l o u , 1975] Jean-Claude M i e l l o u . I t e r a t i o n s chao- In t h i s paper we proposed m o n o t o n i c asynchronous i t - tiques a retards; etudes de la convergence dans le e r a t i o n as a correct and efficient way of i m p l e m e n t i n g cas d'espaes p a r t i e l l e m e n t ordonnes ( C h a o t i c itera- parallel discrete r e l a x a t i o n for systems for w h i c h m o n o - tions w i t h delay; studies of convergence for the case t o n i c i t y is a necessary correctness c o n d i t i o n . Mono- of p a r t i a l l y ordered spaces). Comptes Rendus Heb- tonic asynchronous i t e r a t i o n uses an o p t i m i s t i c scheme dom ad aires des Seances De L 'Academic des Sciences, to c o m p u t e a possible next state of the system. T h i s 280, Series A ( 4 ) : 2 3 3 ~ 2 3 6 , J a n u a r y 1975. In French. o p t i m i s t i c scheme is h i g h l y efficient b u t is not necessar- i l y m o n o t o n i c (i.e., correct). A n application-specific test [Parker, 1987] D. S t o t t Parker. P a r t i a l order p r o g r a m - for m o n o t o n i c i t y is then applied to the c o m p u t e d state. m i n g . Technical R e p o r t CSD-870067, C o m p u t e r Sci- If the test succeeds, the state t r a n s i t i o n is made ( a t o m - ence D e p a r t m e n t , U n i v e r s i t y of C a l i f o r n i a , Los Ange- i c a l l y ) . O t h e r w i s e , the c o m p u t a t i o n is repeated using les, C a l i f o r n i a , December 1987. the current state. We have applied our technique to [ P o h l m a n n , 1991] Werner P o h l m a n n . A fixed p o i n t ap- the parallel i m p l e m e n t a t i o n of a c o n s t r a i n t satisfaction proach t o parallel discrete event s i m u l a t i o n . A c t a I n - system t h a t computes g l o b a l l y consistent solutions, for f o r m a t i c a , 2 8 ( 7 ) : 6 1 1 - 6 2 9 , October 1991. w h i c h m o n o t o n i c i t y is a necessary correctness c o n d i t i o n t h a t is not a u t o m a t i c a l l y satisfied. We believe m o n o t o n i c [Rosenfeld et a i , 1976] A z r i e l Rosenfeld, R o b e r t A. asynchronous i t e r a t i o n is applicable to parallel discrete H u m m e l , and Steven W. Zucker. Scene l a b e l i n g by re- r e l a x a t i o n in general. It w i l l be interesting to see if this l a x a t i o n o p e r a t i o n s . I E E E Transactions o n Systems, application-specific m o n o t o n i c i t y test is easy to derive M a n and Cybernetics, 6 ( 6 ) : 4 2 0 - 4 3 3 , June 1976. for other discrete r e l a x a t i o n p r o b l e m s . [ S c h m i d t , 1986] D a v i d A. S c h m i d t . D e n o t a t i o n a l Se- m a n t i c s : A Methodology f o r Language Development. 6 Acknowledgements A l l y n and B a c o n , B o s t o n , Massachusetts, 1986. We t h a n k Suresh K r i s h n a for t r a n s l a t i n g M i e l l o u ' s paper [Stoy, 1977] Joseph E. Stoy. D e n o t a t i o n a l Semantics: o n chaotic i t e r a t i o n s f r o m French i n t o E n g l i s h , and E d The Scott-Strachey Approach to P r o g r a m m i n g L a n - W a n g for e x p l a i n i n g the subtleties o f l a t t i c e theory. W e guage Theory. M I T Press, C a m b r i d g e , Massachusetts, also t h a n k C h u - C h e o w L i m , E d W a n g , L u i g i Semenzato 1977. and K a t h y Yelick for t h e i r c o m m e n t s on various drafts o f t h i s paper. [Tanaka a n d U z u h a r a , 1990] T o m o y u k i T a n a k a and Shigeru U z u h a r a . M u l t i p r o - References cessor C o m m o n Lisp o n T O P - 1 . I n Proceedings o f the Second I E E E Symposium o n P a r a l l e l and D i s t r i b u t e d [ B a u d e t , 1978] G e r a r d M. B a u d e t . Asynchronous iter- Processing, D a l l a s , Texas, December 1990. a t i v e m e t h o d s for multiprocessors. J o u r n a l of the A C M , 2 5 ( 2 ) : 2 2 6 - 2 4 4 , A p r i l 1978. [ W a l t z , 1972] D. L. W a l t z . G e n e r a t i n g semantic descrip- [Chazan and M i r a n k e r , 1969] D . Chazan and W . M i - t i o n s f r o m d r a w i n g s of scenes w i t h shadows. Techni- ranker. C h a o t i c r e l a x a t i o n . L i n e a r Algebra and its cal R e p o r t A I - T R - 2 7 1 , M I T L a b o r a t o r y for C o m p u t e r A p p l i c a t i o n s , 2 : 1 9 9 - 2 2 2 , 1969. Science, C a m b r i d g e , Massachusetts, 1972. [Franz I n c . , 1990] Franz Inc. Allegro C L i P M a n u a l , re- [Zorn et al., 1989] B e n j a m i n Z o r n , K i n s o n H o , James lease 3.0.3 e d i t i o n , M a r c h 1990. L a r u s , L u i g i Semenzato, and Paul Hilfinger. M u l t i - [Guesgen, 1989] Hans Werner Guesgen. C O N S A T : A processing extensions i n Spur Lisp. I E E E Software, System f o r C o n s t r a i n t S a t i s f a c t i o n . Research Notes 6 ( 4 ) : 4 1 - 4 9 , J u l y 1989. Ho, Hilfinger, and Guesgen 273