VIEWS: 3 PAGES: 21 POSTED ON: 9/26/2011 Public Domain
Solution of Satisfiability Problems by Unconstrained Quadratic Programming Talk given by Gary Kochenberger University of Colorado & Hearin Center for Enterprise Science Other Team Members Fred Glover Bahram Alidaee Keith Womer Haitao Li Introduction: sThe Tenet of Linearity in Combinatorial Optimization sMetaheuristics Methods call this devotion into question sRecent work suggests that nonlinear representations may offer a fruitful alternative sIn this research we investigate the use of such methods for satisfiability problems (SAT) Definition of SAT: A SAT instance is a Boolean CNF Formula with: s A set of n variables: x1 , x2 ,..., xn s A set of literals: either a variable or its negation s A set of Clauses: C1 , C 2 ,..., C m . Each Clause consists only of literals connected by logical “or” connectives s CNF: C1 C 2 ... C m . Where is the logical “and” connective. Goal: Find an assignment of Truth values such that the Boolean function evaluates to True. For Max SAT, we want an assignment that satisfies the maximum number of clauses. Applications: s Propositional Logic s Data Base Design and Analysis s Automated Reasoning s SAILOR 21 s Computer Aided Design s Machine vision s Robotics s Scheduling s Integrated Circuit Design Zero/One (IP) formulation of SAT and Max-SAT: max y0 m1 y j j xi xi y j j 1, m subject to iPj iN j s Pj and N j : literals in clause m s x j 0/1 (logical) variables s y j : whether or not clause m is satisfied s y j 1 i SAT Max 2 Sat Example: (Hansen & Jaumard) (n = 4, m = 12) Clause # Clause 1 x1 x2 2 x1 x2 3 x1 x2 4 x1 x2 5 x1 x3 6 x1 x3 7 x2 x3 8 x2 x4 9 x2 x3 10 x2 x3 11 x3 x4 12 x3 x4 IP Model: max y1 y2 ... y12 st x1 x2 y1 0 x1 x2 y2 1 x1 x2 y3 1 x1 x2 y4 2 x1 x3 y5 1 x1 x3 y6 2 x2 x3 y7 1 x2 x4 y8 0 x2 x3 y9 1 x2 x3 y10 2 x3 x4 y11 0 x3 x4 y12 2 s 16 variables and 12 constraints s y1 x1 x2 x3 0 , all other variables equal to 1 s 11 of the 12 clauses are satisfied s (by setting x4 equal to 1 and x1 , x2 , x3 equal to 0) Max 2-SAT & Unconstrained Quadratic Binary Programming: s Quadratic Penalty Function s Unconstrained Quadratic Binary Program s 2-SAT w Three possible clause types w Each with linear constraint w Each with equivalent exact penalty Three possibilities are: (a) No negations: Classical constraint: xi x j 1 Exact Penalty: (1 xi x j xi x j ) (b) One negation: Classical constraint: xi x j 1 Exact Penalty: ( x j xi x j ) (c) Two negations: Classical constraint: xi x j 1 Exact penalty: ( xi x j ) For our example: Clause # Clause Quadratic Penalty (penj) 1 x1 x2 (1 x1 x2 x1 x2 ) 2 x1 x2 ( x2 x1 x2 ) 3 x1 x2 ( x1 x1 x2 ) 4 x1 x2 ( x1 x2 ) 5 x1 x3 ( x1 x1 x3 ) 6 x1 x3 ( x1 x3 ) 7 x2 x3 ( x3 x2 x3 ) 8 x2 x4 (1 x2 x4 x2 x4 ) 9 x2 x3 ( x2 x2 x3 ) 10 x2 x3 ( x2 x3 ) 11 x3 x4 (1 x3 x4 x3 x4 ) 12 x3 x4 ( x3 x4 ) Form Penalty Function: m SAT (UQP): min x0 pen j j 1 s penj is the quadratic penalty associated with clause j s In the form of xQx (i.e., UQP) s Advances is solving UQP For our example we get: min x0 3 x1 2 x4 x2 x3 x2 x4 2 x3 x4 or, min x0 3 ( xQx) / 2 where the matrix Q is given by 2 0 0 0 0 0 1 1 0 1 0 2 0 1 2 4 Solving SAT (UQP) gives: x0 1 at x1 x2 x3 0, x4 1 Remarks: s This unconstrained quadratic model in 4 variables is equivalent to the 16 variable, 12 constraint IP s The general SAT (UQP) model will be of the form min x0 c xQx , where c is a constant and x0 0 s Model applies both to the SAT and Max-SAT w If x0 equals zero, we have a SAT solution w Else, our solution is a result for the Max-SAT problem s Eliminates the need for the “y” variables w Problem with m = 5,000 clauses and 200 variables w SAT (UQP) model with a Q matrix of size 200 by 200 (very modest) w IP model would have 5,200 variables and 5000 constraints Random Problems xQx results CPLEX results (IP) ID n # # time # # # Time clauses Viol Viol iters nodes 2Sat 100a 100 626 53 1 sec 53 601598 21226 147 sec 100b 100 795 98 1 sec 99 44 mil 1.7 mil 10 hrs 100c 100 953 129 1 sec 130 78 mil 2.7 mil 10 hrs 2Sat 200a 200 1891 232 3 sec 251 72 mil 3.2 mil 10 hrs 200b 200 2067 284 3 sec 315 70 mil 2.7 mil 10 hrs 200c 200 1699 219 3 sec 233 74 mil 3.5 mil 10 hrs 2Sat 300a 300 5585 905 6 sec 1054 14 mil 1.2 mil 10 hrs 300b 300 5590 915 6 sec 992 28 mil .9 mil 10 hrs 300c 300 4791 717 6 sec 810 13 mil 1.3 mil 10 hrs 2Sat 400a 400 5762 888 10 sec 1043 10 mil 1.0 mil 10 hrs 400b 400 6506 1012 10 sec 1248 9 mil 1.1 mil 10 hrs 400c 400 6507 991 10 sec 1106 23 mil 1.1 mil 10 hrs 2Sat 500a 500 5370 742 17 sec 964 10 mil 1.2 mil 10 hrs 500b 500 5841 806 17 sec 1023 10 mil 1.0 mil 10 hrs 500c 500 6304 907 17 sec 1104 11 mil 1.1 mil 10 hrs 500d 500 22883 3566 17 sec (NA) 2Sat 1000a 1000 10878 1505 61 sec (NA) • Random Problems….available from authors • xQx times rounded up • xQx code run for 50 SPAN cycles • 2Sat 500d and 1000a too large for CPLEX • CPLEX results from version 8.0 The Problems from Literature (Brian Borchers) Best known xQx xQx Maxsat3 Maxsat n m Solution solution time solution time 50 100 4 4 <1 4 .4 50 150 8 8 <1 8 1.5 50 200 16 16 <1 16 116.2 50 250 22 22 <1 22 652.4 50 300 32 32 <1 32 8,763 50 350 41 41 <1 NA > 12 hr 50 400 45 45 <1 NA > 12 hr 50 450 63 63 <1 NA > 12 hr 50 500 66 66 <1 NA > 12 hr 100 200 5 5 <2 5 3.2 100 300 15 15 <2 15 13,770 100 400 29 29 <2 NA > 12 hr 100 500 44 44 <2 NA > 12 hr 100 600 ? 65 <2 NA > 12 hr 150 300 4 4 <3 4 4.1 150 450 22 22 <3 NA > 12 hr 150 600 38 38 <3 NA > 12 hr 1. All times in seconds unless noted otherwise. 2. Maxsat is an exact method developed by Borchers & Furman 3. Maxsat results obtained on IBM RS/6000-590 4. xQx results obtained 1.6 MHZ PC. 5. Each xQx run was for 50 SPAN cycles 6. Problem 100_600 was previously unsolved. 3-SAT & Max 3-SAT: s Four clause types s Each with classical constraint & penalty 1. No Negations: Classical constraint: xi x j xk 1 Exact Penalty: (1 xi x j xk xi x j xk xi x j xi xk x j xk ) 2. One Negations: Classical constraint: xi x j xk 1 Exact Penalty: ( xk xi xk x j xk xi x j xk ) 3. Two Negations: Classical constraint: xi x j xk 1 Exact Penalty: ( x x x x x ) j k i j k 4. Three Negations: Classical constraint: xi x j xk 1 Exact Penalty: ( xi x j xk ) s As before, minimize the sum of the penalty terms s Penalty function is cubic s Convert to xQx via procedure of Hammer and Boros: w Replace the product term xy by a new variable z w Add the penalty term P( xy 2xz 2 yz 3z) g penalty is zero only when z xy g else it is positive 3-Sat Example: (n = 5, m = 12) Clause # Clause 1 x1 x2 x3 2 x1 x2 x3 3 x1 x2 x3 4 x2 x3 x4 5 x2 x3 x4 6 x2 x3 x4 7 x2 x4 x5 8 x2 x3 x5 9 x2 x3 x5 10 x3 x4 x5 11 x3 x4 x5 12 x3 x4 x5 Forming the penalty function: 12 x0 pen j 3 x1 x2 2 x4 2 x5 2 x1 x3 4 x2 x3 3x4 x5 j 1 x1 x2 x3 3x2 x3 x4 x2 x4 x5 x3 x4 x5 2 x2 x3 x5 s five cubic terms s introduce two new variables w x x2 x3 w 6 x7 x4 x5 s with associated penalties 12 x0 pen j 3 x1 x2 2 x4 2 x5 2 x1 x3 4 x2 x3 3 x4 x5 j 1 x1 x6 3 x4 x6 x2 x7 x3 x7 2 x5 x6 P ( x 2 x3 2 x 2 x 6 2 x3 x 6 3 x 6 ) P ( x 4 x5 2 x 4 x 7 2 x5 x 7 3 x 7 ) Which is of the form: x0 3 ( xQx ) / 2 Taking (arbitrarily) the penalty P to be 10, the 7-by-7 Q matrix is: 2 0 2 0 0 1 0 0 2 6 0 0 20 1 2 6 0 0 0 20 1 Q 0 0 0 4 13 3 20 0 0 0 13 4 2 20 1 20 20 3 2 60 0 0 1 1 20 20 0 60 Minimizing xQx yields x1 x2 x3 x6 1, x4 x5 x7 0 for which xQx 6 . Thus, x0 0 implying that all 12 clauses are satisfied at this solution. Remarks pertaining to 3-SAT and Max 3-Sat: s For 3-SAT, # clauses does matter s SAT (UQP) could be as large as (n + m) s But terms combine (and cancel) s And, our ability to solve larger UQP continues to advance s General SAT problems Solving xQx (An Overview): s Tabu Search (TS) method is centered around the use of strategic oscillation. s The method alternates between constructive phases (“add moves”) and destructive phases (“drop moves”). s To control the underlying search process, we use a memory structure that is updated at critical events. s A parameter span is used to indicate the amplitude of oscillation about a critical event. s We begin with span equal to 1 and gradually increase it to some limiting value. s For each value of span, a series of alternating constructive and destructive phases is executed before progressing to the next span value. s At the limiting point, span is gradually decreased, allowing again for a series of alternating constructive and destructive phases. s When span reaches a value of 1, a complete span cycle has been completed and the next cycle is launched. Solving xQx (An Overview), continued: s The search process is typically allowed to run for a pre-set number of span cycles. s Information stored at critical events is used to influence the search process by penalizing potentially attractive add moves and inducing drop moves associated with assignments of values to variables in recent critical solutions. s Cumulative critical event information is used to introduce a subtle long-term bias into the search process by means of additional penalties and inducements similar to those discussed above. s Other standard elements of tabu search such as short and long- term memory structures are also included. s A complete description of the method is given in Glover, Kochenberger, Alidaee (Management Science). Where we are headed: s Further testing of max 2-SAT case w More test problems w State of the Art comparisons s Extend to 3- SAT case w Code and test generator w Converting “cubic” penalty function to xQx w Test on standard test bed w Assess performance/make comparisons s Extend to General SAT case w Convert back to xQx w Examine other solution possibilities w Long term plans s Develop improved heuristic method for solving xQx w Enable larger instances to be solved w Useful for other problem classes Wrap-up: s Shown xQx to be a fruitful modeling and solution framework for 2-SAT and Max 2-SAT problems. s Indicated how xQx could be applied to more general SAT problems. s Encouraged by our progress to date and with the prospects for future findings and contributions.