VIEWS: 5 PAGES: 50 POSTED ON: 11/8/2009 Public Domain
The satisfiability threshold and clusters of solutions in the 3-SAT problem Elitza Maneva IBM Almaden Research Center 3-SAT Variables: x1, x2, …, xn take values {TRUE, FALSE} Constraints: (x1 or x2 or_not x3) , (not x2 or x_ or not x6), … 4 _ (x1 x2 x3) ( x2 x4 x6) … 1 x1 x2 0 x4 x5 x8 0 x3 x6 x7 1 0 Random 3-SAT x1 x2 x3 x4 x5 x6 x7 x8 n m = n Red = proved, green = unproved Random walk Survey propagation Satisfiable Belief propagation Satisfiable Myopic PLR 0 1.63 Not satisfiable Not satisfiable 3.52 3.95 4.27 4.51 Rigorous bounds for random 3-SAT 2002 2002 Kaporis Hajiaghayi Kirousis Sorkin Lalas 3.52 1999: [Friedgut] there is a sharp threshold of satisfiability c(n) Rigorous bounds for random 3-SAT 0 1.63 3.52 4.51 5.19 Pure Literal Rule Algorithm: If any variable appears only positive or only negative assign it 1 or 0 respectively Simplify the formula by removing the satisfied clauses Repeat _ _ _ _ _ (x1 x2 x3) ( x2 x4 x5) (x1 x2 x4) (x3 x4 x5) 1 1 0 1 Rigorous bounds for random 3-SAT 0 1.63 3.52 4.51 5.19 Myopic Algorithms: Choose a variable according to # positive and negative occurrences Assign the variable the more popular value Simplify the formula by 1. removing the satisfied clauses 2. removing the FALSE literals 3. assigning variables in unit clauses 4. assigning pure variables Repeat Best rule: maximum |# positive occurr. – # negative occurr.| Rigorous bounds for random 3-SAT 0 1.63 3.52 4.51 5.19 E [# solutions] = 2n Pr [00…0 is a solution] = = 2n (1-1/8)m = = (2 (7/8))n For >5.191, E [# solutions] 0, so Pr [satisfiable] 0 Rigorous bounds for random 3-SAT 0 1.63 3.52 4.51 4.67 5.19 E [# positively prime solutions] 0 Positively prime solution: a solution in which no variable assigned 1 can be converted to 0. Fact: If there exists a solution, there exists a positively prime solution. Rigorous bounds for random 3-SAT 0 1.63 3.52 4.51 4.67 5.19 E [# symmetrically prime solutions] 0 Random 3-SAT x1 x2 x3 x4 x5 x6 x7 x8 n m = n Red = proved, green = unproved Random walk Survey propagation Belief propagation Satisfiable Myopic PLR 0 1.63 3.52 3.95 4.27 4.51 Random Walk Algorithms [Alekhnovich, Ben-Sasson `03] [Seitz, Alava, Orponen `05] [Ardelius, Aurell `06] ASAT: Pick an unsatisfied clause Pick a variable in the clause Flip it only with prob. p if number of unsatisfied clauses does not increase Experiment: Takes O(n) steps for < 4.21. Simple Random Walk: Pick an unsatisfied clause Pick a variable in the clause Flip the variable Theorem: Finds a solution in O(n) steps for < 1.63. Random 3-SAT x1 x2 x3 x4 x5 x6 x7 x8 n m = n Red = proved, green = unproved Random walk Survey propagation Belief propagation Satisfiable Myopic PLR 0 1.63 3.52 3.95 4.27 4.51 We can find solutions via inference Suppose the formula is satisfiable. Consider the uniform distribution over satisfying assignments: Pr[x1, x2, …, xn] (x1, x2, …, xn) Simple Claim: If we can compute Pr[xi=1], then we can find a solution fast. Decimation: Assign variables one by one to a value that has highest probability. Fact: We cannot hope to compute Pr[xi=1] exactly Heuristics for guessing the best variable to assign: 1. Pure Literal Rule (PLR): Choose a variable that appears always positive / always negative. 2. Myopic Rule: Choose a variable based on number of positive and negative occurrences. 3. Belief Propagation: Estimate Pr[xi=1] by belief propagation and choose variable with largest estimated bias. Computing Pr[x1=0] on a tree formula 108 192 36 48 1 1 x1 3 4 12 12 1 1 1 1 3 4 4 3 #Solns with 0 #Solns with 1 1 1 1 1 1 1 1 1 #Solutions with 0 #Solutions with 1 Vectors can be normalized .36 .64 .43 .57 .5 .5 .43 .57 x1 .43 .57 .5 .5 .5 .5 .5 .5 .57 .43 .5 .5 .5 .5 .5 .5 .5 .5 Vectors can be normalized … and thought of as messages x1 What if the graph is not a tree? Belief propagation Belief propagation x3 x2 (x1, x2 , x3) x1 x11 Pr[x1, …, xn] Πa a(xN(a)) x4 x5 x6 x10 x9 x8 x7 Belief Propagation [Pearl ‘88] x1 x2 x3 x4 x5 x6 x7 n m Given: Pr[x1 …x7] a(x1, x3) b(x1, x2) c(x1, x4) … i.e. Markov Random Field (MRF) Goal: Compute Pr[x1] (i.e. marginal) Belief programming algorithm. bN(i)/c It is exact only when the recurrence relation holds, i.e.: M c i (x ) = Σ (x ) Π M j c (xj) : tree. j N(c)\i 1. if the graph iis xaj N(c)\i c N(c) 2. if the Estimated marginals:like a tree: large cycles graph behaves j Message passing rules: M i c (xi) = Π M b i (xi) propagation is a dynamic i(xi) = Π cN(i) M c i (xi) Applications of belief propagation • • • • • • Statistical learning theory Vision Error-correcting codes (Turbo, LDPC, LT) Lossy data-compression Computational biology Sensor networks Limitations of BP x1 x2 x3 x4 x5 x6 x7 x8 n m = n Random walks Survey propagation Belief propagation Satisfiable Myopic PLR 0 1.63 3.52 3.95 4.27 4.51 Reason for failure of Belief Propagation • Messages from different neighbors are assumed to be almost independent i.e. there are no long-range correlations Random walks No long-range propagationLong-range Survey correlations correlations Belief propagation exist Myopic PLR 0 1.63 3.52 3.95 4.27 4.51 Reason for failure of Belief Propagation • Messages from different neighbors are assumed to be almost independent i.e. there are no long-range correlations Fix: 1-step Replica Symmetry Breaking Ansatz • The distribution can be decomposed into ―phases‖ • There are no long-range correlations within a phase • Each phase consists of similar assignments – ―clusters‖ • Messages become distributions of distributions • An approximation yields 3-dimensional messages: Survey Propagation [Mezard, Parisi, Zecchina ‗02] • Survey propagation finds a phase, then WalkSAT is used to find a solution in the phase Reason for failure of Belief Propagation • Messages from different neighbors are assumed to be almost independent i.e. there are no long-range correlations Fix: 1-step Replica Symmetry Breaking Ansatz • The distribution can be decomposed into ―phases‖ Pr[x1, x2, …, xn] = p Pr [x1, x2, …, xn] fixed variables Space of solutions Satisfying assignments in {0, 1}n 0110 phases 1011 01101 Survey propagation .12 .81 .07 0 1 Survey propagation x1 x2 x3 x4 x5 x6 x7 I‘m 0 with prob 10%, 1 with prob 70%, whichever (i.e. ) 20% x8 You have to satisfy Mu me jc Mci= with prob. 60% ———————— u s jN(c)\i M j c+M j c+M jc Muic = (1Msic = (1Mic = b Nua (i) (1- Mbi )) b Nsa (i) (1-Mbi) (1-Mbi) b Nsc (i) (1- Mbi )) b Nuc (i) b N(i)\c (1- Mbi ) Combinatorial interpretation • Can survey propagation be thought of as inference on cluster assignments? Not precisely, but close. • We define a related concept of core/cover assignments • Assignments in the same cluster share the same core • However, different cluster may have the same core Finding the core of a solution 0 1 1 0 0 0 0 1 Finding the core of a solution 0 1 1 0 0 0 0 1 Finding the core of a solution 0 1 1 0 0 0 0 0 unconstrained variables Finding the core of a solution 0 1 1 0 0 0 0 Finding the core of a solution 0 1 1 0 0 0 Finding the core of a solution 0 1 0 0 0 Finding the core of a solution 1 0 0 0 Such a fully constrained partial assignment is called a cover. Extending the space of assignments {0, 1}n assignments 0110 Partial assignments {0,1,}n core # stars core 1011 01101 Survey propagation is a belief propagation algorithm [Maneva, Mossel, Wainwright ‘05] [Braunstein, Zecchina ‘05] Theorem: Survey propagation is equivalent to belief propagation on the uniform distribution over cover assignments. But, we still need to look at all partial assignments. Peeling Experiment for 3-SAT, n =105 100000 90000 80000 2 2.5 3 3.5 4 4.1 4.2 # unconstrained 70000 60000 50000 40000 30000 20000 10000 0 0 20000 40000 60000 80000 100000 # stars Clusters and partial assignments {0, 1}n assignments 0110 Partial assignments 01101 1011 01101 # stars Definition of the new distribution Partial assignments Formula 0111 1111 1010 1 n() 011 011 11 111 111 11 010 100 3 2 no() 11 4 1 1 1 1 1. Includes all assignments without contradictions or implications 2. Weight of partial assignments: Pr[] n() (1- ) no() Vanilla BP 3. A family of belief propagation algorithms: 0 1 SP Pr[] n() (1- ) no() {0, 1}n assignments 0110 Partial assignments =0 =1 core # stars 1011 core Vanilla BP 0 01101 SP 1 This is the correct picture for 9-SAT and above. [Achlioptas, Ricci-Tersenghi ‗06] Clustering for k-SAT What is known? 2-SAT: a single cluster 3-SAT to 7-SAT: not known 8-SAT and above: exponential number of clusters (with second moment method) [Mezard, Mora, Zecchina `05] [Achlioptas, Ricci-Tersenghi `06] 9-SAT and above: clusters have non-trivial cores (with differential equations method) [Achlioptas, Ricci-Tersenghi `06] Convex geometry / Antimatroid 0111 1111 1010 011 011 111 111 010 100 11 11 11 1 1 1 1 Total weight is 1 for every Rigorous bounds for random 3-SAT 0 3.52 4.51 4.9 5.19 E [total weight of partial assignments] 0 ( = 0.8) Fact: If there exists a solution, the weight of partial assignments is at least 1. Rigorous bounds for random 3-SAT 0 3.52 4.453 4.51 5.19 Theorem [ Maneva, Sinclair ] For > 4.453 one of the following holds: 1. there are no satisfying assignments with high probability; 2. the core of every satisfying assignment is (,,…,). Random 3-SAT x1 x2 x3 x4 x5 x6 x7 x8 n m = n Red = proved, green = unproved Random walk Survey propagation Belief propagation Satisfiable Myopic PLR 0 1.63 3.52 3.95 4.27 4.51 Challenges • Improve the bounds on the threshold • Prove algorithms work with high probability • Find an algorithm for certifying that a formula with n clauses for large has no solution Thank you