VIEWS: 0 PAGES: 99 POSTED ON: 5/7/2013
Chapter 4: Basic Graph Algorithms and Computational Complexity Massoud Pedram Dept. of EE University of Southern California Outline Analysis of Algorithms Graph Algorithms Dynamic Programming Mathematical Programming Stochastic Search Greedy Algorithms Divide and Conquer Floorplanning M. Pedram Analysis of Algorithms Big-O notation O(f(n)) means that as n , the execution time t is at most cf(n) for some constant c Small-o notation o(f(n)) means that as n , the execution time t is at least cf(n) for some constant c Theta notation (f(n)) means that as n , the execution time t is at most c1f(n) and at least c2f(n) for some constants c1 and c2 where n = input size M. Pedram Big-O Notation Express the execution time as a function of the input size n. Since only the growth rate matters, we can ignore the multiplicative constants and the lower order terms, e.g., 3n2+6n+2.7 is O(n2) n1.1 + 10000000000n is O(n1.1) n1.1 is O(n1.1) M. Pedram Effect of Multiplicative Constant 800 f(n)=n2 Run time 600 400 f(n)=10n 200 0 10 20 25 n M. Pedram Growth Rate of Some Functions Polynomial Functions O(log n), O(log2n), O(n0.5), O(n), O(n log n), O(n1.5), O(n log2n), O(n2), O(n3), O(n4) Exponential Functions O(nlog n), O(2n), O(3n), O(4n), O(nn), O(n!) M. Pedram Exponential Functions Exponential functions increase rapidly, e.g., 2n will double whenever n is increased by 1 n 2n 1ms x 2n 10 103 0.001 s 20 106 1s 30 109 16.7 mins 40 1012 11.6 days 50 1015 31.7 years 60 1018 31710 years M. Pedram NP-Complete The class NP-Complete is a set of problems that, we believe, has no polynomial time algorithms Therefore, they are hard problems If a problem is NP-complete, there is no hope to solve it efficiently M. Pedram Solution Type for NP-Complete Problems Exponential time algorithms Special case algorithms Approximation algorithms Heuristic algorithms M. Pedram Graph Algorithms Basic Graph Definitions A graph G(V,E) is a collection of vertices in V connected by a set of edges E Dual of a graph G is obtained by replacing each face of G with one vertex and connecting two such vertices by an edge if the two corresponding faces share a common edge in G A B A B D D C C Graph G Dual Graph G M. Pedram Types of Graph Commonly Used in VLSI CAD Interval graphs Permutation graphs Comparability graphs Vertical and horizontal constraint graphs Neighborhood graphs and rectangular dualization M. Pedram Interval Graphs A B C E D F G A B C E F G D V = Set of intervals E = {(vi,vj) | vi and vj intersect} M. Pedram Permutation Graphs A B C D E B E D A C A E B D C V = Set of lines E = {(vi,vj) | vi and vj intersect} M. Pedram Comparability Graphs Orientable Property: Each edge can be assigned a one-way direction in such a way that the resulting graph G(V,F) satisfies the following property: (a, b) F and (b, c) F (a, c) F An undirected graph which is transitively orientable is called a comparability graph ? Orientable graph Non-orientable graph M. Pedram Horizontal Constraint Graphs A wA B C wB wA D A B D C wC Weighted directed graph: V = Set of modules E = {(vi,vj) | vj on the right of vi} weight(vi, vj) = width of module vi M. Pedram Vertical Constraint Graphs A B C hB D hA A B D C Weighted directed graph: V = Set of modules E = {(vi,vj) | vj above vi} weight(vi, vj) = height of module vi M. Pedram Neighborhood Graphs A B C D A B D C V = Set of modules E = {(vi,vj) | vj and vi share an edge} M. Pedram Rectangular Dualization Rectangular dualization does the inverse operation of the neighborhood graph construction A B A C D B D C Each rectangle represents a vertex vi and vj share an edge exactly if e(vi,vj) exists Some graphs have no rectangular duals M. Pedram Basic Graph Algorithms Commonly Used in VLSI CAD Minimum Spanning Tree (MST) Problems Steiner Minimum Tree (SMT) Problems Rectilinear SMT Problems Partitioning Problems Network Flow Problems M. Pedram Minimum Spanning Tree (MST) Problem: Given a connected undirected weighted graph G(V,E), construct a minimum weighted tree T connecting all vertices in V Note: A graph is connected exactly if there is at least one path for any pair of vertices M. Pedram Minimum Spanning Tree (MST) Kruskal’s algorithm: O(m log n) Prim’s algorithm : O(m log n) Can be improved to O(n log n) A greedy algorithm Note: m = no. of edges; n = no. of vertices M. Pedram Kruskal’s Algorithm Sort the edges T= Consider the edges in ascending order of their weights until T contains n-1 edges: Add an edge e to T exactly if adding e to T does not create any cycle in T M. Pedram Kruskal’s Algorithm Example e e M. Pedram Kruskal’s Algorithm Example (Cont’d) e e M. Pedram Kruskal’s Algorithm Example (Cont’d) M. Pedram Prim’s Algorithm T= S = {v} for an arbitrary vertex v Repeat until S contains all the vertices: Add the lightest edge e(v1,v2) to T where v1S and v2(V-S). Add v2 to S M. Pedram Prim’s algorithm Example M. Pedram Prim’s algorithm Example (Cont’d) M. Pedram Steiner Minimum Tree (SMT) Problem: Given an undirected weighted graph G(V,E) and a demand set D V, find a minimum cost tree T which spans a set of vertices V’V such that DV’ Elements of V’-D, which have a degree larger than 2 in T, are called the Steiner points M. Pedram Steiner Minimum Tree (SMT) When D = V, SMT = MST When |D| = 2, SMT = Single Pair Shortest Path Problem The SMT problem is NP-Complete M. Pedram Rectilinear Steiner Tree (RST) A Steiner tree whose edges are constrained to be vertical or horizontal Steiner point Demand point M. Pedram Rectilinear Steiner Minimum Tree (RSMT) Similar to the SMT problem, except that the tree T is a minimum cost rectilinear Steiner tree This problem is NP-Complete but we can get an approximate solution to this problem by making use of the MST M. Pedram Approximate Solution to RSMT Construct a MST T Obtain a RST T’ from T by connecting the vertices in T using vertical and horizontal edges There can be many solutions or MST a better one M. Pedram Approximate Solution to RSMT It is proved that: WMST 1.5 WRSMT Let W be the weight of the Steiner tree obtained by this method, i.e., by “rectilinearizing” the MST What is the smallest b such that: W bWMST M. Pedram Other Graph Problems Minimum Clique Covering Problem Maximum Independent Set Problem Minimum Coloring Problem Maximum Clique Problem Network Flow Problem M. Pedram Minimum Clique Covering A clique of a graph G(V,E) is a set of vertices V’ V such that every pair of vertices in V’ are joined by an edge The clique cover number, k, of a graph G(V,E) is the minimum no. of cliques needed to cover all of the vertices of G The problem of finding the clique cover number is NP-complete for general graphs M. Pedram Maximum Independent Set (MIS) An independent set of a graph G(V,E) is a set of vertices V’ V such that no pair of vertices in V’ are joined by an edge The MIS problem is to find the independence (stability) number, a, of a graph, i.e., the size of the largest independent set in the graph This problem is NP-complete for general graphs What is the relationship between the clique cover number and the independence number? Answer: a (G) k (G) M. Pedram Graph Problems in Interval Graphs What is the meaning of a clique cover in an interval graph? What is the meaning of a MIS in an interval graph? The clique covering and MIS in an interval graph can be found in O(n log n) time where n is the no. of intervals. How? Answer for MIS: Sort the 2n points in ascending order Scan list from left to right until encounter a right endpoint Output the interval having this right endpoint as a member of MIS and delete all intervals containing this point Repeat until done M. Pedram Minimum Chromatic Number The chromatic number, c, of a graph G(V,E) is the minimum no. of colors needed to color the vertices such that every pair of vertices joined by an edge are colored differently The problem of finding the chromatic number is NP-complete for general graphs M. Pedram Maximum Clique The maximum clique problem is to find the clique number, w, of a graph, i.e., the size of the largest clique in the graph This problem is NP-complete for general graphs What is the relationship between the chromatic number and the clique number? Answer: w (G) c (G) M. Pedram Graph Problems in Interval Graphs (Cont’d) What is the meaning of the chromatic number and the clique number in an interval graph? The chromatic number and the clique number of an interval graph can be found in O(n log n) time. How? Answer for maximum clique {simple O(n2) algorithm} A = Sort intervals (I); cliq=0; max-cliq=0; for i=1 to 2n do if A[i]=Left then cliq++; if cliq > max-cliq then max-cliq=cliq; else cliq--; Return max-cliq; M. Pedram Perfect Graphs A graph G(V,E) is called perfect of all of its induced sub-graphs satisfy the following two properties: P1: a (GI ) k (GI ) P2: w (GI ) c (GI ) Interval graphs, permutation graphs and orientable graphs are examples of the perfect graphs; Cycle of odd length > 3 is not a perfect graph The four key graph problems can be solved in polynomial time for the class of perfect graphs M. Pedram Partitioning Problem Problem: Given an undirected graph G(V,E), partition V into two sets V1 and V2 of equal sizes such that the number of edges |E1| between V1 and V2 is minimized E1 is called the cut M. Pedram Partitioning Problem More general versions of this problem: Specify the sizes of V1 and V2 Partition V into k sets where k 2 All these problems are NP-complete M. Pedram Network Flow Problem Problem: Given a weighted directed graph G(V,E). One vertex sV is designated as the source and one vertex tV as the sink. Find the maximum flow from s to t such that the flow along each edge does not exceed the capacity (weight) of the edge M. Pedram Network Flow Problem For example: 3 a b 10 7 4 5 s 5 6 t 7 10 c d 2 What is the maximum flow in this network? (see next slide) Network flow problem can be solved in polynomial time M. Pedram Network Flow Problem The maximum flow is 9 3/3 a b 9/10 3/7 4/4 5 s 2/5 6 t 7 c 2/2 d 6/10 A well known theorem: Maximum Flow = Minimum Cut M. Pedram Network Flow Problem The minimum cut is 9 3 a b 10 7 4 5 s 5 6 t 7 10 c 2 d Notice that we only count the edges going from s to t M. Pedram Network Flow Problem Unfortunately, we cannot use this network flow method to solve the partitioning problem. Why? M. Pedram Dynamic Programming Dynamic Programming In dynamic programming, an array A[1..n] (can be of higher dimension) of data is defined and stored. The computation of A[k] is dependent on the values of A[j] where j < k, so we can compute their values one by one, i.e., A[1], then A[2], then A[3], M. Pedram Dynamic Programming For example: s t How many shortest multi-bend routes can there be from s to t? M. Pedram Dynamic Programming We define an array A[1..8, 1..6] where A[x,y] is the number of routes from s to (x,y): x s 1 1 1 1 1 1 1 1 2 3 A[1,k] = 1 1 3 A[j,1] = 1 y 1 A[j,k] = A[j-1,k] + A[j,k-1] 1 1 t How about if we only allow 1-bend? M. Pedram Dynamic Programming in Maze Routing Criteria: Follow Shortest Manhattan Distance Always route towards t with minimum distance s A[1,k] = 1 A[j,1] = 1 A[j,k] = A[j-1,k] + A[j,k-1] t M. Pedram Dynamic Programming in Maze Routing Allow detours, i.e., allow routing away from t (routes do not follow the shortest Manhattan distance detour s If at most 2 detours are allowed, what is the DP for this problem? A[1,k] =? A[j,1] = ? t A[j,k] =? detour M. Pedram Mathematical Programming In mathematical programming, the problem is expressed as an objective function and a set of constraints. The followings are common problems that are solvable in polynomial time: Linear Programming Quadratic Programming Geometric Programming M. Pedram Mathematical Programming For Example: Let A1, A2 and A3 be the area of the three modules 1 2 What should be their dimensions (w1, w2, and w3) in order to 3 minimize the size of the bounding rectangle? M. Pedram Mathematical Programming Objective: Minimize WH 1 2 Constraints: W w1 + w2 W w3 3 H A3/w3 + A1/w1 H A3/w3 + A2/w2 M. Pedram Linear Programming (LP) Both the objective and constraint functions are linear LP can be solved optimally Many solvers are available that can efficiently solve problems with a large number of variables http://www-fp.mcs.anl.gov/otc/Guide/SoftwareGuide/Categories/linearprog.html M. Pedram Integer Linear Programming (ILP) ILP is a variant of LP in which all of the variables are integer ILP is NP-complete LP that contains both integer and real variables is called mixed integer linear programming M. Pedram Stochastic Search Local Search Tabu Search Genetic Algorithm Simulated Annealing M. Pedram Local Search Local search is a simple searching algorithm that always move “downhill” in the solution space The notion of neighborhood N(f) of a feasible solution f is used. Given f, N(f) is a set of feasible solutions that are “close” to f based on some criteria M. Pedram Local Search Algorithm: Initialize a feasible solution f Do G {g | g N(f) and cost(g) < cost(f)} If (G ), assign f as one element in G while (G ) Return f M. Pedram Local Search It is called first improvement if we always assign f with the first element found with a lower cost in the do-while- loop It is called steepest descent if we always assign f with the element of the lowest cost in the do-while-loop The biggest problem of local search is getting stuck in a local minimum M. Pedram Tabu Search Tabu search allows “uphill” moves Given a neighborhood set N(f) of a feasible solution f, the principle of tabu search is to move to the cheapest element gN(f) even when cost(g) > cost(f) A Tabu list containing the k last visited solutions is maintained to avoid circular search pattern of length k M. Pedram Tabu Search Algorithm: Initialize a feasible solution f best f, Q an empty queue Do G {g | g N(f) and g Q} If (G ) Assign f as the cheapest element in G If |Q| < k, enqueue(Q, f); else dequeue(Q), enqueue(Q, f) If cost(best) > cost(f), best f while (G ) and not stop() Return best M. Pedram Genetic Algorithms (GA) Instead of keeping just one current solution, GA keeps track of a set P of feasible solutions, called population In each iteration, the current population Pi is replaced by the next one Pi+1 To generate a feasible solution hPi+1, two solutions f and g are selected from Pi and h is generated such that it inherits properties from both parents f and g . This is called crossover M. Pedram Genetic Algorithms (GA) A lower cost solution in a population will have a higher chance of being selected as the parent Mutation may occur to avoid getting stuck in a local minimum M. Pedram Genetic Algorithms (GA) Algorithm: Initialize P with k feasible solutions Do newP For (i = 1; i k; i++} Select two low cost solutions from P as f and g h crossover(f, g) Add h to newP P newP while not stop() Return the best solution in P M. Pedram Markov Chain Let G(V,E) be a directed graph with V={S1,S2,…..Sn} Let C: VZ be a vertex-weighting function We write Ci for C(Si) Let P: E[0,1] be an edge-weighting function (notation: Pij=P(Si,Sj) such that: S j N ( Si ) Pij 1 for all SiV N(Si) is the set of next next states of Si We call (G,P) (finite) time-homogenous Markov chain Pij is called the conditional probability of edge(Si,Sj) Assume Pij>0, if Pij=0, simply delete edge(Si,Sj) from E M. Pedram Irredundant Markov Chain Irredundant Markov Chain A Markov chain (G,P)is irreducible if G is strongly connected P=[pij] is the matrix whose <i,j> entry is pij Stationary distribution i is the probability of being in state i A state distribution of a Markov Chain (G,P) is stationary if P Chapman Kolmogorov equations i 1 M. Pedram Markov Chain Example 1/2 1 2/3 1/3 2 1 1/2 3 0 2/ 3 1/ 3 1 2 3 0 1/ 2 1/ 2 1 2 3 1 0 0 1 2 3 1 1 3/10 2 4 /10 3 3/10 M. Pedram Simulated Annealing (SA) Simulated annealing is a powerful technique to provide high quality solutions to some difficult combinatorial problems It keeps a variable Temperature (T) which determines the behavior of the annealing process. This variable T is initialized to a very large value at the beginning, and will be gradually decreased (cooled down). M. Pedram Simulated Annealing Algorithm: Initialize T and a feasible solution f While (T a threshold) • Make a slight modification to f to get g • Check if g is better than f, i.e., cost(g) cost(f)? • If yes, accept g, i.e., f g; else, compute p as e-k(cost(g)-cost(f))/T where k is a positive constant, and then, accept g with probability p • Update T M. Pedram Basic Ingredients of Simulated Annealing Solution Space Neighboring Structure Cost Function Annealing Schedule Moves are selected randomly, and the probability that a move is accepted is proportional to system’s current temp M. Pedram Simulated Annealing 500 modules This is a floorplanning result obtained by simulated annealing M. Pedram Floorplanning Hierarchical Design Several blocks after partitioning: Need to: Put the blocks together. Design each block. Which step to go first? M. Pedram Hierarchical Design How to put the blocks together without knowing their shapes and the positions of the I/O pins? If we design the blocks first, those blocks may not be able to form a tight packing M. Pedram Floorplanning The floorplanning problem is to plan the positions and shapes of the modules at the beginning of the design cycle to optimize the circuit performance: chip area total wire length delay of critical path routability others, e.g., noise, heat dissipation, etc. M. Pedram Floorplanning v.s. Placement Both determines block positions to optimize the circuit performance. Floorplanning: Details like shapes of blocks, I/O pin positions, etc. are not yet fixed (blocks with flexible shape are called soft blocks) Placement: Details like module shapes and I/O pin positions are fixed (blocks with no flexibility in shape are called hard blocks) M. Pedram Floorplanning Problem Input: n Blocks with areas A1, ... , An Bounds ri and si on the aspect ratio of block Bi Output: Coordinates (xi, yi), width wi and height hi for each block such that hi wi = Ai and ri hi/wi si Objective: To optimize the circuit performance M. Pedram Bounds on Aspect Ratios If there is no bound on the aspect ratios, we can surely pack very tightly: However we don’t want to layout blocks as long strips, so we require ri hi/wi si for each i M. Pedram Bounds on Aspect Ratios We can also allow several shapes for each block: For hard blocks, the orientations can be changed: M. Pedram Objective Function A commonly used objective function is a weighted sum of area and wirelength: cost = aA + bL where A is the total area of the packing, L is the total wire length, and a and b are constants M. Pedram Wire length Estimation Exact wire length of each net is not known until routing is done In floorplanning, even pin positions are not known yet Some possible wire length estimations: Center-to-center estimation Half-perimeter estimation M. Pedram Dead space Dead space is the space that is wasted: Dead space Minimizing area is the same as minimizing dead space Dead space percentage is computed as (A - iAi) / A 100% M. Pedram Mixed Integer Linear Program A mathematical program such that: The objective is a linear function All constraints are linear functions Some variables are real numbers and some are integers, i.e., “mixed integer” It is almost like a linear program, except that some variables are integers M. Pedram Problem Formulation Minimize the packing area: W Assume that one dimension W is fixed Minimize the other dimension Y Need to have constraints Y so that blocks do not overlap Associate each block Bi with 4 variables: xi and yi: coordinates of its lower left corner wi and hi: width and height M. Pedram Non-overlapping Constraints For two non-overlapping blocks Bi and Bj, at least one of the following four linear constraints must be satisfied: hi Bi hj Bj (xi, yi) wi (xj, yj) wj M. Pedram Integer Variables Use integer (0 or 1) variables xij and yij: xij=0 and yij =0 if (1) is true xij=0 and yij =1 if (2) is true xij=1 and yij =0 if (3) is true xij=1 and yij =1 if (4) is true Let W and H be upper bounds on the total width and height. Non-overlapping constraints: (1' ) xi wi x j W ( xij yij ) (2' ) xi w j x j W (1 xij yij ) (3' ) yi hi y j H (1 xij yij ) (4' ) yi h j y j H (2 xij yij ) M. Pedram Formulation M. Pedram Formulation with Hard Blocks If the blocks can be rotated, use a 0-1 integer variable zi for each block Bi s.t. zi = 0 if Bi is in the original orientation and zi = 1 if Bi is rotated 90o M. Pedram Formulation with Soft Blocks If Bi is a soft block, wihi=Ai. But this constraint is quadratic! Linearized by taking the first two terms of the Taylor expression of hi=Ai/wi at wimax (max. width of block Bi) hi =himin+li(wimax-wi) where himin =Ai/wimax and li=Ai/wimax2 M. Pedram Formulation with Soft Blocks If Bi is soft and Bj is hard: If both Bi and Bj are soft: M. Pedram Solving Linear Program Linear Programming (LP) can be solved by classical optimization techniques in polynomial time. Mixed Integer LP (MILP) is NP-Complete. The run time of the best known algorithm is exponential to the no. of variables and equations M. Pedram Complexity For a problem with n blocks, and for the simplest case, i.e., all blocks are hard: 4n continuous variables (xi, yi, wi, hi) n(n-1) integer variables (xij, yij) 2n2 linear constraints Practically, this method can only solve small size problems. M. Pedram Successive Augmentation A classical greedy approach to keep the problem size small: repeatedly pick a small subset of blocks to formulate a MILP, solve it together with the previously picked blocks with fixed locations and shapes: Next subset Y Partial Solution M. Pedram