VIEWS: 7 PAGES: 51 POSTED ON: 7/11/2011 Public Domain
Analysis of Algorithms CS 4413 Briana B. Morrison With thanks to Dr. Hung Algorithmics: the study of algorithms. Algorithmics: is the core of computer science. Algorithmics: the spirit of computing. How important is this course?! 2 The Course Purpose: a rigorous introduction to the design and analysis of algorithms – Not a lab or programming course – Not a math course, either But, need know how to program! Minimum math knowledge! 3 Course Objectives Basic design concepts and analysis methods of algorithms Design methodologies include, but not limited to: – Searching/Sorting – Divide-and-Conquer – Greedy Method – Dynamic Programming – Search and Traversal – Branch-and-Bound 4 Programming/project grading Elegant solution Logical thinking Quality of writing and coding Concise writing is a virtue. Tell the reader only what is important. 5 Research Project Guidelines Research projects should consist of the following components: – Define the problem to be solved – Background information – Algorithms and Applications – Implementation – Drawbacks – New applications and discoveries 6 Research Project Guidelines Some examples of research projects: – Dynamic Programming – SNAKE – Marko Chain Monte Carlo (MCMC) (and Image analysis) – Wavelet Transforms – Others 7 From Problems to Programs There are many steps involved in writing a computer program to solve a given problem. The steps go from – 1) problem formulation and specification, – 2) to design of the solution, – 3) to implementation, testing and documentation, and finally – 4) to evaluation of the solution. 8 From Problems to Programs … Algorithms and Data Structures are the building blocks of most computer programs. Half the battle is knowing what problem to solve. Almost any branch of mathematics or science can be called into service to help model some problem domain. – Examples: problems essentially numerical in nature. 9 Algorithms An algorithm is a finite sequence of instructions, each of which has a clear meaning and can be performed with a finite amount of effort in a finite length of time. Pseudo-language is used to present algorithms. Pseudo-language is a combination of the constructs of a programming language together with informal English statements. 10 Algorithms … Example 1: A mathematical model can be used to help design a traffic light for a complicated intersection of roads (figure 1.1). Figure 1.1. An intersection. 11 Figure 1.2. Graph showing incompatible turns. 12 Algorithms … Figure 1.3. Table of incompatible turns. 13 Graph Coloring Problem Figure 1.4. A graph. 14 Heuristic An algorithm that quickly produces good but not necessarily optimal solutions is called a heuristic. One reasonable heuristics for graph coloring is the so-called “greedy” algorithm. 15 Greedy Algorithm Initially we try to color as many vertices as possible with the first color, then as many as possible of the uncolored vertices with the second color, and so on. To color vertices with a new color, we perform the folowing steps: 1) Select some uncolored vertex and color it with the new color. 2) Scan the list of uncolored vertices. For each uncolored vertex, determine whether it has an edge to any vertex already colored with the new color. If there is no such edge, color the present vertex with the new color. 16 Graph Coloring Problem … Figure 1.5. A coloring of the graph of Fig. 1.2. 17 Pseudo-Language and Stepwise Refinement Once we have an appropriate mathematical model for a problem, we can formulate an algorithm in terms of that model. The initial versions of the algorithm are often couched in general statements that will have to be refined subsequently into smaller, more definite instructions. 18 First refinement of greedy algorithm procedure greedy ( var G: GRAPH; Var newclr: SET ); { greedy assigns to newclr a set of vertices of G that may be given the same color} . begin newclr := 0; for each uncolored vertex v of G do if v is not adjacent to any vertex in newclr then begin mark v colored; add v to newclr end end; {greedy} 19 Refinement of greedy algorithm procedure greedy ( var G: GRAPH; var newclr: SET ); begin newclr := 0; for each uncolored vertex v of G do begin found := false; for each vertex w in newclr do if there is an edge between v and w in G then found := true; if found = false then begin { v is adjacent to no vertex in newclr } mark v colored; add v to newclr end end 20 end; {greedy} Refined greedy algorithm procedure greedy ( var G: GRAPH; var newclr: LIST); { greedy assigns to newclr those vertices that may be given the same color} var found: boolean; v, w: integer; begin newclr := 0; v : = first uncolored vertex in G; while v < > null do begin found: = false; w : = first vertex in newclr; while w < > null do begin if there is an edge between v and w in G then found := true; w : = next vertex in newclr end; if found = false do begin mark v colored; add v to newclr end; v : = next uncolored vertex in G end 21 end; {greedy} Abstract Data Types Procedures generalize the notion of an operator. Another advantage of procedures is that they can be used to encapsulate parts of an algorithm by localizing in one section of a program all the statements relevant to a certain aspect of a program. 22 Abstract Data Types (ADT) … Think of ADT as a mathematical model with a collection of operations defined on that model. Example: Set of integers, together with the operations of union, intersection, and set difference, form a simple example of an ADT. The two properties of procedures mentioned above – generalization and encapsulation – apply equally well to abstract data types. An implementation of an ADT is a translation, into statements of a programming language. 23 Data types, Data Structures, and Abstract Data Types In a programming language, the data type of a variable is the set of values that the variable may assume. Abstract Data Type is a mathematical model, together with various operations defined on the model. To represent the mathematical model underlying an ADT we use data structures, which are collection of variables, possibly of several different data types, connected in various ways. 24 The problem solving process program An Mathematical Abstract data Data structures model types Pseudo - Informal language Java program algorithm program 25 An Introduction Introduction Proof by induction Asymptotic notation Review: Induction Suppose – S(k) is true for fixed constant k Often k = 0 – S(n) S(n+1) for all n >= k Then S(n) is true for all n >= k 27 Proof By Induction Claim:S(n) is true for all n >= k Basis: – Show formula is true when n = k Inductive hypothesis: – Assume formula is true for an arbitrary n Step: – Show that formula is then true for n+1 28 Induction Example: Gaussian Closed Form Prove 1 + 2 + 3 + … + n = n(n+1) / 2 – Basis: If n = 0, then 0 = 0(0+1) / 2 – Inductive hypothesis: Assume 1 + 2 + 3 + … + n = n(n+1) / 2 – Step (show true for n+1): 1 + 2 + … + n + n+1 = (1 + 2 + …+ n) + (n+1) = n(n+1)/2 + n+1 = [n(n+1) + 2(n+1)]/2 = (n+1)(n+2)/2 = (n+1)(n+1 + 1) / 2 29 Induction Example: Geometric Closed Form Prove a0 + a1 + … + an = (an+1 - 1)/(a - 1) for all a 1 – Basis: show that a0 = (a0+1 - 1)/(a - 1) a0 = 1 = (a1 - 1)/(a - 1) – Inductive hypothesis: Assume a0 + a1 + … + an = (an+1 - 1)/(a - 1) – Step (show true for n+1): a0 + a1 + … + an+1 = a0 + a1 + … + an + an+1 = (an+1 - 1)/(a - 1) + an+1 = (an+1+1 - 1)/(a - 1) 30 Induction Example: Gaussian Closed Form Prove that 12 + 22 + 32 + … + n2 = (2n3 + 3n2 + n)/6 (sum of squares) What is this called? Polynomial Series. 31 Induction… We’ve been using weak induction Strong induction also holds – Basis: show S(0) – Hypothesis: assume S(k) holds for arbitrary k <= n – Step: Show S(n+1) follows Another variation: – Basis: show S(0), S(1) – Hypothesis: assume S(n) and S(n+1) are true – Step: show S(n+2) follows 32 Chapter 1 Algorithms: Efficiency, Analysis, and Order 33 Algorithms Looking at individual modules that accomplish a specific task. Need an algorithm that will solve all instances (with any values of parameters) of problem Solution needs to be efficient – solves problem within the required resource constraints 34 Algorithms Problem: is a questions to which we seek an answer. Example: Sort a list S of n numbers in nondecreasing order. The answer is the numbers in sorted sequence. Other examples? 35 Algorithms … 1.1 - Sequential search 1.2 - Add array members 1.3 - Exchange sort (bubble sort) 1.4 - Matrix multiplication 1.5 – Binary search 36 Developing efficient Algorithms Sequential search versus binary search 1.6 - The Fibonacci sequence (recursive) 1.7 - The Fibonacci sequence (iterative) 37 Analysis of Algorithms Complexity analysis Analysis of algorithm 1.2 Analysis of algorithm 1.3 Analysis of algorithm 1.4 Analysis of algorithm 1.5 Analysis of algorithm 1.1 38 Analysis of Algorithms Algorithm analysis measured the efficiency of an algorithm as the input size becomes large. – Critical resource is most often its running time – Can also take into account the space required to run the program 39 Complexity Analysis Looking for a measure that is independent of the computer, the programming language, the programmer, and all the complex details of the algorithm. Analyze the algorithm efficiency by determining the number of times some basic operation is done as a function of the size of the input. 40 Complexity Analysis… After determining the input size, pick some instruction (or group of instructions) such that the total work done by the algorithm is roughly proportional to the number of times this instruction is done. In general, time complexity analysis of an algorithm is the determination of how many times the basic operation is done for each value of the input size. 41 Order What is order? An intuitive introduction to order A rigorous introduction to order Using a limit to determine order 42 Order … Is 0.01n2 > 100n? Linear-time algorithms Quadratic-time algorithms ( (n2 ) algorithm) Pure quadratic Complete quadratic 43 Asymptotic Performance In this course, we care most about asymptotic performance – How does the algorithm behave as the problem size gets very large? Running time Memory/storage requirements Bandwidth/power requirements/logic gates/etc. 44 Asymptotic Notation By now you should have an intuitive feel for asymptotic (big-O) notation: – What does O(n) running time mean? O(n2)? O(n log n)? – How does asymptotic running time relate to asymptotic memory usage? Our first task is to define this notation more formally and completely 45 Analysis of Algorithms Analysis is performed with respect to a computational model We will usually use a generic uniprocessor random- access machine (RAM) – All memory equally expensive to access – No concurrent operations – All reasonable instructions take unit time Except, of course, function calls – Constant word size Unless we are explicitly manipulating bits 46 Input Size Time and space complexity – This is generally a function of the input size E.g., sorting, multiplication – How we characterize input size depends: Sorting: number of input items Multiplication: total number of bits Graph algorithms: number of nodes & edges Etc 47 Running Time Algorithm FindMax Int findMax (E, n) 1. max = E [0]; 2. for (index = 1; index < n; index ++) 3. if (max < E [index]) 4. max = E [index]; 5. return max; 48 Running Time Number of primitive steps that are executed – Except for time of executing a function call, most statements roughly require the same amount of time y = m * x + b c = 5 / 9 * (t - 32 ) z = f(x) + g(y) We can be more exact if need be 49 Analysis Worst case – Provides an upper bound on running time – An absolute guarantee Average case – Provides the expected running time – Very useful, but treat with care: what is “average”? Random (equally likely) inputs Real-life inputs 50 The running time of a program