VIEWS: 8 PAGES: 15 POSTED ON: 4/19/2011 Public Domain
CSCI 341 – Discrete Structures Analyzing Algorithms Friday, October 7, 2010 Sorting Algorithms – Assignment #5 Comparing sorting algorithms: Bubble Sort Selection Sort Insertion Sort Shell Sort Quick Sort Choose array lengths so times are appropriate Learn about the sorting algorithms: http://www.cs.pitt.edu/~kirk/cs1501/animations/Sort1.html Optimal Algorithm Problem Suppose algorithm A solves problem P Ex. Think of sorting n items…inputs I = random Is A the best solution to P? data, sorted data, reverse, Count the number of operations order, almost sorted, etc. We are really interested in time. Is it the fastest algorithm? Worst-case function for A: WA (n) max{time( I ) | I is an input and size(I)=n} Algorithm A is optimal in the worst case for problem P if for any algorithm B that exists, or ever will exist, the following holds: WA (n) WB (n) for all n > 0 Analyzing Algorithms How to find an algorithm that is optimal in the worst case? 1) Find algorithm A to solve P. Do analysis of A to find worst-case function WA. 2) Find a lower bound: function F such that F(n) < WB(n) for all n>0 and for all algorithms that solve P. 3) Compare F and WA. If F = WA, then A is optimal in worst case. Note: A vast majority of problems do not have optimal algorithms. Example: Matrix Multiplication Suppose you are multiplying two matrices, A and B, both of which are n x n. The product C is also n x n. Each element in the product C is found by finding a dot n product: cij aik bkj k 1 a1,1 a1,2 a1,3 a1,4 b1,1 x x x c1,1 x x x x x x x x x x b2,1 x x x x x x x x b3,1 x x x x x x x x x x x b4,1 x x x x x x x How many total multiplications are needed? n3 Strassen[1969] proved you only need n2.81 multiplications A known lower bound is n2 Example: Maximum Problem: Find the maximum in a list of n items To find the maximum, we need to examine all n elements Therefore, n-1 is a lower bound on the number of comparisons This algorithm is optimal: max = a[1] for (i=2; i<n; i++) if (a[i] > max) max = a[i] Decision Trees Trees can sometimes be used to represent the decision process of an algorithm Nodes represent decisions, leaves represent outcomes Lower bounds may equate to the depth of the tree If a decision tree has depth d, then a path from root to a leaf contains d+1 nodes, making d decisions on the way Suppose a problem has n possible outcomes and can be solved by a binary decision tree algorithm What is the best binary decision tree algorithm? If we cannot answer this question, can we find a bound on the depth of any binary tree to solve the problem? Example: Pan balance Suppose out of 8 coins that look alike, your job is to find the one coin that is heavier that the other 7 A pan balance is used. How many weighings are needed? Since the problem has 8 outcomes, there are 8 leaves Depth of tree is 3. The number of leaves is at most 23 (or 8) Three is lower bound using a binary decision tree If d is the depth of a binary decision tree with n possible outcomes, then n<2d Therefore, the depth of the tree must be at least log 2 n Example: Pan balance Suppose some of our weighings do not include all of the coins Now there are 3 possible results: i) Left pan is heavier, ii) right pan is heavier, and iii) pans weigh the same Our decision tree becomes a ternary tree There are 8 leaves, so we must have with a ternary tree 3d>8, or d>2 Therefore, 2 is a lower bound for the number of weighings Since the algorithm solves the problem in 2 weighings, it is optimal Closed Forms Closed form – expression that can be computed by applying a fixed number of familiar operations to the arguments Example: n(n+1)/2 is closed form 1+2+...+n is not closed form (contains ellpsis) The following notation is used to represent the sum of n terms a1,a2,…,an: n a k 1 k a1 a2 ... an Example: Adding Odd Numbers Find sum: 1+3+5+...+(2n+1) n (2k 1) k 0 You can use Summation Facts (page 290 of your textbook) to help simply n n 2k 1 expressions. k 0 k 0 n n 2 k 1 k 0 k 0 Let’s double-check this 2n(n 1) for n=5: n 1 2 1+3+5+7+9+11 The counting techniques Counting that we are examining often come in handy when analyzing algorithms. Suppose a set S={w,x,y} How many arrangements (sequences in which the order does matter) can we make of the three letters? wxy, wyx, xwy, xyw, zwx, zxw There are n! permutations of an n-element set (ex. 3!=6) How many arrangements of S consisting of 2 letters? wx, wy, xw, xy, zw, zy Permutations of n objects taken r at a time: P(n, r ) n! (n r )! 3! ex., P(3, 2) 6 (3 2)! More Counting Suppose T={w,x,y,z} How many combinations (groups in which the order does not matter) of three letters can be made from T? wxy, wxz, wyz, xyz n! Combinations of n items taken r at a time: C (n, r ) r !(n r )! 4! ex., P(4,3) 4 3!(4 3)! Sorting In sorting n items, how many comparisons are needed? A decision tree would need a leaf for every possible arrangement of the n items….=> n! leaves The binary tree depth: n!<2d, thus log 2 n ! d Worst-case lower bound for the number of comparisons needed to sort n items: log 2 n! The expression log 2 n! is hard to calculate for large n, but it is approximately n log 2 n In other words, n log 2 n is the worst-case lower bound for comparison sorting. I know Yea, let’s Dakota has Wrapping things up…. do it!!! Rock on!! been sleeping Yea, I wonder if for I think we quite a anyone besides should have a while That was Nikolai surprise quiz. Great a lot of understood it all Who’s with me? idea! I’m material in! It’s been a long week. Maybe Twig and Collette should let the class out early