CSCI 341 – Discrete Structures by gjjur4356


									CSCI 341 – Discrete Structures
         Analyzing Algorithms
              Friday, October 7, 2010
Sorting Algorithms – Assignment #5
   Comparing sorting algorithms:
     Bubble Sort
     Selection Sort
     Insertion Sort
     Shell Sort
     Quick Sort

   Choose array lengths so times are appropriate
   Learn about the sorting algorithms:
Optimal Algorithm Problem
   Suppose algorithm A solves problem P             Ex. Think of sorting n
                                                     items…inputs I = random
   Is A the best solution to P?                     data, sorted data, reverse,
       Count the number of operations               order, almost sorted, etc.

       We are really interested in time. Is it the fastest algorithm?
   Worst-case function for A:
         WA (n)  max{time( I ) | I is an input and size(I)=n}

   Algorithm A is optimal in the worst case for problem P if
    for any algorithm B that exists, or ever will exist, the
    following holds:
          WA (n)  WB (n) for all n > 0
Analyzing Algorithms
            How to find an algorithm that is
              optimal in the worst case?
1)   Find algorithm A to solve P. Do analysis of A to find
     worst-case function WA.
2)   Find a lower bound: function F such that F(n) < WB(n)
     for all n>0 and for all algorithms that solve P.
3)   Compare F and WA. If F = WA, then A is optimal in worst

Note: A vast majority of problems do not have optimal algorithms.
Example: Matrix Multiplication
   Suppose you are multiplying two matrices, A and B, both
    of which are n x n. The product C is also n x n.
   Each element in the product C is found by finding a dot
    product: cij   aik bkj
                         k 1
        a1,1   a1,2   a1,3     a1,4   b1,1   x   x x  c1,1   x x x
        x                                         x x  x
                x      x        x  b2,1
                                                x              x x x
        x       x      x        x  b3,1      x   x x  x      x x x
                                                                  
        x       x      x        x  b4,1
                                               x   x x  x
                                                                 x x x

   How many total multiplications are needed?                             n3
   Strassen[1969] proved you only need n2.81 multiplications
   A known lower bound is n2
Example: Maximum
   Problem: Find the maximum in a list of n items
   To find the maximum, we need to examine all n elements
   Therefore, n-1 is a lower bound on the number of
   This algorithm is optimal:
      max = a[1]
      for (i=2; i<n; i++)
        if (a[i] > max)
          max = a[i]
Decision Trees
   Trees can sometimes be used to represent the decision
    process of an algorithm
   Nodes represent decisions, leaves represent outcomes
   Lower bounds may equate to the depth of the tree
   If a decision tree has depth d, then a path from root to a
    leaf contains d+1 nodes, making d decisions on the way
   Suppose a problem has n possible outcomes and can be
    solved by a binary decision tree algorithm
   What is the best binary decision tree algorithm?
   If we cannot answer this question, can we find a bound on
    the depth of any binary tree to solve the problem?
Example: Pan balance
 Suppose out of 8 coins that look alike, your job is to find
  the one coin that is heavier that the other 7
 A pan balance is used. How many weighings are needed?
  Since the problem has 8
   outcomes, there are 8 leaves
  Depth of tree is 3. The number
   of leaves is at most 23 (or 8)
  Three is lower bound using a
   binary decision tree
       If d is the depth of a binary decision tree with n possible
        outcomes, then n<2d
       Therefore, the depth of the tree must be at least log 2 n 
                                                                  
Example: Pan balance
   Suppose some of our weighings do not include all of the coins
   Now there are 3 possible results: i) Left pan is heavier, ii) right
    pan is heavier, and iii) pans weigh the same
   Our decision tree becomes a ternary tree
       There are 8 leaves, so we must
        have with a ternary tree 3d>8,
        or d>2
       Therefore, 2 is a lower bound
        for the number of weighings

       Since the algorithm solves the problem in 2 weighings, it
        is optimal
Closed Forms
   Closed form – expression that can be computed by
    applying a fixed number of familiar operations to the
   Example: n(n+1)/2 is closed form
   1+2+...+n is not closed form (contains ellpsis)
   The following notation is used to represent the sum of n
    terms a1,a2,…,an:

            k 1
                   k    a1  a2  ...  an
Example: Adding Odd Numbers
   Find sum: 1+3+5+...+(2n+1)
          (2k  1)
          k 0
                             You can use Summation
                             Facts (page 290 of your
                             textbook) to help simply
           n        n

           2k  1
          k 0     k 0
               n    n
      2    k  1
           k 0    k 0
                             Let’s double-check this
         2n(n  1)           for n=5:
                   n 1
            2                1+3+5+7+9+11
                                              The counting techniques
Counting                                      that we are examining
                                              often come in handy when
                                              analyzing algorithms.
   Suppose a set S={w,x,y}
   How many arrangements (sequences in which the order
    does matter) can we make of the three letters?
   wxy, wyx, xwy, xyw, zwx, zxw
   There are n! permutations of an n-element set (ex. 3!=6)
   How many arrangements of S consisting of 2 letters?
   wx, wy, xw, xy, zw, zy
   Permutations of n objects taken r at a time: P(n, r )  n!
                                                             (n  r )!
               ex., P(3, 2)             6
                                (3  2)!
More Counting
   Suppose T={w,x,y,z}
   How many combinations (groups in which the order
    does not matter) of three letters can be made from T?
   wxy, wxz, wyz, xyz
   Combinations of n items taken r at a time: C (n, r )  r !(n  r )!

                      ex., P(4,3)               4
                                      3!(4  3)!
   In sorting n items, how many comparisons are needed?
   A decision tree would need a leaf for every possible
    arrangement of the n items….=> n! leaves
   The binary tree depth: n!<2d, thus log 2 n !  d
                                                
   Worst-case lower bound for the number of comparisons
    needed to sort n items: log 2 n!
                                       
   The expression log 2 n! is hard to calculate for large n,
                            
    but it is approximately n log 2 n
   In other words, n log 2 n is the worst-case lower bound
    for comparison sorting.
                                                                I know
               Yea, let’s                                    Dakota has
  Wrapping things up….
                do it!!!
                                              Rock on!!     been sleeping
                            Yea, I wonder if                  for
                                                     I think we quite a
                             anyone besides        should have a while
   That was                      Nikolai           surprise quiz.             Great
    a lot of                understood it all     Who’s with me?            idea! I’m
   material                                                                    in!

It’s been a
 Twig and
should let
 the class
 out early

To top