Docstoc

Chapter 3 Proofs

Document Sample
Chapter 3 Proofs Powered By Docstoc
					             Chapter 3: Proofs
• Chapter 3 introduces the concept of formal proofs
   – Among the techniques that we will use are:
      • Induction (3.3, 3.4)
      • Recurrence Equations and Recurrence Trees (3.6, 3.7)
   – While the chapter also introduces recursion and proving
     the correctness of code, we will skip those sections
   – Why Proofs?
      • Proofs are the result of reasoning with logical statements and
        produce propositions, theorems, lemmas and corollaries
      • We will use proofs in order to prove limitations of algorithms
      • Without a proof, we might have an intuition about an algorithm
        but we cannot know the algorithm’s complexity or whether the
        algorithm is optimal or not
          – We will explore formal proofs and proof sketches, but we will
            also “hand wave” our way through some proofs for brevity
 Format of a Theorem or Proposition
• Two parts of a proposition/theorem:
   – Assumption (premises or hypotheses) that must be true for the
     conclusion to be true
   – Conclusion or goal statement, which is the result of applying
     the theorem/proposition
   – Example: x  W ( A(x) => C(x) )
      • A(x) is the assumption – for us to draw the conclusion, the assumption
        must hold and in this case, it means that if a given x is in A
      • C(x) is the conclusion – what is known to be true if the assumption is
        true
      • Example: All men are mortal, so if x is a man, then x is mortal
          – This can be written as x (man(x) => mortal(x) )
      • Example: if x is greater than 0, then x * 2 is greater than 0
          – This can be written as x (x  N+ => 2 * x  N+)
         Two-Column Proof Format
 • A proof must start with assumptions and use known (true) statements
   (from other theorems, lemmas, etc) or laws and various inference rules
   to reach the conclusion
 • When some known statement or law is applied, we annotate it in the
   proof in a second column
 • That is, the second column illustrates how we reached this point in the
   proof
Example: For all x, y  R, if x and y are constants with x < y then 2xn  o(2yn)
    Statement                                               Justification
1. First show that lim ninfinity2yn / 2xn = infinity
2. 2yn / 2xn = 2(y-x)n                                      Math Identity (MI)
3. y - x > 0 and is a constant                              Stated in theorem as
                                                            an assumption + MI
4. lim ninfinity2(y - x) * n = infinity                    (3)+math property (MP)
5. lim ninfinity2yn / 2xn = infinity                       (2) + (4) + substitution
6. 2xn  o(2yn)                                             (5) + definition of o
                                                            (definition 1.17)
                  Induction Proofs
• Some proofs can be done
   – as shown on the last slide, by using various other theorems,
     definitions and mathematical identities
• others might apply the logic-based strategies of
   – proof by contradiction or the application of rules of inference
• Some proofs can only be solved through induction
   – This is especially true with statements about an infinite set of
     objects such as numbers
• In induction
   – we must prove the base case is true
   – we then assume that the proof is true for the recursive or
     general case when n = some value k
   – and use this to prove that the result holds for n = k + 1
                              Example
 n >= 0, i=0n i(i+1)/2 = n(n+1)(n+2)/6
  Statement                         Justification
1.   The proof is by induction on n
2.   The base case is n = 0
3.   i=00 i(i+1)/2 = 0 * 1 / 2 = 0 and              Math
     0*(0+1)(0+2)/6 = 0
4.   For n greater than 0, assume the result holds
     for some k > 0 and < n and prove for n=k+1
5.   i=0n-1 i(i+1)/2 = (n-1)(n)(n+1)/6              (4) substituting k+1
6.   i=0n i(i+1)/2 = (n-1)(n)(n+1)/6 + n*(n+1)/2    (5)
7.   (n-1)(n)(n+1)/6 + n*(n+1)/2 = (n3-n)/6+(n2+n)/2
     = (n3 + 3*n2 + 2*n) / 6
     = n(n+1)(n+2) / 6                               Math
8.   Therefore, i=0n i(i+1)/2 = n(n+1)(n+2)/6       (6) + (7)
       Determining Complexity
• In chapter 1, we looked at examples of deriving
  computational complexity by counting the number
  of comparisons
   – We count comparisons and multiply by the number of
     loop iterations
• What about if the code using recursion?
   – We need to determine the number of recursive calls
     which isn’t as easy as counting loop iterations
• To determine number of recursive calls, we create
  a recurrence equation for the recursive code
   – The recurrence equation defines a function that can be
     used to derive the number of recursive calls and
     therefore the complexity of the code
                              Example
                                          seqSearchRec(E, index, num, K)
• Recall the sequential search             1. if (index >= num)
  algorithm from chapter 1, here we
  solve it through recursion               2. ans = -1;
• Counting only comparisons and            3. else if (E[index] == K)
  ignoring the base case we have 1         4. ans = index;
  comparison in instruction 3 which
  is performed during every recursive      5. else
  call (excluding the base case)           6. ans = seqSearchRec
• How many times is the method                      (E, index+1, num, K);
  called recursively?                      7. return ans;
• We develop a recurrence relation:
   – T(n) = (1 + max(0, T(n-1)) or         The bounding function can be
   – T(n) = T(n-1) + 1                     Determined by noticing that
   – But what is the complexity of this    T(n) = T(n-1) + 1 = T(n-2) + 2
     algorithm? We must find a             = T(n-3) + 3 = … = T(0) + n
     function that bounds T(n)             Since T(0) = 0, T(n)  (n)
                          Example
• Recall binary search (see algorithm 1.4)
• How much work does it do?
   – If we start with n=last – first + 1, then either we have reached
     the base case (first > last), or we have found the item at mid =
     (first + last) / 2, or we have to search either first .. mid – 1 or
     mid + 1 .. last
   – The amount of work done recursively is then either the base
     case, the item is found, or a search over n/2 – 1 items
   – Thus, the recurrence relation is T(n) = T(n/2 – 1) + 1
   – For convenience, we will round this off to T(n) = T(n/2) + 1
   – T(n) = T(n/2) + 1 = T(n/4) + 2 = T(n/8) + 3 = … = T(n/2 d) + d
   – T(1) = 0 and n/ 2d = 1 when n = 2d, or d = log n
   – Therefore, we have T(1) + d = 0 + log n so T(n)  (log n)
                        Example
• Consider a situation where two recursive calls are
  made, each of which takes half the amount of
  work (mergesort is an example like this)
• For an example, consider that each recursive call
  does 1 comparison and then calls both the left-side
  and the right-side of the remaining data
   – T(n) = T(n/2) + T(n/2) + 1
   – T(n) = 2 * T(n/2) + 1 = 4 * T(n/4) + 3 = 8 * T(n/8) + 7
     = … = 2d * T(n/2d) + 2d - 1 = 2d * T(1) + 2d - 1
   – Again, d = log n and T(1) = 0
   – T(n) = 2d * 0 + 2d -1 = 0 + n – 1 so T(n)  (n)
      • Does this make sense that a recursive method that divides the
        data into two sets will have a complexity of (n)?
                      A Variation
• From our previous example, what would happen
  if the data is not divided exactly in half each
  iteration?
  – For instance, imagine that we pick a point 2/3 over
    so that we have recursive calls over 1/3 – 1 and 2/3
    – 1 of the data?
  – Assume the “split point” is r where 0<r<n
     • T(n) = T(n – 1 – r) + T(r) + 1 (with T(0) = 1)
  – How do we solve this?
     • By substitution, this solves to be T(n) = 2*n+1  (n) for
       any r in the range 0<r<n
        – We will need this result when we visit the quicksort algorithm
             Common Recurrences
• With recursion we generally see one of four behaviors:
   – We divide the space into a subspace to examine during the next
     recursive call
      • T(n) = T(n/2) + x with T(1) = 0
      • This is the case with binary search
   – We divide the space into a subspace and examine each
     subspace recursively
      • T(n) = T(n/2) + T(n/2) + x with T(1) = 0
   – We subtract some amount from the space and examine the
     remainder in the next recursive call
      • T(n) = T(n-c) + x with T(0) = 1
   – We subtract some amount and then have to perform more than
     one recursive call
      • T(n) = T(n-c) + T(n-c) + x with T(0) = 1
          – The first two are known as “divide and conquer”, the third is “chip and
            conquer”, the fourth is “chip and be conquered”
                 More Generally…
• We will combine cases 1 and 2 to be:
   – T(n) = b * T(n/c) + f(n)
      • b is the number of times the method must be called recursively each
        time
      • b = 1 for binary search, b = 2 for mergesort
      • c is the amount we divide the search space by each time
      • f(n) is the amount of work for all recursive calls of size n/c
• The third case is
   – T(n) = T(n - c) + f(n)
      • the sequential search examined previously is an example of this
• The fourth case is
   – T(n) = b * T(n – c) + f(n)
      • if b > 1 then we will wind up with a very large complexity
      • if c is not the same for each recursive call, we may still be able to
        identify the complexity easily enough if each c falls within a range
                        Recursion Trees
• To better visualize what goes on with the recursive
  method, we can draw a recursion tree
   – The recursion tree denotes a node which contains the current size and
     the non-recursive cost for that node
   – For instance, T(n) = T(n/2) + T(n/2) + n is given to the below
   – Notice that in each level, the non-recursive cost is n
   – How many levels are there? Since we divide n by 2, we can do this
     log n times, or log n levels
       • Assume n = 8, then T(n) = n + 2 * (n / 2) + 4 * (n / 4) = 3 * n or n log n
   – T(n)  (n log n)
            Deriving the Complexity
• Now let’s determine how to compute the complexity
  given a recurrence relation
• The general form of our recurrence relation is one of
   – T(n) = b * T(n / c) + f(n)
   – T(n) = b * T(n – c) + f(n)
• Each of these will have a different way to calculate the
  complexity, we start with the first one
   –   Compute E = log b / log c (E is the critical exponent)
   –   The depth of our recursion tree is D = log n / log c
   –   The zeroth row has a non-recursive cost of f(n)
   –   The Dth row has nE leaves
   –   With a base cost of 1, the Dth row requires (nE)
   –   What is the non-recursive cost of the in-between rows?
          Little Master Theorem
• We use this theorem to tell us the complexity of
  the algorithm given the non-recursive work
   – T(n)  (nE) if the non-recursive work increases in a
     geometric series
       • This is because the lowest level dominates all other levels and
         there are nE nodes in the lowest level
   – T(n)  (f(n) log n)) if the non-recursive work stays roughly
     constant per level
       • This is because there are log n levels, each of which does
         roughly the same amount of work, f(n)
   – T(n)  (f(n)) if the non-recursive work decreases in a
     geometric series per level
       • This is because the top level dominates in terms of the amount
         of work, and the amount of work at the top level is f(n)
                  Master Theorem
• A more precise way to state the conclusions of the
  Little Master Theorem is given in the Master
  Theorem
  – Assume a recurrence equation of T(n) = b * T (n / c) +
    f(n) where E = log b / log c
  – Then:
     • If f(n)  O(nE-e) for some positive e, then T(n)  (nE),
       which is proportional to the number of leaves in the recursion
       tree
     • If f(n)  (nE), then T(n)  (f(n) * log n) as all node depths
       contribute equally
     • If f(n)  (nE+e) for some positive e and f(n) T(n)  O(nE+d)
       for some d > e, then T(n)  (f(n)) which is proportional to
       the nonrecursive cost of the root node of the recursion tree
                              Examples
• T(n) = 3 * T(n / 4) + n
   – E = log 3 / log 4 < 1
   – f(n) = n which is in O(nE+e) (since E < 1, some positive e must
     be added to E so that f(n) is in O(n1) so the third part of the
     Master Theorem applies
   – Thus, T(n)  (f(n)) or T(n)  (n)
      • Does this make sense? Yes, each level requires n operations, and there
        are log4n levels with n decreasing each time, so the most work is done at
        the root and so the work done at the root bounds the overall process
• T(n) = 2 * T(n / 2) + log n
   – E = log 2 / log 2 = 1
   – f(n) = log n
      • In this case, Master Theorem part 1 applies since log n  O(nE-e), so T(n)
         (n) the most work occurs at the leaf level
          – there will be n occurrences of (1) which is (n) at this level
                   Chip Solutions
• For the chip and conquer/be conquered, we need a
  different approach
   – Recall T(n) = b * T(n – c) + f(n)
   – Assume d is the depth of the tree (d = n / c or
     approximately that depending on whether c is uniform)
   – T(n) = b n/c h=0n/c f (c * h) / bh
      • where h = (n / c) – d
      • h = 0 at the leaves, and increases toward the root
   – T(n)   (b n/c ) unless b = 1, then we have one of:
      • T(n)   (na+1) if f(n) is in the polynomial na, or
      • T(n)   (n log n) if f(n) is in log n
   – Examples:
      • T(n) = 2 * T(n – 1) + 1 results in T(n)   (2n/1) or  (2n)
      • T(n) = T(n – 5) + log n results in T(n)   (n log n)
      • T(n) = T(n – 1) + n2 results in T(n)  (n3)

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:3
posted:10/4/2010
language:English
pages:18