# Chapter 3 Proofs

Document Sample

```					             Chapter 3: Proofs
• Chapter 3 introduces the concept of formal proofs
– Among the techniques that we will use are:
• Induction (3.3, 3.4)
• Recurrence Equations and Recurrence Trees (3.6, 3.7)
– While the chapter also introduces recursion and proving
the correctness of code, we will skip those sections
– Why Proofs?
• Proofs are the result of reasoning with logical statements and
produce propositions, theorems, lemmas and corollaries
• We will use proofs in order to prove limitations of algorithms
• Without a proof, we might have an intuition about an algorithm
but we cannot know the algorithm’s complexity or whether the
algorithm is optimal or not
– We will explore formal proofs and proof sketches, but we will
also “hand wave” our way through some proofs for brevity
Format of a Theorem or Proposition
• Two parts of a proposition/theorem:
– Assumption (premises or hypotheses) that must be true for the
conclusion to be true
– Conclusion or goal statement, which is the result of applying
the theorem/proposition
– Example: x  W ( A(x) => C(x) )
• A(x) is the assumption – for us to draw the conclusion, the assumption
must hold and in this case, it means that if a given x is in A
• C(x) is the conclusion – what is known to be true if the assumption is
true
• Example: All men are mortal, so if x is a man, then x is mortal
– This can be written as x (man(x) => mortal(x) )
• Example: if x is greater than 0, then x * 2 is greater than 0
– This can be written as x (x  N+ => 2 * x  N+)
Two-Column Proof Format
• A proof must start with assumptions and use known (true) statements
(from other theorems, lemmas, etc) or laws and various inference rules
to reach the conclusion
• When some known statement or law is applied, we annotate it in the
proof in a second column
• That is, the second column illustrates how we reached this point in the
proof
Example: For all x, y  R, if x and y are constants with x < y then 2xn  o(2yn)
Statement                                               Justification
1. First show that lim ninfinity2yn / 2xn = infinity
2. 2yn / 2xn = 2(y-x)n                                      Math Identity (MI)
3. y - x > 0 and is a constant                              Stated in theorem as
an assumption + MI
4. lim ninfinity2(y - x) * n = infinity                    (3)+math property (MP)
5. lim ninfinity2yn / 2xn = infinity                       (2) + (4) + substitution
6. 2xn  o(2yn)                                             (5) + definition of o
(definition 1.17)
Induction Proofs
• Some proofs can be done
– as shown on the last slide, by using various other theorems,
definitions and mathematical identities
• others might apply the logic-based strategies of
– proof by contradiction or the application of rules of inference
• Some proofs can only be solved through induction
– This is especially true with statements about an infinite set of
objects such as numbers
• In induction
– we must prove the base case is true
– we then assume that the proof is true for the recursive or
general case when n = some value k
– and use this to prove that the result holds for n = k + 1
Example
 n >= 0, i=0n i(i+1)/2 = n(n+1)(n+2)/6
Statement                         Justification
1.   The proof is by induction on n
2.   The base case is n = 0
3.   i=00 i(i+1)/2 = 0 * 1 / 2 = 0 and              Math
0*(0+1)(0+2)/6 = 0
4.   For n greater than 0, assume the result holds
for some k > 0 and < n and prove for n=k+1
5.   i=0n-1 i(i+1)/2 = (n-1)(n)(n+1)/6              (4) substituting k+1
6.   i=0n i(i+1)/2 = (n-1)(n)(n+1)/6 + n*(n+1)/2    (5)
7.   (n-1)(n)(n+1)/6 + n*(n+1)/2 = (n3-n)/6+(n2+n)/2
= (n3 + 3*n2 + 2*n) / 6
= n(n+1)(n+2) / 6                               Math
8.   Therefore, i=0n i(i+1)/2 = n(n+1)(n+2)/6       (6) + (7)
Determining Complexity
• In chapter 1, we looked at examples of deriving
computational complexity by counting the number
of comparisons
– We count comparisons and multiply by the number of
loop iterations
• What about if the code using recursion?
– We need to determine the number of recursive calls
which isn’t as easy as counting loop iterations
• To determine number of recursive calls, we create
a recurrence equation for the recursive code
– The recurrence equation defines a function that can be
used to derive the number of recursive calls and
therefore the complexity of the code
Example
seqSearchRec(E, index, num, K)
• Recall the sequential search             1. if (index >= num)
algorithm from chapter 1, here we
solve it through recursion               2. ans = -1;
• Counting only comparisons and            3. else if (E[index] == K)
ignoring the base case we have 1         4. ans = index;
comparison in instruction 3 which
is performed during every recursive      5. else
call (excluding the base case)           6. ans = seqSearchRec
• How many times is the method                      (E, index+1, num, K);
called recursively?                      7. return ans;
• We develop a recurrence relation:
– T(n) = (1 + max(0, T(n-1)) or         The bounding function can be
– T(n) = T(n-1) + 1                     Determined by noticing that
– But what is the complexity of this    T(n) = T(n-1) + 1 = T(n-2) + 2
algorithm? We must find a             = T(n-3) + 3 = … = T(0) + n
function that bounds T(n)             Since T(0) = 0, T(n)  (n)
Example
• Recall binary search (see algorithm 1.4)
• How much work does it do?
– If we start with n=last – first + 1, then either we have reached
the base case (first > last), or we have found the item at mid =
(first + last) / 2, or we have to search either first .. mid – 1 or
mid + 1 .. last
– The amount of work done recursively is then either the base
case, the item is found, or a search over n/2 – 1 items
– Thus, the recurrence relation is T(n) = T(n/2 – 1) + 1
– For convenience, we will round this off to T(n) = T(n/2) + 1
– T(n) = T(n/2) + 1 = T(n/4) + 2 = T(n/8) + 3 = … = T(n/2 d) + d
– T(1) = 0 and n/ 2d = 1 when n = 2d, or d = log n
– Therefore, we have T(1) + d = 0 + log n so T(n)  (log n)
Example
• Consider a situation where two recursive calls are
made, each of which takes half the amount of
work (mergesort is an example like this)
• For an example, consider that each recursive call
does 1 comparison and then calls both the left-side
and the right-side of the remaining data
– T(n) = T(n/2) + T(n/2) + 1
– T(n) = 2 * T(n/2) + 1 = 4 * T(n/4) + 3 = 8 * T(n/8) + 7
= … = 2d * T(n/2d) + 2d - 1 = 2d * T(1) + 2d - 1
– Again, d = log n and T(1) = 0
– T(n) = 2d * 0 + 2d -1 = 0 + n – 1 so T(n)  (n)
• Does this make sense that a recursive method that divides the
data into two sets will have a complexity of (n)?
A Variation
• From our previous example, what would happen
if the data is not divided exactly in half each
iteration?
– For instance, imagine that we pick a point 2/3 over
so that we have recursive calls over 1/3 – 1 and 2/3
– 1 of the data?
– Assume the “split point” is r where 0<r<n
• T(n) = T(n – 1 – r) + T(r) + 1 (with T(0) = 1)
– How do we solve this?
• By substitution, this solves to be T(n) = 2*n+1  (n) for
any r in the range 0<r<n
– We will need this result when we visit the quicksort algorithm
Common Recurrences
• With recursion we generally see one of four behaviors:
– We divide the space into a subspace to examine during the next
recursive call
• T(n) = T(n/2) + x with T(1) = 0
• This is the case with binary search
– We divide the space into a subspace and examine each
subspace recursively
• T(n) = T(n/2) + T(n/2) + x with T(1) = 0
– We subtract some amount from the space and examine the
remainder in the next recursive call
• T(n) = T(n-c) + x with T(0) = 1
– We subtract some amount and then have to perform more than
one recursive call
• T(n) = T(n-c) + T(n-c) + x with T(0) = 1
– The first two are known as “divide and conquer”, the third is “chip and
conquer”, the fourth is “chip and be conquered”
More Generally…
• We will combine cases 1 and 2 to be:
– T(n) = b * T(n/c) + f(n)
• b is the number of times the method must be called recursively each
time
• b = 1 for binary search, b = 2 for mergesort
• c is the amount we divide the search space by each time
• f(n) is the amount of work for all recursive calls of size n/c
• The third case is
– T(n) = T(n - c) + f(n)
• the sequential search examined previously is an example of this
• The fourth case is
– T(n) = b * T(n – c) + f(n)
• if b > 1 then we will wind up with a very large complexity
• if c is not the same for each recursive call, we may still be able to
identify the complexity easily enough if each c falls within a range
Recursion Trees
• To better visualize what goes on with the recursive
method, we can draw a recursion tree
– The recursion tree denotes a node which contains the current size and
the non-recursive cost for that node
– For instance, T(n) = T(n/2) + T(n/2) + n is given to the below
– Notice that in each level, the non-recursive cost is n
– How many levels are there? Since we divide n by 2, we can do this
log n times, or log n levels
• Assume n = 8, then T(n) = n + 2 * (n / 2) + 4 * (n / 4) = 3 * n or n log n
– T(n)  (n log n)
Deriving the Complexity
• Now let’s determine how to compute the complexity
given a recurrence relation
• The general form of our recurrence relation is one of
– T(n) = b * T(n / c) + f(n)
– T(n) = b * T(n – c) + f(n)
• Each of these will have a different way to calculate the
complexity, we start with the first one
–   Compute E = log b / log c (E is the critical exponent)
–   The depth of our recursion tree is D = log n / log c
–   The zeroth row has a non-recursive cost of f(n)
–   The Dth row has nE leaves
–   With a base cost of 1, the Dth row requires (nE)
–   What is the non-recursive cost of the in-between rows?
Little Master Theorem
• We use this theorem to tell us the complexity of
the algorithm given the non-recursive work
– T(n)  (nE) if the non-recursive work increases in a
geometric series
• This is because the lowest level dominates all other levels and
there are nE nodes in the lowest level
– T(n)  (f(n) log n)) if the non-recursive work stays roughly
constant per level
• This is because there are log n levels, each of which does
roughly the same amount of work, f(n)
– T(n)  (f(n)) if the non-recursive work decreases in a
geometric series per level
• This is because the top level dominates in terms of the amount
of work, and the amount of work at the top level is f(n)
Master Theorem
• A more precise way to state the conclusions of the
Little Master Theorem is given in the Master
Theorem
– Assume a recurrence equation of T(n) = b * T (n / c) +
f(n) where E = log b / log c
– Then:
• If f(n)  O(nE-e) for some positive e, then T(n)  (nE),
which is proportional to the number of leaves in the recursion
tree
• If f(n)  (nE), then T(n)  (f(n) * log n) as all node depths
contribute equally
• If f(n)  (nE+e) for some positive e and f(n) T(n)  O(nE+d)
for some d > e, then T(n)  (f(n)) which is proportional to
the nonrecursive cost of the root node of the recursion tree
Examples
• T(n) = 3 * T(n / 4) + n
– E = log 3 / log 4 < 1
– f(n) = n which is in O(nE+e) (since E < 1, some positive e must
be added to E so that f(n) is in O(n1) so the third part of the
Master Theorem applies
– Thus, T(n)  (f(n)) or T(n)  (n)
• Does this make sense? Yes, each level requires n operations, and there
are log4n levels with n decreasing each time, so the most work is done at
the root and so the work done at the root bounds the overall process
• T(n) = 2 * T(n / 2) + log n
– E = log 2 / log 2 = 1
– f(n) = log n
• In this case, Master Theorem part 1 applies since log n  O(nE-e), so T(n)
 (n) the most work occurs at the leaf level
– there will be n occurrences of (1) which is (n) at this level
Chip Solutions
• For the chip and conquer/be conquered, we need a
different approach
– Recall T(n) = b * T(n – c) + f(n)
– Assume d is the depth of the tree (d = n / c or
approximately that depending on whether c is uniform)
– T(n) = b n/c h=0n/c f (c * h) / bh
• where h = (n / c) – d
• h = 0 at the leaves, and increases toward the root
– T(n)   (b n/c ) unless b = 1, then we have one of:
• T(n)   (na+1) if f(n) is in the polynomial na, or
• T(n)   (n log n) if f(n) is in log n
– Examples:
• T(n) = 2 * T(n – 1) + 1 results in T(n)   (2n/1) or  (2n)
• T(n) = T(n – 5) + log n results in T(n)   (n log n)
• T(n) = T(n – 1) + n2 results in T(n)  (n3)

```
DOCUMENT INFO
Shared By:
Categories:
Stats:
 views: 3 posted: 10/4/2010 language: English pages: 18