Document Sample

Chapter 3: Proofs • Chapter 3 introduces the concept of formal proofs – Among the techniques that we will use are: • Induction (3.3, 3.4) • Recurrence Equations and Recurrence Trees (3.6, 3.7) – While the chapter also introduces recursion and proving the correctness of code, we will skip those sections – Why Proofs? • Proofs are the result of reasoning with logical statements and produce propositions, theorems, lemmas and corollaries • We will use proofs in order to prove limitations of algorithms • Without a proof, we might have an intuition about an algorithm but we cannot know the algorithm’s complexity or whether the algorithm is optimal or not – We will explore formal proofs and proof sketches, but we will also “hand wave” our way through some proofs for brevity Format of a Theorem or Proposition • Two parts of a proposition/theorem: – Assumption (premises or hypotheses) that must be true for the conclusion to be true – Conclusion or goal statement, which is the result of applying the theorem/proposition – Example: x W ( A(x) => C(x) ) • A(x) is the assumption – for us to draw the conclusion, the assumption must hold and in this case, it means that if a given x is in A • C(x) is the conclusion – what is known to be true if the assumption is true • Example: All men are mortal, so if x is a man, then x is mortal – This can be written as x (man(x) => mortal(x) ) • Example: if x is greater than 0, then x * 2 is greater than 0 – This can be written as x (x N+ => 2 * x N+) Two-Column Proof Format • A proof must start with assumptions and use known (true) statements (from other theorems, lemmas, etc) or laws and various inference rules to reach the conclusion • When some known statement or law is applied, we annotate it in the proof in a second column • That is, the second column illustrates how we reached this point in the proof Example: For all x, y R, if x and y are constants with x < y then 2xn o(2yn) Statement Justification 1. First show that lim ninfinity2yn / 2xn = infinity 2. 2yn / 2xn = 2(y-x)n Math Identity (MI) 3. y - x > 0 and is a constant Stated in theorem as an assumption + MI 4. lim ninfinity2(y - x) * n = infinity (3)+math property (MP) 5. lim ninfinity2yn / 2xn = infinity (2) + (4) + substitution 6. 2xn o(2yn) (5) + definition of o (definition 1.17) Induction Proofs • Some proofs can be done – as shown on the last slide, by using various other theorems, definitions and mathematical identities • others might apply the logic-based strategies of – proof by contradiction or the application of rules of inference • Some proofs can only be solved through induction – This is especially true with statements about an infinite set of objects such as numbers • In induction – we must prove the base case is true – we then assume that the proof is true for the recursive or general case when n = some value k – and use this to prove that the result holds for n = k + 1 Example n >= 0, i=0n i(i+1)/2 = n(n+1)(n+2)/6 Statement Justification 1. The proof is by induction on n 2. The base case is n = 0 3. i=00 i(i+1)/2 = 0 * 1 / 2 = 0 and Math 0*(0+1)(0+2)/6 = 0 4. For n greater than 0, assume the result holds for some k > 0 and < n and prove for n=k+1 5. i=0n-1 i(i+1)/2 = (n-1)(n)(n+1)/6 (4) substituting k+1 6. i=0n i(i+1)/2 = (n-1)(n)(n+1)/6 + n*(n+1)/2 (5) 7. (n-1)(n)(n+1)/6 + n*(n+1)/2 = (n3-n)/6+(n2+n)/2 = (n3 + 3*n2 + 2*n) / 6 = n(n+1)(n+2) / 6 Math 8. Therefore, i=0n i(i+1)/2 = n(n+1)(n+2)/6 (6) + (7) Determining Complexity • In chapter 1, we looked at examples of deriving computational complexity by counting the number of comparisons – We count comparisons and multiply by the number of loop iterations • What about if the code using recursion? – We need to determine the number of recursive calls which isn’t as easy as counting loop iterations • To determine number of recursive calls, we create a recurrence equation for the recursive code – The recurrence equation defines a function that can be used to derive the number of recursive calls and therefore the complexity of the code Example seqSearchRec(E, index, num, K) • Recall the sequential search 1. if (index >= num) algorithm from chapter 1, here we solve it through recursion 2. ans = -1; • Counting only comparisons and 3. else if (E[index] == K) ignoring the base case we have 1 4. ans = index; comparison in instruction 3 which is performed during every recursive 5. else call (excluding the base case) 6. ans = seqSearchRec • How many times is the method (E, index+1, num, K); called recursively? 7. return ans; • We develop a recurrence relation: – T(n) = (1 + max(0, T(n-1)) or The bounding function can be – T(n) = T(n-1) + 1 Determined by noticing that – But what is the complexity of this T(n) = T(n-1) + 1 = T(n-2) + 2 algorithm? We must find a = T(n-3) + 3 = … = T(0) + n function that bounds T(n) Since T(0) = 0, T(n) (n) Example • Recall binary search (see algorithm 1.4) • How much work does it do? – If we start with n=last – first + 1, then either we have reached the base case (first > last), or we have found the item at mid = (first + last) / 2, or we have to search either first .. mid – 1 or mid + 1 .. last – The amount of work done recursively is then either the base case, the item is found, or a search over n/2 – 1 items – Thus, the recurrence relation is T(n) = T(n/2 – 1) + 1 – For convenience, we will round this off to T(n) = T(n/2) + 1 – T(n) = T(n/2) + 1 = T(n/4) + 2 = T(n/8) + 3 = … = T(n/2 d) + d – T(1) = 0 and n/ 2d = 1 when n = 2d, or d = log n – Therefore, we have T(1) + d = 0 + log n so T(n) (log n) Example • Consider a situation where two recursive calls are made, each of which takes half the amount of work (mergesort is an example like this) • For an example, consider that each recursive call does 1 comparison and then calls both the left-side and the right-side of the remaining data – T(n) = T(n/2) + T(n/2) + 1 – T(n) = 2 * T(n/2) + 1 = 4 * T(n/4) + 3 = 8 * T(n/8) + 7 = … = 2d * T(n/2d) + 2d - 1 = 2d * T(1) + 2d - 1 – Again, d = log n and T(1) = 0 – T(n) = 2d * 0 + 2d -1 = 0 + n – 1 so T(n) (n) • Does this make sense that a recursive method that divides the data into two sets will have a complexity of (n)? A Variation • From our previous example, what would happen if the data is not divided exactly in half each iteration? – For instance, imagine that we pick a point 2/3 over so that we have recursive calls over 1/3 – 1 and 2/3 – 1 of the data? – Assume the “split point” is r where 0<r<n • T(n) = T(n – 1 – r) + T(r) + 1 (with T(0) = 1) – How do we solve this? • By substitution, this solves to be T(n) = 2*n+1 (n) for any r in the range 0<r<n – We will need this result when we visit the quicksort algorithm Common Recurrences • With recursion we generally see one of four behaviors: – We divide the space into a subspace to examine during the next recursive call • T(n) = T(n/2) + x with T(1) = 0 • This is the case with binary search – We divide the space into a subspace and examine each subspace recursively • T(n) = T(n/2) + T(n/2) + x with T(1) = 0 – We subtract some amount from the space and examine the remainder in the next recursive call • T(n) = T(n-c) + x with T(0) = 1 – We subtract some amount and then have to perform more than one recursive call • T(n) = T(n-c) + T(n-c) + x with T(0) = 1 – The first two are known as “divide and conquer”, the third is “chip and conquer”, the fourth is “chip and be conquered” More Generally… • We will combine cases 1 and 2 to be: – T(n) = b * T(n/c) + f(n) • b is the number of times the method must be called recursively each time • b = 1 for binary search, b = 2 for mergesort • c is the amount we divide the search space by each time • f(n) is the amount of work for all recursive calls of size n/c • The third case is – T(n) = T(n - c) + f(n) • the sequential search examined previously is an example of this • The fourth case is – T(n) = b * T(n – c) + f(n) • if b > 1 then we will wind up with a very large complexity • if c is not the same for each recursive call, we may still be able to identify the complexity easily enough if each c falls within a range Recursion Trees • To better visualize what goes on with the recursive method, we can draw a recursion tree – The recursion tree denotes a node which contains the current size and the non-recursive cost for that node – For instance, T(n) = T(n/2) + T(n/2) + n is given to the below – Notice that in each level, the non-recursive cost is n – How many levels are there? Since we divide n by 2, we can do this log n times, or log n levels • Assume n = 8, then T(n) = n + 2 * (n / 2) + 4 * (n / 4) = 3 * n or n log n – T(n) (n log n) Deriving the Complexity • Now let’s determine how to compute the complexity given a recurrence relation • The general form of our recurrence relation is one of – T(n) = b * T(n / c) + f(n) – T(n) = b * T(n – c) + f(n) • Each of these will have a different way to calculate the complexity, we start with the first one – Compute E = log b / log c (E is the critical exponent) – The depth of our recursion tree is D = log n / log c – The zeroth row has a non-recursive cost of f(n) – The Dth row has nE leaves – With a base cost of 1, the Dth row requires (nE) – What is the non-recursive cost of the in-between rows? Little Master Theorem • We use this theorem to tell us the complexity of the algorithm given the non-recursive work – T(n) (nE) if the non-recursive work increases in a geometric series • This is because the lowest level dominates all other levels and there are nE nodes in the lowest level – T(n) (f(n) log n)) if the non-recursive work stays roughly constant per level • This is because there are log n levels, each of which does roughly the same amount of work, f(n) – T(n) (f(n)) if the non-recursive work decreases in a geometric series per level • This is because the top level dominates in terms of the amount of work, and the amount of work at the top level is f(n) Master Theorem • A more precise way to state the conclusions of the Little Master Theorem is given in the Master Theorem – Assume a recurrence equation of T(n) = b * T (n / c) + f(n) where E = log b / log c – Then: • If f(n) O(nE-e) for some positive e, then T(n) (nE), which is proportional to the number of leaves in the recursion tree • If f(n) (nE), then T(n) (f(n) * log n) as all node depths contribute equally • If f(n) (nE+e) for some positive e and f(n) T(n) O(nE+d) for some d > e, then T(n) (f(n)) which is proportional to the nonrecursive cost of the root node of the recursion tree Examples • T(n) = 3 * T(n / 4) + n – E = log 3 / log 4 < 1 – f(n) = n which is in O(nE+e) (since E < 1, some positive e must be added to E so that f(n) is in O(n1) so the third part of the Master Theorem applies – Thus, T(n) (f(n)) or T(n) (n) • Does this make sense? Yes, each level requires n operations, and there are log4n levels with n decreasing each time, so the most work is done at the root and so the work done at the root bounds the overall process • T(n) = 2 * T(n / 2) + log n – E = log 2 / log 2 = 1 – f(n) = log n • In this case, Master Theorem part 1 applies since log n O(nE-e), so T(n) (n) the most work occurs at the leaf level – there will be n occurrences of (1) which is (n) at this level Chip Solutions • For the chip and conquer/be conquered, we need a different approach – Recall T(n) = b * T(n – c) + f(n) – Assume d is the depth of the tree (d = n / c or approximately that depending on whether c is uniform) – T(n) = b n/c h=0n/c f (c * h) / bh • where h = (n / c) – d • h = 0 at the leaves, and increases toward the root – T(n) (b n/c ) unless b = 1, then we have one of: • T(n) (na+1) if f(n) is in the polynomial na, or • T(n) (n log n) if f(n) is in log n – Examples: • T(n) = 2 * T(n – 1) + 1 results in T(n) (2n/1) or (2n) • T(n) = T(n – 5) + log n results in T(n) (n log n) • T(n) = T(n – 1) + n2 results in T(n) (n3)

DOCUMENT INFO

Shared By:

Categories:

Tags:
how to, final exam, Chapter 4, Higher Mathematics, midterm exam, Office Hours, Propositional Logic, proof terms, writing style, proof technique

Stats:

views: | 3 |

posted: | 10/4/2010 |

language: | English |

pages: | 18 |

OTHER DOCS BY chenshu

How are you planning on using Docstoc?
BUSINESS
PERSONAL

Feel free to Contact Us with any questions you might have.