Personal Information

							Master Theorem -- August 23, 2007                                                    Page: 1


Personal Information
Charles O. Shields, Jr.
ECCS 4.226
cshields@utdallas.edu

Objective Today:
Objective today is 3-fold:
1)     review the Master Theorem and look at some examples
2)     give some hints on a couple of the homework problems, 4-2 and 4-6
3)     If time allows, look into some further Divide and Conquer algorithms:
       a)      fast multiplication

The Master Theorem (Introduction)
     Rivest, "Intro to Algorithms", p. 73

        The Master Theorem is a tool that can be used to solve recurrences of the form
T(n) = aT(n/b) + f(n), where a1 and b>1 are constants and f(n) is an asymptotically
positive function.
        Basically we have:

1)        a problem of size n, which is divided into "a" subproblems of size n/b.
2)        each subproblem of size n/b can be solved recursively in time T(n/b).
3)        the cost of dividing the problem and combining the results is described by f(n).

The Master Theorem
       Let a1 and b>1 be constants, f(n) be a function, and T(n) be defined on the non-
negative integers by the recurrence:

       T(n) = aT(n/b) + f(n). Then T(n) can be bounded asymptotically by the
following three cases:

Case 1:      if f(n) = O(nlog b a – e) for some constant e>0, then T(n) = (nlog b a).
Case 2:      if f(n) = (nlog b a), then T(n) = (nlog b a log n).
Case 3:      if f(n) = (nlog b a + e) for some constant e>0, and if a*f(n/b)  c*f(n) for
             some constant c<1 and large n (i.e. the “Regularity Condition”), then T(n) =
             (f(n)).

        To determine in which case a given recurrence fits, we compare f(n) to the
function nlog b a. The case is determined by which of the two functions is asymptotically
greater. Cases 1 and 3 are either (nlog b a) or (f(n)), depending on which is larger. If
the two functions have the same growth rate, we multiply by a logarithmic factor, and the
final solution is:
Master Theorem -- August 23, 2007                                                         Page: 2


                         T(n)    = (nlog b a log n) = (f(n) * log n)

Notes:

          Why wasn’t Case 1 expressed as f(n) = o(nlog b a)?
          Note that the asymptotic differences between n log b a and f(n) mentioned in Cases
(1) and (3) must be polynomial. That is, f(n) must be either (a) asymptotically less than
nlog b a by a factor of n for some >0 (for Case 1), or (b) asymptotically greater than nlog b a
by a factor of n (for Case 3). If this is not the case, then the Master Theorem may not be
applicable.

         Note also that the Regularity Condition in Case 3 holds automatically whenever
f(n) is a polynomial. Therefore, when f(n) is a polynomial, the Regularity Condition does
not need to be checked.
         Here is a brief proof:

Proposition: If f(n) = (nlog b a + ) for some constant >0, and if f(n) = nk for some k0,
             then a*f(n/b)  c*f(n) for some c<0.

Proof:
        Since f(n) = (nlog b a + ) for some constant >0, and since f(n) = nk, we know that
k>logba. Treating both sides as an exponent, we have: bk>blog b a => bk > a. Therefore,
1>a/bk.
        Let c = a/bk. We then have: a*f(n/b) = a*(n/b)k = (a/bk )(nk ) = c*nk , and therefore,
the regularity condition is satisfied.

Examples

         Let’s work a few examples. We’ll start with a couple from Problem 4-4 on page
86.

4-4(a)
         T(n)   = 3T(n/2) + nlgn.

        Here, f(n) = nlgn and nlog b a = nlg 3 ~= n1.585 . Since nlgn = O(nlg 3 - ) for any 0 < 
< 0.58, we have T(n) = (nlg 3) by Case 1 of the MT.

         What if this were T(n) = 2T(n/2) + nlgn? Would the MT apply? No, falls into the
gap between Cases 2 and 3. There is no  such that ne is asymptotically less than lg n. In
fact, the opposite applies for any .

4-4(b)
         T(n)   = 4T(n/2) + n2n
Master Theorem -- August 23, 2007                                                        Page: 3


        Here, f(n) = n2n = n2.5 and nlog b a = n2. Since n2.5 = (n2 + ) (for <0.5), Case 3
of the MT applies. Since f(n) is a polynomial, we don’t have to check the regularity
condition. Therefore, T(n) = (n2n).

Practical Example: Maximum Subsequence Sum Problem

Maximum Subsequence Sum:
        Given (possible negative) integers A1, A2, ..., AN, find the maximum value of
k=ijAk. That is, find the maximum sum for any consecutive subsequence of integers
from A.

Example: Input: A = -2, 11, -4, 13, -5, -2

        Here, the maximum subsequence sum is 20, derived by summing A 2 through A4 .

        There are many algorithms to solve this problem, and indeed, the data structures
book has examples ranging from (n) to (n3). There is, however, an (n lg n)
algorithm that uses divide and conquer.

          Note that if we divide the sequence A in half, the max sub sum can be found in
one of three locations: (a) completely in the left half, (b) completely in the right half, or
(c) it straddles the two halves. The last possibility can be calculated in linear time by
calculating (i) the max sum on the left half that uses the last element of the left half, and
(ii) the max sum on the right half that uses the first element of the right half, and (iii)
adding these two together. This result will form the max sub sum that spans the center.
The overall result is then the maximum of (a), (b), and (c).




        Given this analysis, the recurrence relation that describes this algorithm is:

        T(n)    = 2T(n/2) + (n)

       Solving this with the MT, we see that T(n) falls into Case 2, and therefore, T(n) =
(n lg n).


Other examples of Master Theorem
Master Theorem -- August 23, 2007                                                     Page: 4


        T(n) = 9T(n/3) + n. Here a=9, b=3, f(n) = n, log39 =2. Note that case 1 applies,
f(n) = n = O(nlog b a - e) = O(n1.9), where e=0.1. (Note that n  (n2)). So T(n) = (nlog 3 9)
= (n2).
        T(n) = T(2n/3) + 1. a=1, b=3/2, f(n) = n0 = 1, and nlog b a = n^{log 3/21} = n0 = 1.
Case 2 applies, since f(n) = (1). So T(n) = (nlog b a log n) = (log n).
        T(n) = 2T(n/2) + n2. Then a=2, b=2, f(n)=n2, and nlog b a = n1 . Then n2 = (n1.1),
where e=0.1. Also note that 2f(n/2) = 1/2n2  c*n2  c>1/2, so the regularity condition
applies. Therefore Case 3 applies, and T(n) = (n2).

Some suggestions for the Homework

4-2    Finding the missing integer

        Here we are given an array A[1...n] that contains all of the integers between 0 and
n except one. We have to determine which integer is missing in O(n) time, when the only
operation available returns but a single bit from some element in the array.
        We could, of course, scan through the array, starting with the lowest order bit and
proceeding to higher order bits, to determine which number is missing. But since it takes
O(n) time to make a single scan, and since the number of scans is O(lg n), which is the
number of bits in n, the total time complexity of this procedure would be O(n lg n). This
is too great for our purposes.
        Divide and conquer can help us reduce the time complexity, but we must find
some way to eliminate some of the numbers from subsequent scans. How can we do this
when we can only look at one bit at a time?
        Let’s begin by collecting more information about the problem. For hints, let’s
look at and consider the following:

1)     Let n equal an even number, say n=6, and see what the binary representation of
       the numbers from 0 to 6 are.
2)     Do the same thing for an odd number, like n=5.

Say n=6:

0          000
1          001
2          010
3          011
4          100
5          101
6          110

3)     How can we use the bit-return operation to determine whether a number is even or
       odd?
4)     If n is even, how many even numbers should there be (if all numbers were there)?
       How many odd numbers? What about if n is odd?
Master Theorem -- August 23, 2007                                                    Page: 5


5)       Can we determine if the missing number is even or odd?
6)       What would the recurrence be?

      It is possible to create a table of values that can be used to determine if an even or
odd number is missing, etc.

                                   Total               Missing even        Missing odd
n even          Even               n/2 + 1             n/2                 n/2 + 1
                Odd                n/2                 n/2                 n/2 - 1
n odd           Even               n/2                 etc
                Odd                n/2


Problem 4.6 (b), p. 87

        Here we have a rig that is designed to hold two VLSI chips. The chips test each
other and return a report according to the following rules:
1)      if chip A is good, it will report the truth about chip B
2)      if chip A is bad, it may report the truth about B or it may not. That is, it can
        return either “G” or “B”.

         We have to find a way to reduce the size of the subproblems. Note that:
1)       if both chips are good, a “GG” will be returned.
2)       if at least one chip is bad, it is impossible to get a “GG”.
3)       you can get a “GG” returned if both chips are bad

         Consider an example:

Actual chip status:   G B          G G         B B        B G         G G
Reported chip status: B B          G G         G G        B B         G G
Take example of:                   G           B                      G

        The idea for the algorithm is, first, do a pairwise check, and look for the GG
pattern. If a GG pattern is found, take one of these pairs and place in another set. The
result should be a set that has more G’s than B’s.
        Can there be more B’s than G’s? Don’t think so, due to the majority condition.

         Consider the case when there are an odd number of chips:

         G B        G B         G B          B G      G

        Note that if there are an odd number of chips, the last one can be tested against all
others. If the last one is good, then there will be  N/2 G-G’s returned. If the last one is
bad, there must be < N/2 G-G’s returned (since the only way to get a G-G if one is bad is
for them both to be bad). Therefore, it is possible to determine, in an odd case, if the last
Master Theorem -- August 23, 2007                                                      Page: 6


one is good or bad (by counting the number of G-G’s when that chip is run against all
others).

       What is the recurrence for this operation?

       T(n)    = T(n/2) + cn

since the size of the chip set being examined is at least cut in half, and it takes linear time
to do the pairwise search.