Document Sample
					                               ANALYSIS OF ALGORITHMS

Binary Search Optimality

Suppose that we are considering a sorted list (in non-decreasing order) of n integers and
the problem is to find x in the list. We will show that Binary Search is optimal (in the
worst-case) among all (deterministic) algorithms that solve this problem.
Suppose that we have an algorithm B that is not Binary Search and is trying to find x.
This algorithm if it is deterministic, can be described using a binary tree. The first node
(the root) represents the first element that x will be compare with, let’s say Ai (not
necessarily the middle element).

A1 A2 ……… Ai ……………… An

Next suppose that x it is not equal to Ai , then if it is a smart algorithm, if x is less that Ai ,
it will look at an element to the left of Ai , let’s say Ad, otherwise it will compare x with
an element to the right of Ai , let’s say Am.

A1 A2 … Ad…… Ai ………… Am…… An

Clearly we can model this algorithm using a binary tree, where the left child of a node Aj,
represents the element of the array to the left of Aj , that we will compare x with if x is
less than Aj. If x is larger than Aj, we will compare x with an element to the right of Aj .

                                                             Level 1
                                 <            >
                            Ad                    Am
                                                             Level 2
                      <              > <           >

                                                               Level 3

                                                           Level 4 = depth

Clearly each node represents an element of the array, but the question is if perhaps there
is one element of the array, let’s say Ak, that is not represented by any node of the binary
tree. The answer is “no”. Why?.
Consider the following input:

x --- A1 A2 … Ad…… Ai ………… Am…x… An , where x appears exactly one time
in the array, at position k. Since the binary tree does not have a node representing Ak, the
output of the algorithm will show that x does not appear, an incorrect answer.

Let N be the number of nodes of the binary tree, clearly N  n.
Next we notice that the worst-case scenario is the length of the longest path between the
root an leave of the binary tree. Clearly this longest path is between the root and a node at


Suppose that n=13 and we are considering algorithm A5, where we jump every 3 places,
starting by comparing x with A1. We can described this algorithm by the following binary



                   A2               A6

                             A5               A9

                                      A8                    A12


       The depth of this binary tree is 7 (seven) thus the worst-case WB (13)=7.

Thus the ideal binary search tree will have the smallest possible depth. Thus the answer is
that this binary tree should have every level full (2level-1 nodes per level) except of course
at Level=depth.

Thus if we have 13 nodes the ideal binary tree should look like

                                                                                When all the levels
                                                                                are full except for
                                                                                the last one
                                                                                (Level=depth) we
                                                                                call this binary tree
                                                                                a balanced binary
Lemma1: A balanced binary tree with n nodes has depth=  log n  + 1  log n.

Since as we showed previously the number of nodes of the tree must at least n (the
number of elements of the array), then for any algorithm B the worst-case,
WB (n)  log n.

Thus if we have an algorithm that can be described by a balance binary tree that has
exactly n nodes, then the worst-case is optimal.

Theorem 1: The corresponding binary tree of Binary Search is a balanced tree with n

Consider the corresponding binary tree for Binary Search on a list of 13 elements:


                         A3                         A10

         A1                            A5             A8              A12

                    A2        A4     A6             A9     A11          A13