Analysis of Algorithms CS 465_665_3_

Document Sample

```					Analysis of Algorithms
CS 477/677

Instructor: Monica Nicolescu
The Heap Data Structure
• Def: A heap is a nearly complete binary tree with
the following two properties:
– Structural property: all levels are full, except
possibly the last one, which is filled from left to right
– Order (heap) property: for any node x
Parent(x) ≥ x

8
It doesn’t matter that 4 in
7           4       level 1 is smaller than 5 in
5       2               level 2
Heap
CS 477/677                         2
Array Representation of Heaps
• A heap can be stored as an
array A.
– Root of tree is A[1]
– Left child of A[i] = A[2i]
– Right child of A[i] = A[2i + 1]
– Parent of A[i] = A[ i/2 ]
– Heapsize[A] ≤ length[A]
• The elements in the subarray
A[(n/2+1) .. n] are leaves
• The root is the maximum
element of the heap
A heap is a binary tree that is filled in order
CS 477/677                  3
Maintaining the Heap Property
•   Suppose a node is smaller than a
child
– Left and Right subtrees of i are max-heaps
•   Invariant:
– the heap condition is violated only at that
node
•   To eliminate the violation:
– Exchange with larger child
– Move down the tree
– Continue until node is not smaller than
children

CS 477/677         4
Building a Heap
• Convert an array A[1 … n] into a max-heap (n = length[A])
• The elements in the subarray A[(n/2+1) .. n] are leaves
• Apply MAX-HEAPIFY on elements between 1 and n/2
1

4
2                    3

1                    3
4                5       6           7

8
2   9       10
16 9                10
14       8       7

A:   4   1   3    2   16       9       10 14       8       7

CS 477/677                               5
Heapsort
• Goal:
– Sort an array using heap representations

• Idea:
– Build a max-heap from the array
– Swap the root (the maximum element) with the last
element in the array
– “Discard” this last node by decreasing the heap size
– Call MAX-HEAPIFY on the new root
– Repeat this process until only one node remains

CS 477/677                      6
Alg: HEAPSORT(A)

1. BUILD-MAX-HEAP(A)                  O(n)
2. for i ← length[A] downto 2
3.      do exchange A[1] ↔ A[i]                n-1 times

4.         MAX-HEAPIFY(A, 1, i - 1)   O(lgn)

•    Running time: O(nlgn)

CS 477/677                   7
Example:                              A=[7, 4, 3, 1, 2]

MAX-HEAPIFY(A, 1, 4)   MAX-HEAPIFY(A, 1, 3)   MAX-HEAPIFY(A, 1, 2)

MAX-HEAPIFY(A, 1, 1)

CS 477/677                              8
HEAP-MAXIMUM
Goal:
– Return the largest element of the heap

Running time: O(1)
Alg: HEAP-MAXIMUM(A)
1.      return A[1]
Heap A:

Heap-Maximum(A) returns 7
CS 477/677                       9
HEAP-EXTRACT-MAX
Goal:
–    Extract the largest element of the heap (i.e., return the max
value and also remove that element from the heap
Idea:
–    Exchange the root element with the last
–    Decrease the size of the heap by 1 element
–    Call MAX-HEAPIFY on the new root, on a heap of size n-1

Heap A:                         Root is the largest element

CS 477/677                                 10
HEAP-EXTRACT-MAX
Alg: HEAP-EXTRACT-MAX(A, n)

1. if n < 1
2.     then error “heap underflow”

3.   max ← A[1]

4.   A[1] ← A[n]

5.   MAX-HEAPIFY(A, 1, n-1)               remakes heap

6.   return max
Running time: O(lgn)
CS 477/677                    11
Example: HEAP-EXTRACT-MAX
16                                                          1

14                10       max = 16                         14                   10
8            7    9        3                               8             7       9        3
2       4    1                                         2           4
Heap size decreased with 1

14

Call MAX-HEAPIFY(A, 1, n-1)
8                   10
4           7       9             3
2       1

CS 477/677                                                     12
HEAP-INCREASE-KEY
• Goal:
– Increases the key of an element i in the heap
• Idea:
– Increment the key of A[i] to its new value
– If the max-heap property does not hold anymore:
traverse a path toward the root to find the proper
place for the newly increased key
16

14                10
8 i        7    9        3
Key [i] ← 15 2     4    1

CS 477/677                  13
HEAP-INCREASE-KEY
Alg: HEAP-INCREASE-KEY(A, i, key)

1.   if key < A[i]
2.      then error “new key is smaller than current key”
3.   A[i] ← key
4.   while i > 1 and A[PARENT(i)] < A[i]              16
5.       do exchange A[i] ↔ A[PARENT(i)]
14              10
6.          i ← PARENT(i)
8 i       7   9             3
2     4   1
•    Running time: O(lgn)
Key [i] ← 15

CS 477/677                               14
Example: HEAP-INCREASE-KEY
16                                               16

14                10                             14                10
8 i           7    9        3                    8 i    7           9        3
2     4       1                                  2     15 1

Key [i ] ← 15

16                                               16
i
14                10                             15                10
i
15            7    9        3                    14            7    9        3
2        8    1                                  2        8    1

CS 477/677                                       15
MAX-HEAP-INSERT
• Goal:
16
– Inserts a new element into a max-
heap                                         14                10
8            7    9         3
• Idea:
2       4    1       -
– Expand the max-heap with a new
16
element whose key is -
– Calls HEAP-INCREASE-KEY to                   14                10
set the key of the new node to its       8            7    9         3
correct value and maintain the       2       4    1       15

max-heap property

CS 477/677                                    16
MAX-HEAP-INSERT

16
Alg: MAX-HEAP-INSERT(A, key, n)
14                10
1. heap-size[A] ← n + 1                8            7    9         3
2       4    1       -
2. A[n + 1] ← -

3. HEAP-INCREASE-KEY(A, n + 1, key)

Running time: O(lgn)

CS 477/677                              17
Example: MAX-HEAP-INSERT
Insert value 15:                  Increase the key to 15
- Start by inserting -           Call HEAP-INCREASE-KEY on A[11] = 15
16                                      16

14                   10                    14                     10
8                7    9        3          8              7    9             3
2       4     1          -               2         4    1       15

The restored heap containing

16                                           16

14                10                         15                     10

8                15 9          3                8             14 9               3

2        4       1        7                   2          4    1       7
CS 477/677                                        18
Summary
• We can perform the following operations on
heaps:
– MAX-HEAPIFY                    O(lgn)
– BUILD-MAX-HEAP                 O(n)
– HEAP-SORT                      O(nlgn)
– MAX-HEAP-INSERT                O(lgn)
– HEAP-EXTRACT-MAX               O(lgn)
– HEAP-INCREASE-KEY              O(lgn)
– HEAP-MAXIMUM                   O(1)
CS 477/677               19
The Search Problem
• Find items with keys matching a given search key
Applications:
• Keeping track of customer account information at
a bank
– Search through records to check balances and perform
transactions
• Keep track of reservations on flights
– Search to find empty seats, cancel/modify reservations
• Search engine
– Looks for all documents containing a given word

CS 477/677                     20
Symbol Tables (Dictionaries)
• Dictionary = data structure that supports two
basic operations: insert a new item and return
an item with a given key
• Queries: return information about the set
– Search (S, k)
– Minimum (S), Maximum (S)
– Successor (S, x), Predecessor (S, x)
• Modifying operations: change the set
– Insert (S, k)
– Delete (S, k)
CS 477/677                21
Implementations of Symbol Tables
• Key-indexed-array
– Key values are distinct, small numbers
– Store the items in an array, indexed by keys
• Ordered/unordered arrays
Insert       Search
key-indexed array         1           1
ordered array             N           N
ordered list              N           N
unordered array           1           N
unordered list            1           N
CS 477/677                22
Binary Search Trees
• Support many dynamic set operations
– SEARCH, MINIMUM, MAXIMUM, PREDECESSOR,
SUCCESSOR, INSERT

• Running time of basic operations on binary
search trees
– On average: (lgn)
• The expected height of the tree is lgn

– In the worst case: (n)
• The tree is a linear chain of n nodes

CS 477/677        23
Binary Search Trees
• Tree representation:
– A linked data structure in which
each node is an object
• Node representation:                                parent
L              R
–   Key field                                      key data
–   Satellite data
–   Left: pointer to left child
Left child    Right child
–   Right: pointer to right child
–   p: pointer to parent (p [root [T]] =
NIL)
• Satisfies the binary-search-tree
property
CS 477/677                            24
Binary Search Tree Example

• Binary search tree property:
– If y is in left subtree of x,                     5

then key [y] ≤ key [x]
3           7
2       5           9
– If y is in right subtree of x,
then key [y] ≥ key [x]

CS 477/677                           25
Traversing a Binary Search Tree
•   Inorder tree walk:
– Prints the keys of a binary tree in sorted order
– Root is printed between the values of its left and
right subtrees: left, root, right
•   Preorder tree walk:
– root printed first: root, left, right
•   Postorder tree walk: left, right, root
– root printed last
5            Inorder: 2 3 5 5 7 9
Preorder: 5 3 2 5 7 9
3           7
2       5           9    Postorder: 2 5 3 9 7 5
CS 477/677                  26
Traversing a Binary Search Tree
Alg: INORDER-TREE-WALK(x)
1.    if x  NIL
2.       then INORDER-TREE-WALK ( left [x] )
3.            print key [x]
4.            INORDER-TREE-WALK ( right [x] )
• E.g.:                  5

3           7
Output: 2 3 5 5 7 9
2       5           9

• Running time:
– (n), where n is the size of the tree rooted at x
CS 477/677                         27
Searching for a Key
•   Given a pointer to the root of a tree and a key k:
5
–   Return a pointer to a node with key k
if one exists                                            3           7
–   Otherwise return NIL                                 2       4            9
•   Idea
–   Starting at the root: trace down a path by comparing k with the
key of the current node:
•   If the keys are equal: we have found the key
•   If k < key[x] search in the left subtree of x
•   If k > key[x] search in the right subtree of x

CS 477/677                             28
Searching for a Key
Alg: TREE-SEARCH(x, k)                  5

1. if x = NIL or k = key [x]     3        7

2.      then return x         2     4         9

3. if k < key [x]
4.      then return TREE-SEARCH(left [x], k )
5.      else return TREE-SEARCH(right [x], k )

Running Time: O (h),
h – the height of the tree

CS 477/677             29
Example: TREE-SEARCH

15
• Search for key 13:
6            18                     – 15  6  7  13
3           7 17          20
2       4           13
9

CS 477/677                       30
Iterative Tree Search

Alg: ITERATIVE-TREE-SEARCH(x, k)
1. while x  NIL and k  key [x]
2.     do if k < key [x]
3.         then x  left [x]
4.         else x  right [x]
5. return x

CS 477/677    31
Finding the Minimum in a Binary
Search Tree
•   Goal: find the minimum value in a BST
– Following left child pointers from the root,
until a NIL is encountered                              15
Alg: TREE-MINIMUM(x)
1. while left [x]  NIL
6            18
3           7 17          20
2.       do x ← left [x]                  2       4           13
3. return x                                               9

Minimum = 2

Running time: O(h), h – height of tree

CS 477/677                                  32
Finding the Maximum in a Binary
Search Tree
•   Goal: find the maximum value in a BST
– Following right child pointers from the root,
until a NIL is encountered
15
Alg: TREE-MAXIMUM(x)
1. while right [x]  NIL                               6            18

2.       do x ← right [x]                      3           7 17           20

3. return x
2       4           13
9

Maximum = 20
•   Running time: O(h), h – height of tree

CS 477/677                               33
Successor
Def: successor (x ) = y, such that key [y] is the
smallest key > key [x]            15
• E.g.: successor (15) = 17
successor (13) = 15                    6              18

successor (9) = 13                  3        7 17           20
2      4       13
• Case 1: right (x) is non empty                     9

– successor (x ) = the minimum in right (x)
• Case 2: right (x) is empty
– go up the tree until the current node is a left child:
successor (x ) is the parent of the current node
– if you cannot go further (and you reached the root):
x is the largest element
CS 477/677                             34
Finding the Successor
Alg: TREE-SUCCESSOR(x)
1.   if right [x]  NIL
2.      then return TREE-MINIMUM(right [x])
3.   y ← p[x]                                         15
4.   while y  NIL and x = right [y]                   y
6            18
5.      do x ← y
3        7 17              20
6.         y ← p[y]                                        x
2     4         13
7.   return y                                     9

Running time: O (h), h – height of the tree

CS 477/677                          35
Predecessor
Def: predecessor (x ) = y, such that key [y] is the
biggest key < key [x]
• E.g.: predecessor (15) = 13                      15

predecessor (9) = 7                   6                18
predecessor (13) = 9
3        7 17          20

• Case 1: left (x) is non empty
2        4       13
9
– predecessor (x ) = the maximum in left (x)
• Case 2: left (x) is empty
– go up the tree until the current node is a right child:
predecessor (x ) is the parent of the current node
– if you cannot go further (and you reached the root):
x is the smallest element
CS 477/677                              36
Insertion
• Goal:
– Insert value v into a binary search tree
• Idea:
– If key [x] < v move to the right child of x,
Insert value 13
else move to the left child of x
12
– When x is NIL, we found the correct position
– If v < key [y] insert the new node as y’s left child            5             18
else insert it as y’s right child                     2            9 15          19
1       3         13    17
– Begining at the root, go down the tree and maintain:
• Pointer x : traces the downward path (current node)
• Pointer y : parent of x (“trailing pointer” )

CS 477/677                                      37
Example: TREE-INSERT
x, y=NIL                                                y
Insert 13:                 12                                                        12
x
5              18                                   5                     18
2              9 15             19                     2               9 15               19
1           3                 17                       1        3                     17

12                                                            12

x                  y
5              18                                           5                  18
2            9 15             19                        2                9 15               19
1          3                 17                            1           3        13        17
x = NIL
y = 15
CS 477/677                                                38
Alg: TREE-INSERT(T, z)
1.    y ← NIL
2.    x ← root [T]
3.    while x ≠ NIL                                      12

4.        do y ← x                                  5           18
5.           if key [z] < key [x]
2           9 15         19
6.              then x ← left [x]
1       3       13     17
7.              else x ← right [x]
8.    p[z] ← y
9.    if y = NIL
10.      then root [T] ← z                  Tree T was empty
11.      else if key [z] < key [y]
12.               then left [y] ← z
13.               else right [y] ← z        Running time: O(h)
CS 477/677                                     39
• Chapter 6
• Chapter 12

CS 477/677   40

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 6 posted: 7/11/2011 language: English pages: 40