# Greedy Approach by shuifanglj

VIEWS: 23 PAGES: 35

• pg 1
```									Greedy Algorithms (I)

Greed, for lack of a better word, is good!
Greed is right! Greed works!
- Michael Douglas, U.S. actor in the role of Gordon Gecko,
in the film Wall Street, 1987

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Main topics
   Idea of the greedy approach
   Change-making problem
   Minimum spanning tree problem
   Prim’s algorithm
   Kruskal’s algorithm
   Properties of minimum spanning tree
   Bottleneck spanning tree (additive part)

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Expected Outcomes
   Student should be able to
   summarize the idea of the greedy technique
   list several applications of the greedy technique
   apply the greedy technique to change-making problem
   define the minimum spanning tree problem
   describe ideas of Prim and Kruskal’s algorithms for
computing minimum spanning tree
   prove the correctness of the two algorithms
   analyze the time complexities of the two algorithms under
different data structures
   prove the various properties of minimum spanning tree

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Glossary
   Greedy: 贪婪的
   Change-making problem: 找零钱问题
   Cashier: 收银员
   Denomination:货币单位，面额
   Quarter, dime, nickel, penny: 2角5分，1
角， 5分，1分（美国硬币单位）
   Irrevocable：不可取消的，不可逆的
   Spanning tree: 生成树，支撑树
Copyright  Li Zimao @ 2007-2008-1 SCUEC
Anticipatory Set:
Change Making Problem
   How to make 48 cents of change using coins of
denominations of 25 (1 quarter coin), 10 (1
dime coin), 5 (1 nickel coin), and 1 (1 penny
coin) so that the total number of coins is the
smallest?
   The idea:
   Take the coin with largest denomination without
exceeding the remaining amount of cents,
   make the locally best choice at each step.
   Is the solution optimal? Yes, the proof is left as
exercise.
Copyright  Li Zimao @ 2007-2008-1 SCUEC
General Change-Making Problem

Given unlimited amounts of coins of denominations d1 > … > dm ,
give change for amount n with the least number of coins.

Does the greedy algorithm always give an optimal solution for the general
change-making problem? We give the following example,
Example: d1 = 7c, d2 =5c, d3 = 1c, and n = 10c, not always produces an
optimal solution. in fact, for some instances, the problem may not have a
solution at all! consider instance d1 = 7c, d2 =5c, d3 = 3c, and n = 11c.

But, this problem can be solved by dynamic programming, please try to
design an algorithm for the general change-making problem after class.
Copyright  Li Zimao @ 2007-2008-1 SCUEC
Greedy Algorithms
   A greedy algorithm makes a locally optimal choice step by step in
the hope that this choice will lead to a globally optimal solution.
The choice made at each step must be:
  Feasible
   Satisfy the problem’s constraints
   locally optimal
   Be the best local choice among all feasible choices
   Irrevocable
   Once made, the choice can’t be changed on subsequent steps.

   Do greedy algorithms always yield optimal solutions?
   Example: change making problem with a denomination set of 7, 5
and 1, and n =10?

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Applications of the Greedy Strategy
For some problems, yields an optimal solution for every
instance. For most, does not but can be useful for fast
approximations.
 Optimal solutions:
   some instances of change making
   Minimum Spanning Tree (MST)
   Single-source shortest paths
   Huffman codes
   Approximations:
   Traveling Salesman Problem (TSP)
   Knapsack problem
   other optimization problems
Copyright  Li Zimao @ 2007-2008-1 SCUEC
Minimum Spanning Tree (MST)
   Spanning tree of a connected graph G is a connected acyclic
subgraph (tree) of G that includes all of G’s vertices.
   Note: a spanning tree with n vertices has exactly n-1 edges.
   Minimum Spanning Tree of a weighted, connected graph G is a
spanning tree of G of minimum total weight.

   Example:

Copyright  Li Zimao @ 2007-2008-1 SCUEC
MST Problem
   Given a connected, undirected, weighted
graph G= (V, E), find a minimum spanning
tree for it.
   Compute MST through Brute Force?
•   Brute force
•   Generate all possible spanning trees for the given graph.
•   Find the one with minimum total weight.
•   Feasibility of Brute force
•   Possible too many trees (exponential for dense graphs)

   Kruskal: 1956, Prim: 1957
Copyright  Li Zimao @ 2007-2008-1 SCUEC
The Prim’s Algorithm
   Idea of the Prim’s algorithm
   Pseudo-code of the algorithm
   Correctness of the algorithm (important)
   Time complexity of the algorithm

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Idea of Prim
   Grow a single tree by repeatedly adding the
least cost edge (greedy step) that connects a
vertex in the existing tree to a vertex not in
the existing tree
   Intermediate solution is always a subtree of
some minimum spanning tree.

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Prim’s MST algorithm

   “Grow” tree one vertex/edge at a time
Construct a series of expanding subtrees T1, T2, … Tn-1. .At
each stage
construct Ti from Ti-1 by
 adding the minimum weight edge that connecting a
vertex in tree (Ti-1) to a vertex not yet in the tree
   this is the “greedy” step!
   Algorithm stops when all vertices are included
Copyright  Li Zimao @ 2007-2008-1 SCUEC
Pseudocode of the Prim
ALGORITHM Prim(G)
//Prim’s algorithm for constructing a minimum spanning tree
//Input: A weighted connected graph G = (V, E)
//Output: ET, the set of edges composing a minimum spanning tree of G
VT  {v0} //v0 can be arbitrarily selected
ET  
for i 1 to |V|-1 do
find a minimum-weight edge e* = (v*, u*) among all the edges (v, u) such
that v is in VT and u is in V-VT
VT  VT {u*}
ET  ET {e*}
return ET

Copyright  Li Zimao @ 2007-2008-1 SCUEC
An example
5
a                    b
1              4         6                2

3                            7
c                       d                          e

Copyright  Li Zimao @ 2007-2008-1 SCUEC
The Prim’s algorithm is greedy!
   The choice of edges added to current subtree
satisfying the three properties of greedy
algorithms.
   Feasible, each edge added to the tree does not
result in a cycle, guarantee that the final ET is a
spanning tree
   Local optimal, each edge selected to the tree is
always the one with minimum weight among all
the edges crossing VT and V-VT
   Irrevocable, once an edge is added to the tree, it
is not removed in subsequent steps.
Copyright  Li Zimao @ 2007-2008-1 SCUEC
Correctness of Prim
   Prove by induction that this construction process actually
yields MST.
   T0 is a subset of all MSTs
   Suppose that Ti-1 is a subset of some MST T, we should prove that Ti
which is generated from Ti-1 is also a subset of some MST.
   By contradiction, assume that Ti does not belong to any MST.
   Let ei = (u, v) be the minimum weight edge from a vertex in Ti-1 to a
vertex not in Ti-1 used by Prim’s algorithm to expanding Ti-1 to Ti ,
according to our assumption, ei can not belong to MST T.
   Adding ei to T results in a cycle containing another edge e’ = (u’, v’)
connecting a vertex u’ in Ti-1 to a vertex v’ not in it, and w(e’)  w(ei)
according to the greedy Prim’s algorithm.
   Removing e’ from T and adding ei to T results in another spanning tree T’
with weight w(T’)  w(T), indicating that T’ is a minimum spanning tree
including Ti which contradict to assumption that Ti does not belong to any
MST.
Copyright  Li Zimao @ 2007-2008-1 SCUEC
Correctness of Prim

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Implementation of Prim
   How to implement the steps in the Prim’s
algorithm?
   First idea, label each vertex with either 0 or 1, 1
represents the vertex in VT, and 0 otherwise.
   Traverse the edge set to find an minimum weight
edge whose endpoints have different labels.
   Any improvement?

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Notations
   T: the expanding subtree.
   Q: the remaining vertices.

At each stage, the key point of expanding the current subtree T is to
  Determine which vertex in Q is the nearest vertex to T.
   Q can be thought of as a priority queue:
   The key (priority) of each vertex, key[v], means the minimum weight edge from v to
a vertex in T. Key[v] is ∞ if v is not linked to any vertex in T.
   The major operation is to to find and delete the nearest vertex (v, for which key[v] is
the smallest among all the vertices)
   Remove the nearest vertex v from Q and add it and the corresponding edge
to T.
   With the occurrence of that action, the key of v’s neighbors will be changed.

To remember the edges of the MST, an array [] is introduced to record the parent
of each vertex. That is [v] is the vertex in the expanding subtree T that is closest
to v not in T.           Copyright  Li Zimao @ 2007-2008-1 SCUEC
ALGORITHM MST-PRIM( G, w, r ) //w: weight; r: root, the starting vertex
1.    for each u  V[G]
2.      do key[u]  
3.         [u]  NIL // [u] : the parent of u
4.    key[r]  0
5.    Q  V[G]           //Now the priority queue, Q, has been built.
6.    while Q  
7.      do u  Extract-Min(Q) //remove the nearest vertex from Q
8.         for each v  Adj[u] // update the key for each of v’s adjacent nodes.
9.           do if v  Q and w(u,v) < key[v]
10.             then [v]  u
11.                  key[v]  w(u,v)
Copyright  Li Zimao @ 2007-2008-1 SCUEC
Copyright  Li Zimao @ 2007-2008-1 SCUEC
Time Complexity of Prim’s algorithm
   Need priority queue for locating the nearest vertex
   Use unordered array to store the priority queue:
Efficiency: Θ(n2)
   Use binary min-heap to store the priority queue
Efficiency: For graph with n vertices and m edges:

O(m log n)
   Use Fibonacci-heap to store the priority queue:
Efficiency: For graph with n vertices and m edges:

O(n log n + m)
Copyright  Li Zimao @ 2007-2008-1 SCUEC
Kruskal’s MST Algorithm
   Edges are initially sorted by increasing weight
   “grow” MST one edge at a time through a series of
expanding forests F1, F2, …, Fn-1
 intermediate stages usually have forest of trees

(not connected)
   at each stage add minimum weight edge among those not
yet used that does not create a cycle
 need efficient way of detecting/avoiding cycles

   algorithm stops when all vertices are included

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Correctness of Kruskal
   Similar to the proof of Prim
   Prove by induction on the construction
process actually generate a MST
   Consider F0, F1, …, Fn-1

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Basic Kruskal’s Algorithm
ALGORITHM Kruscal(G)
//Input: A weighted connected graph G = <V, E>
//Output: ET, the set of edges composing a minimum spanning tree of G.

Sort E in nondecreasing order of the edge weights w(ei1) <= … <= w(ei|E|)
ET  ; ecounter  0 //initialize the set of tree edges and its size
k0
while encounter < |V| - 1 do
kk+1
if ET U {eik} is acyclic
ET  ET U {eik} ; ecounter  ecounter + 1
return ET

Copyright  Li Zimao @ 2007-2008-1 SCUEC

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Copyright  Li Zimao @ 2007-2008-1 SCUEC
Kruskal: Time Complexity
1.    O(EV) when disjoint-set data structure are not used.
2.    When use disjoint-set data structure with union-by-rank and path-
compression heuristics:
a)   Initializing the set A in line 1 takes O(1) time.
b)   O(V) MAKE-SET operations in lines 2-3
c)   Sort the edges in line 4 takes O(ElogE) time.
d)   The for loop of lines 5-8 performs O(E) FIND-SET and UNION
operations on the disjoint-set forest
−    b) and d) take a total of O((V+E)(V))
−    Line 9: O(V+E)
Note that: E  V-1 and (V) = O(logV) = O(logE) and E  V2
SO total time is: O(ElogV)
Copyright  Li Zimao @ 2007-2008-1 SCUEC
Properties of MST
   Property 1:
 Let (u, v) be a minimum-weight edge in a graph G = (V, E),
then (u, v) belongs to some minimum spanning tree of G.
 Proof hint: suppose (u, v) is not in MST T, construct
another MST T’ including (u, v).

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Properties of MST
   Property 2
   A graph has a unique minimum spanning
tree if all the edge weights are pairwise
distinct.
   The converse does not hold.

Copyright  Li Zimao @ 2007-2008-1 SCUEC
Properties of MST
   Property 3
   Let T be a minimum spanning tree of a graph G, and let T’
be an arbitrary spanning tree of G, suppose the edges of
each tree are sorted in non-decreasing order, that is,
w(e1)w(e2) … w(en-1) and w(e1’)w(e2’) … w(en-1’),
then for 1  i  n-1, w(ei) w(ei’).

   Property 4
   Let T be a minimum spanning tree of a graph G, and let L be
the sorted list of the edge weights of T, then for any other
minimum spanning tree T’ of G, the list L is also the sorted
T’.
list of edge weights ofZimao @ 2007-2008-1 SCUEC
Properties of MST
   Property 5
   Let e=(u, v) be a maximum-weight edge on some cycle of
G. Prove that there is a minimum spanning tree that does
not include e.
   Proof. Arbitrarily choose a MST T. If T does not contain e,
it is proved. Otherwise, T\e is disconnected, and suppose X,
Y are the two connected components of T\e. Let e is on
cycle C in G. Let P=C\e. Then there is an edge (x,y) on P
such that xX, and y Y. And w(x, y)  w(e).
   T’=T\e+(x,y) is a spanning tree and w(T’)  w(T).
   Also we have w(T)  w(T’), so w(T’) = w(T).
   T’ is a MST not including e.
Copyright  Li Zimao @ 2007-2008-1 SCUEC
Bottleneck Spanning Tree
   A bottleneck spanning tree T of a connected, weighted and
undirected graph G is a spanning tree of G whose largest
edge weight is minimum over all spanning trees of G.
 Let T1, T2, …, Tm are all the spanning trees of G, and the
largest edge of each tree is et1, et2, …, etm, suppose w(eti)
 w(etj) for 1  j  m and j  i, then Ti is a bottleneck
spanning tree.
 The value of a BST T is the weight of the maximum-
weight edge in T.
 The bottleneck spanning tree may not be unique.

Copyright  Li Zimao @ 2007-2008-1 SCUEC
BST vs MST
   Every minimum spanning tree is a bottleneck spanning tree.
   Property 4 implies it.
   An easier proof: Let T be a MST and T’ be a BST, let the maximum-
weight edge in T and T’ be e and e’, respectively. Suppose for the
contrary that the MST T is not a BST, then we have w(e)>w(e’), which
also indicates that the weight of e is greater than that of any edges in T’.
Removing e from T disconnects T into two subtrees T1 and T2, there
must exist an edge f in T’ connecting T1 and T2, otherwise, T is not
connected. T1T2 {f} forms a new tree T’’ with w(T’’) = w(T) - w(e)
+ w(f) < w(T), A contradiction to the fact that T is MST, thus, A MST
is also a BST.

Copyright  Li Zimao @ 2007-2008-1 SCUEC

```
To top