Greedy algorithm Greedy Algorithms are algorithms which follow the problem solving meta-heuristic of making the locally optimum choice at each stage with the hope of finding the global optimum. For instance, applying the greedy strategy to the traveling salesman problem yields the following algorithm: "At each stage visit the unvisited city nearest to the current city". In general, greedy algorithms have five pillars: 1. A candidate set, from which a solution is created 2. A selection function, which chooses the best candidate to be added to the solution 3. A feasibility function, that is used to determine if a candidate can be used to contribute to a solution 4. An objective function, which assigns a value to a solution, or a partial solution, and 5. A solution function, which will indicate when we have discovered a complete solution There are two ingredients that are exhibited by most problems that lend themselves to a greedy strategy: 1) Greedy Choice Property: We can make whatever choice seems best at the moment and then solve the subproblems arising after the choice is made. The choice made by a greedy algorithm may depend on choices so far. But, it cannot depend on any future choices or all the solutions to the subproblem, it progresses in a fashion making one greedy choice after another iteratively reducing each given problem into a smaller one. This is the main difference between it (greedy algorithm) and dynamic programming. Dynamic programming is exhaustive and is guaranteed to find the solution. After every algorithmic stage, dynamic programming
ref:mna/ku/aa/greedyalgo 1
makes decisions based on the all the decisions made in the previous stage, and may reconsider the previous stage's algorithmic path to solution. Greedy algorithms make the decision early and change the algorithmic path after decision, and will never reconsider the old decisions. It may not be accurate for some problems. 2) Optimal Sub structure: A problem exhibits optimal substructure, if an optimal solution to the sub-problem contains within its optimal solution to the problem. Applications For most problems, greedy algorithms do not consistently find the globally optimal solution, because they usually do not operate exhaustively on all the data. They can make commitments to certain choices too early which prevent them from finding the best overall solution later. For example, all known greedy algorithms for the graph coloring problem and all other NP-complete problems do not consistently find optimum solutions. Nevertheless, they are useful because they are quick to think up and often give good approximations to the optimum. If a greedy algorithm can be proven to yield the global optimum for a given problem class, it typically becomes the method of choice because it is faster than other optimisation methods like dynamic programming. Examples of such greedy algorithms are Kruskal's algorithm, Dijkstra's algorithms for finding Single-Source Shortest paths and Prim's algorithm for finding minimum spanning trees and the algorithm for finding optimum Huffman trees. The theory of matroids, as well as the even more general theory of greedoids, provides whole classes of such algorithms.
ref:mna/ku/aa/greedyalgo 2
Summary (Greedy Algorithm) An algorithm that always takes the best immediate, or local, solution while finding an answer. Greedy algorithms find the overall, or globally, optimal solution for some optimization problems, but may find less-than-optimal solutions for some instances of other problems. If there is no greedy algorithm that always finds the optimal solution for a problem, one may have to search (exponentially) many possible solutions to find the optimum. Greedy algorithms are usually quicker, since they don't consider the details of possible alternatives Greedy Examples • Kruskal's algorithms • Prim's algorithm • Longest Consecutive Subsequence • Coin Change Problem • Dijkstra's Algorithm • Job Scheduling Problem • As an Approximation Algorithm A Simple Scheduling Problem We are given jobs j1,j2,…,jn, all with know running times t1,t2,…,tn, respectively. We have a single processor. What is the best way to schedule these jobs in order to minimize the average completion time? [Here we assume: nonpreemptive scheduling: Once a job is started, it must run to completion.] As an example, suppose we have the four jobs and associated running times as shown in table below.
ref:mna/ku/aa/greedyalgo 3
Job j1 j2 j3 j4
Time 15 8 3 10
Jobs and Times
One possible schedule is shown below. j1 j2 j3 0 15 23 26 36
Schedule # 1
j4
Because j1 finishes in 15 (time units), j2 in 23, j3 in 26, and j4 in 36, the average completion time is 25. A better schedule, which yields a mean completion time of 17.75, is shown below. j3 j2 j4 j1 0 3 11 21 36
Schedule # 2
The schedule # 2 given above is arranged by shortest job first. We can show that this will always yield an optimal schedule. Let the jobs in the schedule be ji1,ji2,…,jin. The first job finishes in time ti1. The second job finishes in after ti1 + ti2, and the third job finishes after ti1 + ti2 + ti3. From this, we see that the total cost, C of the schedule is
C = ∑(N – k + 1)tik
k=1
n
(A)
C equation, the first ∑k.t (B) Notice that in= (n + 1)∑tik - sum isik independent of the job ordering, so only the second k=1 affects the total cost. Suppose that sum k=1 in an ordering there exists some x > y such that tix < tiy. Then a calculation shows that by swapping j ix and jiy, the second sum increases, decreasing the total cost. Thus, any schedule of jobs in
ref:mna/ku/aa/greedyalgo 4
n
n
which the times are not monotonically nondecreasing must be suboptimal. The only schedules left are those in which the jobs are arranged by smallest running time first, breaking ties arbitrarily. This result indicates that the reason the operating system scheduler generally gives precedence to shorter jobs.
ref:mna/ku/aa/greedyalgo
5