Approximation Algorithms

							Anany Levitin
ACM SIGCSE 1999
Outline
 Introduction
 Four General Design Techniques
 A Test of Generality
 Further Refinements
 Conclusion
Introduction-1
 According to many textbooks, a consensus seems to
  have evolved as to which approaches qualify as major
  techniques for designing algorithms.
 This is includes: divide-and-conquer, greedy approach,
  dynamic programming, backtracking, and branch-and-
  bound.
Introduction-2
 However, this widely accepted taxonomy has serious
 shortcoming.
   First, it includes techniques of different levels of
    generality. For example, it seems obvious that divide-
    and-conquer is more general than greedy approach and
    branch-and-bound.
   Second, it fails to distinguish divide-and-conquer and
    decrease-and-conquer.
   Third, it fails to include brute force and transform-and-
    conquer.
Introduction-3
   Fourth, its linear, as opposed to hierarchical, structure
    fails to reflect important special cases of techniques.
   Finally, it fails to classify many classical algorithms (e.g.,
    Euclid’s algorithm, heapsort, search trees, hashing, etc.)
 This paper seeks to rectify these shortcomings and
 presents new taxonomy.
Four General Design Techniques-1
 Brute Force
    It usually based on the problem’s statement and
     definitions of concepts involved
    This technique can not be overlooked by the following
     reasons
        Applicable to a very wide variety of problems, e.g., computing
         the sum of n numbers, adding two matrices ...
        Useful for solving small-size instances of a problem
        Serving an important theoretical or educational purpose, e.g.,
         NP-hard problem, as a yardstick for more efficient alternatives
         for solving a problem
Four General Design Techniques-2
 Divide-and-conquer
   It based on partitioning a problem into a number of
    smaller subproblems, usually of the same kind and
    ideally of about the same size
   The subproblems are then solved and their solutions
    combined to get a solution to the original problem.
       Divide-before-processing: the bulk of the work is done while
        combining solutions to smaller subproblems, e.g., mergesort.
       Process-before-dividing: processing before a partition into
        subproblems, e.g., quicksort.
Four General Design Techniques-3
 Decrease-and-conquer
   This technique is solving a problem by reducing its
    instance to a smaller one, solving the latter, and then
    extending the obtained solution to get a solution to the
    original instance
        decrease by a constant: insertion sort
        decrease by a constant factor(a.k.a. prune-and-search): binary
         search
        variable size decrease: Euclid’s algorithm
Four General Design Techniques-4
 Transform-and-conquer
    This technique is based on the idea of transformation
        simplification: solves a problem by first transforming its
         instance to another instance of the same problem which
         makes the problem easier to solve, e.g., Gaussian elimination,
         heapsort ...
        representation change: it is based on a transformation of a
         problem’s input to a different representation, e.g., hashing,
         heapsort …
Four General Design Techniques-4 cont.
      preprossing: The idea is to process a part of the input or the
       entire input to get some auxiliary information which speeds
       up solving the problem, e.g., KMP algorithms.
      reduction: An instance of a problem is transformed to an
       instance of a different problem altogether, e.g, NP-hard
       problems.
A Test of Generality-1
 We partition design techniques into two categories:
  more general and less general techniques.
 How to make such a determinate ? We would like to
  suggest the following test. In order to qualify for
  inclusion in the category of most general approaches, a
  technique must yield reasonable algorithms for the two
  problems: sorting and searching.
A Test of Generality-2
                        Sorting           Searching
Brute force             Selection sort    Sequential search
Divide-and-conquer      Mergesort         Applicable
Decrease-and-conquer    Insertion sort    Applicable
Transform-and-conquer   Heapsort          Search trees, hashing


The others - greedy approach, dynamic programming,
backtracking, and branch-and-bound - fail to qualify as the
most general design techniques.
Further Refinements-1
 Local search techniques
    Greedy methods: it builds solutions piece by piece … the
     choice selected is that which produces the largest
     immediate gain while maintaining feasibility, e.g., Prim’s
     algorithm.
    Iterative methods: it start with any feasible solution and
     proceed to improve upon the solution by repeated
     applications of a simple step, e.g., Ford-Fulkerson
     algorithm.
Further Refinements-2
 Dynamic programming
   Bottom-up: a table of solutions to subproblems is filled
    starting with the problem’s smallest subproblems. A
    solution to the original instance of the problem is then
    obtained from the table constructed.
   Top-down: memory function
Further Refinements-3
 State-space-tree techniques
    Backtracking: take coloring problem as an example:



        1         2       Use three colors to color the vertices of
                          this graph. How many different ways ?



        4         3
Further Refinements-3 cont.
                     S


V1       1           2       3



V2           1       2   3




V3   1       2   3
Further Refinements-3 cont.
  Branch-and-Bound: take TSP problem as an example. If
  there are four cities, all feasible solutions are
Further Refinements-3 cont.
  Consider the traveling costs between any two cities:




                                       =3+3+5+4+1
Further Refinements-3 cont.
   Start to branch
   Choose 12




 Otherwise
Further Refinements-3 cont.
  The current branch:
Further Refinements-3 cont.
 The difference between them lies in that backtracking
 is not limited to optimization problems, while branch-
 and-bound is not restricted to a specific way of
 traversing the problem’s space tree.
Conclusion
 This paper gives a new taxonomy of algorithm design
  techniques.
 No matter how many general design techniques are
  recognized, there will always be algorithms that cannot be
  naturally interpreted as an application of those techniques.
 Some algorithms can be interpreted as an application of
  different techniques, e.g., selection sort, as a brute-force
  algorithm and as a decrease-and-conquer method.
 Some algorithms may incorporate ideas of several
  techniques, e.g. Fourier transform takes advantage of both
  the transform and divide-and-conquer ideas.