Analysis of Algorithms CS 4413_1_

Document Sample
Analysis of Algorithms CS 4413_1_ Powered By Docstoc
					Analysis of Algorithms
       CS 4413


          Briana B. Morrison
          With thanks to Dr. Hung
    Algorithmics: the study of algorithms.
Algorithmics: is the core of computer science.
     Algorithmics: the spirit of computing.
        How important is this course?!



                                             2
                    The Course

   Purpose: a rigorous introduction to the
    design and analysis of algorithms
    –   Not a lab or programming course
    –   Not a math course, either
   But, need know how to program!
   Minimum math knowledge!


                                              3
                 Course Objectives

   Basic design concepts and analysis methods of
    algorithms
   Design methodologies include, but not limited to:
    –   Searching/Sorting
    –   Divide-and-Conquer
    –   Greedy Method
    –   Dynamic Programming
    –   Search and Traversal
    –   Branch-and-Bound

                                                        4
        Programming/project grading

   Elegant solution
   Logical thinking
   Quality of writing and coding
   Concise writing is a virtue.
   Tell the reader only what is important.



                                              5
             Research Project Guidelines


   Research projects should consist of the
    following components:
    –   Define the problem to be solved
    –   Background information
    –   Algorithms and Applications
    –   Implementation
    –   Drawbacks
    –   New applications and discoveries

                                              6
            Research Project Guidelines


   Some examples of research projects:
    –   Dynamic Programming
    –   SNAKE
    –   Marko Chain Monte Carlo (MCMC) (and Image
        analysis)
    –   Wavelet Transforms
    –   Others


                                                    7
From Problems to Programs

    There are many steps involved in writing a
    computer program to solve a given problem.
    The steps go from
    –   1) problem formulation and specification,
    –   2) to design of the solution,
    –   3) to implementation, testing and documentation,
        and finally
    –   4) to evaluation of the solution.

                                                           8
From Problems to Programs …

    Algorithms and Data Structures are the
    building blocks of most computer programs.
    Half the battle is knowing what problem to
    solve.
    Almost any branch of mathematics or
    science can be called into service to help
    model some problem domain.
    –   Examples: problems essentially numerical in
        nature.
                                                      9
Algorithms

 An algorithm is a finite sequence of instructions,
each of which has a clear meaning
and can be performed with a finite amount of effort
in a finite length of time.

   Pseudo-language is used to present algorithms.
   Pseudo-language is a combination of the
    constructs of a programming language together with
    informal English statements.
                                                         10
Algorithms …

    Example 1: A mathematical model can be
    used to help design a traffic light for a
    complicated intersection of roads (figure 1.1).



                        Figure 1.1. An intersection.



                                                       11
Figure 1.2. Graph showing incompatible turns.   12
Algorithms …




Figure 1.3. Table of incompatible turns.
                                           13
Graph Coloring Problem




  Figure 1.4. A graph.
                         14
Heuristic

    An algorithm that quickly produces good but
    not necessarily optimal solutions is called a
    heuristic.
    One reasonable heuristics for graph coloring
    is the so-called “greedy” algorithm.




                                                    15
Greedy Algorithm

    Initially we try to color as many vertices as possible
    with the first color, then as many as possible of the
    uncolored vertices with the second color, and so on.
    To color vertices with a new color, we perform the
    folowing steps:
    1) Select some uncolored vertex and color it with the new color.
    2) Scan the list of uncolored vertices. For each uncolored vertex,
       determine whether it has an edge to any vertex already colored
       with the new color. If there is no such edge, color the present
       vertex with the new color.



                                                                         16
Graph Coloring Problem …




  Figure 1.5. A coloring of the graph of Fig. 1.2.
                                                     17
Pseudo-Language and Stepwise
Refinement

    Once we have an appropriate mathematical
    model for a problem, we can formulate an
    algorithm in terms of that model.
    The initial versions of the algorithm are often
    couched in general statements that will have
    to be refined subsequently into smaller, more
    definite instructions.

                                                       18
 First refinement of greedy algorithm

procedure greedy ( var G: GRAPH; Var newclr: SET );
{ greedy assigns to newclr a set of vertices of G that may be
given the same color} .
begin
    newclr := 0;
       for each uncolored vertex v of G do
       if v is not adjacent to any vertex in newclr then begin
                mark v colored;
                add v to newclr
       end
end; {greedy}                                                19
Refinement of greedy algorithm
procedure greedy ( var G: GRAPH; var newclr: SET );
begin
       newclr := 0;
       for each uncolored vertex v of G do begin
           found := false;
           for each vertex w in newclr do
                 if there is an edge between v and w in G then
                         found := true;
           if found = false then begin
                { v is adjacent to no vertex in newclr }
                mark v colored;
                add v to newclr
          end
      end                                                      20
end; {greedy}
 Refined greedy algorithm
procedure greedy ( var G: GRAPH; var newclr: LIST);
{ greedy assigns to newclr those vertices that may be given the same color}
var found: boolean;
    v, w: integer;
begin
  newclr := 0;
  v : = first uncolored vertex in G;
  while v < > null do begin
     found: = false;
     w : = first vertex in newclr;
     while w < > null do begin
         if there is an edge between v and w in G then
            found := true;
         w : = next vertex in newclr
      end;
      if found = false do begin
        mark v colored;
        add v to newclr
      end;
      v : = next uncolored vertex in G
   end                                                                        21
end; {greedy}
Abstract Data Types

    Procedures generalize the notion of an
    operator.
    Another advantage of procedures is that
    they can be used to encapsulate parts of an
    algorithm by localizing in one section of a
    program all the statements relevant to a
    certain aspect of a program.

                                                  22
Abstract Data Types (ADT) …

   Think of ADT as a mathematical model with a
    collection of operations defined on that model.
    Example: Set of integers, together with the
    operations of union, intersection, and set
    difference, form a simple example of an ADT.
    The two properties of procedures mentioned above
    – generalization and encapsulation – apply equally
    well to abstract data types.
    An implementation of an ADT is a translation, into
    statements of a programming language.
                                                          23
Data types, Data Structures, and Abstract
Data Types

    In a programming language, the data type of a
    variable is the set of values that the variable may
    assume.
    Abstract Data Type is a mathematical model,
    together with various operations defined on the
    model.
    To represent the mathematical model underlying an
    ADT we use data structures, which are collection of
    variables, possibly of several different data types,
    connected in various ways.
                                                           24
   The problem solving process program

       An
Mathematical      Abstract data
                                   Data structures
   model             types



                   Pseudo -
  Informal
                   language        Java program
  algorithm
                    program
                                                     25
An Introduction

      Introduction
      Proof by induction
      Asymptotic notation
Review: Induction

   Suppose
    –   S(k) is true for fixed constant k
            Often k = 0
    –   S(n)  S(n+1) for all n >= k
   Then S(n) is true for all n >= k




                                            27
Proof By Induction

   Claim:S(n) is true for all n >= k
   Basis:
    –   Show formula is true when n = k
   Inductive hypothesis:
    –   Assume formula is true for an arbitrary n
   Step:
    –   Show that formula is then true for n+1

                                                    28
Induction Example:
Gaussian Closed Form

   Prove 1 + 2 + 3 + … + n = n(n+1) / 2
    –   Basis:
            If n = 0, then 0 = 0(0+1) / 2
    –   Inductive hypothesis:
            Assume 1 + 2 + 3 + … + n = n(n+1) / 2
    –   Step (show true for n+1):
         1 + 2 + … + n + n+1 = (1 + 2 + …+ n) + (n+1)
         = n(n+1)/2 + n+1 = [n(n+1) + 2(n+1)]/2
         = (n+1)(n+2)/2 = (n+1)(n+1 + 1) / 2

                                                        29
         Induction Example:
         Geometric Closed Form

   Prove a0 + a1 + … + an = (an+1 - 1)/(a - 1) for
    all a  1
    –   Basis: show that a0 = (a0+1 - 1)/(a - 1)
         a0 = 1 = (a1 - 1)/(a - 1)
    –   Inductive hypothesis:
            Assume a0 + a1 + … + an = (an+1 - 1)/(a - 1)
    –   Step (show true for n+1):
         a0 + a1 + … + an+1 = a0 + a1 + … + an + an+1
         = (an+1 - 1)/(a - 1) + an+1 = (an+1+1 - 1)/(a - 1)
                                                              30
Induction Example:
Gaussian Closed Form

   Prove that 12 + 22 + 32 + … + n2 = (2n3 + 3n2
    + n)/6 (sum of squares)

   What is this called?

   Polynomial Series.


                                                    31
                       Induction…

   We’ve been using weak induction
   Strong induction also holds
    –   Basis: show S(0)
    –   Hypothesis: assume S(k) holds for arbitrary k <= n
    –   Step: Show S(n+1) follows
   Another variation:
    –   Basis: show S(0), S(1)
    –   Hypothesis: assume S(n) and S(n+1) are true
    –   Step: show S(n+2) follows

                                                             32
              Chapter 1


Algorithms: Efficiency, Analysis, and Order




                                              33
Algorithms

   Looking at individual modules that
    accomplish a specific task.
   Need an algorithm that will solve all
    instances (with any values of parameters) of
    problem
   Solution needs to be efficient – solves
    problem within the required resource
    constraints
                                                   34
                     Algorithms

   Problem: is a questions to which we seek an answer.
   Example: Sort a list S of n numbers in
    nondecreasing order. The answer is the numbers in
    sorted sequence.
   Other examples?




                                                          35
                   Algorithms …

   1.1 - Sequential search
   1.2 - Add array members
   1.3 - Exchange sort (bubble sort)
   1.4 - Matrix multiplication
   1.5 – Binary search




                                        36
              Developing efficient Algorithms



   Sequential search versus binary search
   1.6 - The Fibonacci sequence (recursive)
   1.7 - The Fibonacci sequence (iterative)




                                                37
              Analysis of Algorithms



 Complexity analysis
 Analysis of algorithm 1.2
 Analysis of algorithm 1.3
 Analysis of algorithm 1.4
 Analysis of algorithm 1.5
 Analysis of algorithm 1.1

                                       38
Analysis of Algorithms

   Algorithm analysis measured the efficiency of
    an algorithm as the input size becomes
    large.
    –   Critical resource is most often its running time
    –   Can also take into account the space required to
        run the program




                                                           39
Complexity Analysis

   Looking for a measure that is independent of
    the computer, the programming language,
    the programmer, and all the complex details
    of the algorithm.
   Analyze the algorithm efficiency by
    determining the number of times some basic
    operation is done as a function of the size of
    the input.
                                                     40
Complexity Analysis…

   After determining the input size, pick some
    instruction (or group of instructions) such that
    the total work done by the algorithm is
    roughly proportional to the number of times
    this instruction is done.
   In general, time complexity analysis of an
    algorithm is the determination of how many
    times the basic operation is done for each
    value of the input size.
                                                       41
                        Order



 What is order?
 An intuitive introduction to order
 A rigorous introduction to order
 Using a limit to determine order




                                       42
                     Order …

   Is 0.01n2 > 100n?
   Linear-time algorithms
   Quadratic-time algorithms ( (n2 ) algorithm)
   Pure quadratic
   Complete quadratic



                                                    43
Asymptotic Performance

   In this course, we care most about
    asymptotic performance
    –   How does the algorithm behave as the problem
        size gets very large?
            Running time
            Memory/storage requirements
            Bandwidth/power requirements/logic gates/etc.




                                                             44
Asymptotic Notation

   By now you should have an intuitive feel for
    asymptotic (big-O) notation:
    –   What does O(n) running time mean? O(n2)?
        O(n log n)?
    –   How does asymptotic running time relate to
        asymptotic memory usage?
   Our first task is to define this notation more
    formally and completely

                                                     45
Analysis of Algorithms

   Analysis is performed with respect to a computational
    model
   We will usually use a generic uniprocessor random-
    access machine (RAM)
     – All memory equally expensive to access
     – No concurrent operations
     – All reasonable instructions take unit time
         Except, of course, function calls
     – Constant word size
         Unless we are explicitly manipulating bits        46
    Input Size

   Time and space complexity
     – This is generally a function of the input size
         E.g., sorting, multiplication
     – How we characterize input size depends:
         Sorting: number of input items
         Multiplication: total number of bits
         Graph algorithms: number of nodes & edges
         Etc

                                                        47
Running Time

   Algorithm FindMax

Int findMax (E, n)
1. max = E [0];
2. for (index = 1; index < n; index ++)
3.      if (max < E [index])
4.          max = E [index];
5. return max;

                                          48
Running Time

   Number of primitive steps that are executed
     – Except for time of executing a function call, most
       statements roughly require the same amount of
       time
         y = m * x + b
          c = 5 / 9 * (t - 32 )
          z = f(x) + g(y)
   We can be more exact if need be

                                                            49
    Analysis

   Worst case
    –   Provides an upper bound on running time
    –   An absolute guarantee
   Average case
    –   Provides the expected running time
    –   Very useful, but treat with care: what is “average”?
            Random (equally likely) inputs
            Real-life inputs

                                                               50
    The running time of a program






				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:7
posted:7/11/2011
language:English
pages:51