# Analysis of Algorithms CS 4413_1_ by hcj

VIEWS: 7 PAGES: 51

• pg 1
```									Analysis of Algorithms
CS 4413

Briana B. Morrison
With thanks to Dr. Hung
Algorithmics: the study of algorithms.
Algorithmics: is the core of computer science.
Algorithmics: the spirit of computing.
How important is this course?!

2
The Course

   Purpose: a rigorous introduction to the
design and analysis of algorithms
–   Not a lab or programming course
–   Not a math course, either
   But, need know how to program!
   Minimum math knowledge!

3
Course Objectives

   Basic design concepts and analysis methods of
algorithms
   Design methodologies include, but not limited to:
–   Searching/Sorting
–   Divide-and-Conquer
–   Greedy Method
–   Dynamic Programming
–   Search and Traversal
–   Branch-and-Bound

4

   Elegant solution
   Logical thinking
   Quality of writing and coding
   Concise writing is a virtue.
   Tell the reader only what is important.

5
Research Project Guidelines

   Research projects should consist of the
following components:
–   Define the problem to be solved
–   Background information
–   Algorithms and Applications
–   Implementation
–   Drawbacks
–   New applications and discoveries

6
Research Project Guidelines

   Some examples of research projects:
–   Dynamic Programming
–   SNAKE
–   Marko Chain Monte Carlo (MCMC) (and Image
analysis)
–   Wavelet Transforms
–   Others

7
From Problems to Programs

    There are many steps involved in writing a
computer program to solve a given problem.
    The steps go from
–   1) problem formulation and specification,
–   2) to design of the solution,
–   3) to implementation, testing and documentation,
and finally
–   4) to evaluation of the solution.

8
From Problems to Programs …

    Algorithms and Data Structures are the
building blocks of most computer programs.
    Half the battle is knowing what problem to
solve.
    Almost any branch of mathematics or
science can be called into service to help
model some problem domain.
–   Examples: problems essentially numerical in
nature.
9
Algorithms

An algorithm is a finite sequence of instructions,
each of which has a clear meaning
and can be performed with a finite amount of effort
in a finite length of time.

   Pseudo-language is used to present algorithms.
   Pseudo-language is a combination of the
constructs of a programming language together with
informal English statements.
10
Algorithms …

    Example 1: A mathematical model can be
used to help design a traffic light for a
complicated intersection of roads (figure 1.1).

Figure 1.1. An intersection.

11
Figure 1.2. Graph showing incompatible turns.   12
Algorithms …

Figure 1.3. Table of incompatible turns.
13
Graph Coloring Problem

Figure 1.4. A graph.
14
Heuristic

    An algorithm that quickly produces good but
not necessarily optimal solutions is called a
heuristic.
    One reasonable heuristics for graph coloring
is the so-called “greedy” algorithm.

15
Greedy Algorithm

    Initially we try to color as many vertices as possible
with the first color, then as many as possible of the
uncolored vertices with the second color, and so on.
    To color vertices with a new color, we perform the
folowing steps:
1) Select some uncolored vertex and color it with the new color.
2) Scan the list of uncolored vertices. For each uncolored vertex,
determine whether it has an edge to any vertex already colored
with the new color. If there is no such edge, color the present
vertex with the new color.

16
Graph Coloring Problem …

Figure 1.5. A coloring of the graph of Fig. 1.2.
17
Pseudo-Language and Stepwise
Refinement

    Once we have an appropriate mathematical
model for a problem, we can formulate an
algorithm in terms of that model.
    The initial versions of the algorithm are often
couched in general statements that will have
to be refined subsequently into smaller, more
definite instructions.

18
First refinement of greedy algorithm

procedure greedy ( var G: GRAPH; Var newclr: SET );
{ greedy assigns to newclr a set of vertices of G that may be
given the same color} .
begin
newclr := 0;
for each uncolored vertex v of G do
if v is not adjacent to any vertex in newclr then begin
mark v colored;
end
end; {greedy}                                                19
Refinement of greedy algorithm
procedure greedy ( var G: GRAPH; var newclr: SET );
begin
newclr := 0;
for each uncolored vertex v of G do begin
found := false;
for each vertex w in newclr do
if there is an edge between v and w in G then
found := true;
if found = false then begin
{ v is adjacent to no vertex in newclr }
mark v colored;
end
end                                                      20
end; {greedy}
Refined greedy algorithm
procedure greedy ( var G: GRAPH; var newclr: LIST);
{ greedy assigns to newclr those vertices that may be given the same color}
var found: boolean;
v, w: integer;
begin
newclr := 0;
v : = first uncolored vertex in G;
while v < > null do begin
found: = false;
w : = first vertex in newclr;
while w < > null do begin
if there is an edge between v and w in G then
found := true;
w : = next vertex in newclr
end;
if found = false do begin
mark v colored;
end;
v : = next uncolored vertex in G
end                                                                        21
end; {greedy}
Abstract Data Types

    Procedures generalize the notion of an
operator.
    Another advantage of procedures is that
they can be used to encapsulate parts of an
algorithm by localizing in one section of a
program all the statements relevant to a
certain aspect of a program.

22

   Think of ADT as a mathematical model with a
collection of operations defined on that model.
    Example: Set of integers, together with the
operations of union, intersection, and set
difference, form a simple example of an ADT.
    The two properties of procedures mentioned above
– generalization and encapsulation – apply equally
well to abstract data types.
    An implementation of an ADT is a translation, into
statements of a programming language.
23
Data types, Data Structures, and Abstract
Data Types

    In a programming language, the data type of a
variable is the set of values that the variable may
assume.
    Abstract Data Type is a mathematical model,
together with various operations defined on the
model.
    To represent the mathematical model underlying an
ADT we use data structures, which are collection of
variables, possibly of several different data types,
connected in various ways.
24
The problem solving process program

   An
Mathematical      Abstract data
Data structures
model             types

Pseudo -
Informal
language        Java program
algorithm
program
25
An Introduction

Introduction
Proof by induction
Asymptotic notation
Review: Induction

   Suppose
–   S(k) is true for fixed constant k
   Often k = 0
–   S(n)  S(n+1) for all n >= k
   Then S(n) is true for all n >= k

27
Proof By Induction

   Claim:S(n) is true for all n >= k
   Basis:
–   Show formula is true when n = k
   Inductive hypothesis:
–   Assume formula is true for an arbitrary n
   Step:
–   Show that formula is then true for n+1

28
Induction Example:
Gaussian Closed Form

   Prove 1 + 2 + 3 + … + n = n(n+1) / 2
–   Basis:
   If n = 0, then 0 = 0(0+1) / 2
–   Inductive hypothesis:
   Assume 1 + 2 + 3 + … + n = n(n+1) / 2
–   Step (show true for n+1):
1 + 2 + … + n + n+1 = (1 + 2 + …+ n) + (n+1)
= n(n+1)/2 + n+1 = [n(n+1) + 2(n+1)]/2
= (n+1)(n+2)/2 = (n+1)(n+1 + 1) / 2

29
Induction Example:
Geometric Closed Form

   Prove a0 + a1 + … + an = (an+1 - 1)/(a - 1) for
all a  1
–   Basis: show that a0 = (a0+1 - 1)/(a - 1)
a0 = 1 = (a1 - 1)/(a - 1)
–   Inductive hypothesis:
   Assume a0 + a1 + … + an = (an+1 - 1)/(a - 1)
–   Step (show true for n+1):
a0 + a1 + … + an+1 = a0 + a1 + … + an + an+1
= (an+1 - 1)/(a - 1) + an+1 = (an+1+1 - 1)/(a - 1)
30
Induction Example:
Gaussian Closed Form

   Prove that 12 + 22 + 32 + … + n2 = (2n3 + 3n2
+ n)/6 (sum of squares)

   What is this called?

   Polynomial Series.

31
Induction…

   We’ve been using weak induction
   Strong induction also holds
–   Basis: show S(0)
–   Hypothesis: assume S(k) holds for arbitrary k <= n
–   Step: Show S(n+1) follows
   Another variation:
–   Basis: show S(0), S(1)
–   Hypothesis: assume S(n) and S(n+1) are true
–   Step: show S(n+2) follows

32
Chapter 1

Algorithms: Efficiency, Analysis, and Order

33
Algorithms

   Looking at individual modules that
   Need an algorithm that will solve all
instances (with any values of parameters) of
problem
   Solution needs to be efficient – solves
problem within the required resource
constraints
34
Algorithms

   Problem: is a questions to which we seek an answer.
   Example: Sort a list S of n numbers in
nondecreasing order. The answer is the numbers in
sorted sequence.
   Other examples?

35
Algorithms …

   1.1 - Sequential search
   1.2 - Add array members
   1.3 - Exchange sort (bubble sort)
   1.4 - Matrix multiplication
   1.5 – Binary search

36
Developing efficient Algorithms

   Sequential search versus binary search
   1.6 - The Fibonacci sequence (recursive)
   1.7 - The Fibonacci sequence (iterative)

37
Analysis of Algorithms

 Complexity analysis
 Analysis of algorithm 1.2
 Analysis of algorithm 1.3
 Analysis of algorithm 1.4
 Analysis of algorithm 1.5
 Analysis of algorithm 1.1

38
Analysis of Algorithms

   Algorithm analysis measured the efficiency of
an algorithm as the input size becomes
large.
–   Critical resource is most often its running time
–   Can also take into account the space required to
run the program

39
Complexity Analysis

   Looking for a measure that is independent of
the computer, the programming language,
the programmer, and all the complex details
of the algorithm.
   Analyze the algorithm efficiency by
determining the number of times some basic
operation is done as a function of the size of
the input.
40
Complexity Analysis…

   After determining the input size, pick some
instruction (or group of instructions) such that
the total work done by the algorithm is
roughly proportional to the number of times
this instruction is done.
   In general, time complexity analysis of an
algorithm is the determination of how many
times the basic operation is done for each
value of the input size.
41
Order

 What is order?
 An intuitive introduction to order
 A rigorous introduction to order
 Using a limit to determine order

42
Order …

   Is 0.01n2 > 100n?
   Linear-time algorithms
   Quadratic-time algorithms ( (n2 ) algorithm)

43
Asymptotic Performance

   In this course, we care most about
asymptotic performance
–   How does the algorithm behave as the problem
size gets very large?
   Running time
   Memory/storage requirements
   Bandwidth/power requirements/logic gates/etc.

44
Asymptotic Notation

   By now you should have an intuitive feel for
asymptotic (big-O) notation:
–   What does O(n) running time mean? O(n2)?
O(n log n)?
–   How does asymptotic running time relate to
asymptotic memory usage?
   Our first task is to define this notation more
formally and completely

45
Analysis of Algorithms

   Analysis is performed with respect to a computational
model
   We will usually use a generic uniprocessor random-
access machine (RAM)
– All memory equally expensive to access
– No concurrent operations
– All reasonable instructions take unit time
 Except, of course, function calls
– Constant word size
 Unless we are explicitly manipulating bits        46
Input Size

   Time and space complexity
– This is generally a function of the input size
 E.g., sorting, multiplication
– How we characterize input size depends:
 Sorting: number of input items
 Multiplication: total number of bits
 Graph algorithms: number of nodes & edges
 Etc

47
Running Time

   Algorithm FindMax

Int findMax (E, n)
1. max = E [0];
2. for (index = 1; index < n; index ++)
3.      if (max < E [index])
4.          max = E [index];
5. return max;

48
Running Time

   Number of primitive steps that are executed
– Except for time of executing a function call, most
statements roughly require the same amount of
time
y = m * x + b
 c = 5 / 9 * (t - 32 )
 z = f(x) + g(y)
   We can be more exact if need be

49
Analysis

   Worst case
–   Provides an upper bound on running time
–   An absolute guarantee
   Average case
–   Provides the expected running time
–   Very useful, but treat with care: what is “average”?
   Random (equally likely) inputs
   Real-life inputs

50
The running time of a program



```
To top