# Asymptotic Notations

Document Sample

```					         Analyzing Algorithms
• Goal
– Find asymptotic complexity of algorithm
• Approach
– Ignore less frequently executed parts of
algorithm
– Find critical section of algorithm
– Determine how many times critical section is
executed as function of problem size
Execution time
• Time computer takes to execute f(n)
operations is cf(n)
• where c
– depends on speed of computer and
– varies from computer to computer
Development of Notation
• Not concerned with small values of n
• Concerned with VERY LARGE values of n
• Asymptotic – refers to study of function f as
n approaches infinity

• Example: f(n) = n2+4n + 20
n2 is the dominant term and the term 4n + 20
becomes insignificant as n grows larger
Development of Notation
• Drop insignificant terms and constants
• Say function is of O(n2) called Big-O of n2

Definition 2.1 A function g(N) is said to be
O(f(N)) if there exist constants c0 and N0 such
that g(N) < c0 f(N) for all N > N0.

Constants c0 and N0 often hide implementation
details important in practice.
Critical Section of Algorithm
• Heart of algorithm
• Dominates overall execution time
• Characteristics
– Operation central to functioning of program
– Contained inside deeply nested loops
– Executed as often as any other part of algorithm
• Sources
– Loops
– Recursion
Critical Section Example 1
• Code (for input size n)
1. A
2. for (int i = 0; i < n; i++)
3.   B
4. C
• Code execution
– A
– B
– C
• Time 
Critical Section Example 1
• Code (for input size n)
1. A
2. for (int i = 0; i < n; i++)
critical
3.   B
section
4. C
• Code execution
– A  once
– B  n times
– C  once
• Time  1 + n + 1 = O(n)
Critical Section Example 2
• Code (for input size n)
1. A
2. for (int i = 0; i < n; i++)
3. B
4. for (int j = 0; j < n; j++)
5.     C
6. D
• Code execution
– A            -C
– B            -D
• Time 
Critical Section Example 2
• Code (for input size n)
1. A
2. for (int i = 0; i < n; i++)
3. B
4. for (int j = 0; j < n; j++)
critical
5.     C
section
6. D
• Code execution
– A  once                C  n2 times
– B  n times             D  once
• Time  1 + n + n2 + 1 = O(n2)
Critical Section Example 3
• Code (for input size n)
1. A
2. for (int i = 0; i < n; i++)
3. for (int j = i+1; j < n; j++)
4.     B
• Code execution
– A
– B
• Time 
Critical Section Example 3
• Code (for input size n)
1. A
2. for (int i = 0; i < n; i++)
3. for (int j = i+1; j < n; j++)   critical
4.     B                           section
• Code execution
– A  once
– B  ½ n (n-1) times
• Time  1 + ½ n2 = O(n2)
Critical Section Example 4
• Code (for input size n)
1. A
2. for (int i = 0; i < n; i++)
3. for (int j = 0; j < 10000; j++)
4.     B
• Code execution
– A
– B
• Time 
Critical Section Example 4
• Code (for input size n)
1. A
2. for (int i = 0; i < n; i++)
3. for (int j = 0; j < 10000; j++)   critical
4.     B                             section
• Code execution
– A  once
– B  10000 n times
• Time  1 + 10000 n = O(n)
Common Big-O functions in
algorithm analysis
•   g(n) = 1 (growth is constant)
•   g(n) = log 2 n (growth is logarithmic)
•   g(n) = n (growth is linear)
•   g(n) = n log 2 n (growth is faster than linear)
•   g(n) = n2 (growth is quadratic)
•   g(n) = 2n (growth is exponential)
n        log2n nlog2n   n^2         2^n

1       0      0         1             2

2       1      2         4             4

4       2      8     16               16

8       3     24     64              256

16       4     64    256           65536

32       5    160   1024      4294967296
Common Growth Functions
(How f(n) grows as n grows)
1200

1100

1000

900

800
n
700
log2n
600                                  nlog2n
n^2
500
2^n
400

300

200

100

0
1   2    3    4    5    6
Recursive Algorithms
• Definition
– An algorithm that calls itself
• Components of a recursive algorithm
1. Base cases
•   Computation with no recursion
2. Recursive cases
•   Recursive calls
•   Combining recursive results
Critical Section Example 5
• Code (for input size n)
1. DoWork (int n)
2. if (n == 1)
3.    A
4. else
5.    DoWork(n/2)
6.    DoWork(n/2)
• Code execution
– A
– DoWork(n/2) 
• Time(1)       Time(n) =
Critical Section Example 5
• Code (for input size n)
1. DoWork (int n)
2. if (n == 1)
3.    A
4. else                             critical
5.    DoWork(n/2)                  sections
6.    DoWork(n/2)
• Code execution
– A  1 times
– DoWork(n/2)  2 times
• Time(1)  1 Time(n) = 2  Time(n/2) + 1
Recurrence Relations
• Definition
– Value of a function at a point is given in terms
of its value at other points
• Examples
–   T(n) = T(n-1) + k
–   T(n) = T(n-1) + n
–   T(n) = T(n-1) + T(n-2)
–   T(n) = T(n/2) + k
–   T(n) = 2  T(n/2) + k
Recurrence Relations
• Base case
– Value of function at some specified points
– Also called boundary values / boundary
conditions
• Base case example
–   T(1) = 0
–   T(1) = 1
–   T(2) = 1
–   T(2) = k
Solving Recurrence Equations
• Back substitution (iteration method)
– Iteratively substitute recurrence into formula
• Example
– T(1) = 5
– T(n) = T(n-1) + 1    = ( T(n-2) + 1 ) + 1
= ( ( T(n-3) + 1 ) + 1 ) + 1
= ( ( ( T(n-4) + 1 ) + 1 ) + 1 ) + 1
=…
= (…( T(1) + 1 ) + … ) + 1
= (…( 5 + 1 ) + … ) + 1
=n+4
Example Recurrence Solutions
• Examples
–   T(n) = T(n-1) + k         O(n)
–   T(n) = T(n-1) + n         O(n2)
–   T(n) = T(n-1) + T(n-2)    O(n!)
–   T(n) = T(n/2) + k         O(log(n))
–   T(n) = 2  T(n/2) + k     O(n)
–   T(n) = 2  T(n-1) + k     O(2n)

• Many additional issues, solution methods
Iterative Factorial Algorithm

int factorial (N)
{                           Time Units to Compute
temp = 1;                 -------------------------------
for i = 1 to N {          N loops.
temp = temp*i;         c for the multiplication.
}
return(temp);
}
N
T ( N )   c  cN
i 1

Thus this requires O(N) computation.
Recursive Factorial Algorithm

int factorial (N)                    Time Units to Compute
{                                    -------------------------------
if (N == 1) return 1;              c for the conditional
else return(N * factorial(N-1));   d for the multiplication and then T(N-1).

T (1)  c
T ( N )  c  d  T ( N  1)

T(N) = c+d+T(N-1) = k+T(N-1) is a recurrence relation.
How do we solve it?
Example: Factorial
T (1)  c
T ( N )  T ( N  1)  f

T ( N )  T ( N  2)  2 f
T ( N )  T ( N  k )  kf
T ( N )  T ( N  ( N  1))  ( N  1) f
T ( N )  c  ( N  1) f  O( N )
Search Algorithm
int a[N]; // An array
int value; // Value to be found
Time Units to Compute
int search ( ) {                      -------------------------------
for (i = 0; i < N; i++) {          N loops
if (a[i] == value) return(i);   c for comparison
}
return(-1);
}                                                 N 1
T ( N )   c  cN
i 0

Thus this is an O(N) algorithm.
Recursive Binary Search Algorithm
int a[N]; // Sorted array
int x; // Value to be found
Time Units to Compute
int bin_search (int left, int right) {   -------------------------------
if (left > right) return –1;         c for comparison.

int mid = (left+right) / 2;          d for computation of mid.
if (x < a[mid])                      c for comparison.
bin_search(left, mid-1);      T(N/2)
else if (x > a[mid])                 c for comparison.
bin_search(mid+1,right);      T(N/2)
else return mid;
}

Thus T(N) = T(N/2) + 3c+d = T(N/2)+f
Analysis
T (1)  c      (assume N  2 )          k

T(N)  T(N/ 2 )  f

T (N )  T (N / 4 )  2 f
T ( N )  T ( N / 2 k )  kf  c  kf
T ( N )  c  k log N  O(log N )

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 12 posted: 11/29/2011 language: English pages: 29