Asymptotic Notations

Document Sample
Asymptotic Notations Powered By Docstoc
					         Analyzing Algorithms
• Goal
  – Find asymptotic complexity of algorithm
• Approach
  – Ignore less frequently executed parts of
    algorithm
  – Find critical section of algorithm
  – Determine how many times critical section is
    executed as function of problem size
            Execution time
• Time computer takes to execute f(n)
  operations is cf(n)
• where c
  – depends on speed of computer and
  – varies from computer to computer
     Development of Notation
• Not concerned with small values of n
• Concerned with VERY LARGE values of n
• Asymptotic – refers to study of function f as
  n approaches infinity

• Example: f(n) = n2+4n + 20
  n2 is the dominant term and the term 4n + 20
  becomes insignificant as n grows larger
      Development of Notation
• Drop insignificant terms and constants
• Say function is of O(n2) called Big-O of n2

Definition 2.1 A function g(N) is said to be
 O(f(N)) if there exist constants c0 and N0 such
 that g(N) < c0 f(N) for all N > N0.

Constants c0 and N0 often hide implementation
 details important in practice.
    Critical Section of Algorithm
• Heart of algorithm
• Dominates overall execution time
• Characteristics
  – Operation central to functioning of program
  – Contained inside deeply nested loops
  – Executed as often as any other part of algorithm
• Sources
  – Loops
  – Recursion
      Critical Section Example 1
• Code (for input size n)
  1. A
  2. for (int i = 0; i < n; i++)
  3.   B
  4. C
• Code execution
  – A
  – B
  – C
• Time 
    Critical Section Example 1
• Code (for input size n)
  1. A
  2. for (int i = 0; i < n; i++)
                                   critical
  3.   B
                                   section
  4. C
• Code execution
  – A  once
  – B  n times
  – C  once
• Time  1 + n + 1 = O(n)
    Critical Section Example 2
• Code (for input size n)
  1. A
  2. for (int i = 0; i < n; i++)
  3. B
  4. for (int j = 0; j < n; j++)
  5.     C
  6. D
• Code execution
  – A            -C
  – B            -D
• Time 
     Critical Section Example 2
• Code (for input size n)
  1. A
  2. for (int i = 0; i < n; i++)
  3. B
  4. for (int j = 0; j < n; j++)
                                       critical
  5.     C
                                       section
  6. D
• Code execution
  – A  once                C  n2 times
  – B  n times             D  once
• Time  1 + n + n2 + 1 = O(n2)
    Critical Section Example 3
• Code (for input size n)
  1. A
  2. for (int i = 0; i < n; i++)
  3. for (int j = i+1; j < n; j++)
  4.     B
• Code execution
  – A
  – B
• Time 
    Critical Section Example 3
• Code (for input size n)
  1. A
  2. for (int i = 0; i < n; i++)
  3. for (int j = i+1; j < n; j++)   critical
  4.     B                           section
• Code execution
  – A  once
  – B  ½ n (n-1) times
• Time  1 + ½ n2 = O(n2)
    Critical Section Example 4
• Code (for input size n)
  1. A
  2. for (int i = 0; i < n; i++)
  3. for (int j = 0; j < 10000; j++)
  4.     B
• Code execution
  – A
  – B
• Time 
    Critical Section Example 4
• Code (for input size n)
  1. A
  2. for (int i = 0; i < n; i++)
  3. for (int j = 0; j < 10000; j++)   critical
  4.     B                             section
• Code execution
  – A  once
  – B  10000 n times
• Time  1 + 10000 n = O(n)
       Common Big-O functions in
          algorithm analysis
•   g(n) = 1 (growth is constant)
•   g(n) = log 2 n (growth is logarithmic)
•   g(n) = n (growth is linear)
•   g(n) = n log 2 n (growth is faster than linear)
•   g(n) = n2 (growth is quadratic)
•   g(n) = 2n (growth is exponential)
n        log2n nlog2n   n^2         2^n

     1       0      0         1             2

     2       1      2         4             4

     4       2      8     16               16

     8       3     24     64              256

    16       4     64    256           65536

    32       5    160   1024      4294967296
        Common Growth Functions
       (How f(n) grows as n grows)
1200

1100

1000

900

800
                                     n
700
                                     log2n
600                                  nlog2n
                                     n^2
500
                                     2^n
400

300

200

100

  0
       1   2    3    4    5    6
         Recursive Algorithms
• Definition
  – An algorithm that calls itself
• Components of a recursive algorithm
  1. Base cases
     •   Computation with no recursion
  2. Recursive cases
     •   Recursive calls
     •   Combining recursive results
    Critical Section Example 5
• Code (for input size n)
  1. DoWork (int n)
  2. if (n == 1)
  3.    A
  4. else
  5.    DoWork(n/2)
  6.    DoWork(n/2)
• Code execution
  – A
  – DoWork(n/2) 
• Time(1)       Time(n) =
      Critical Section Example 5
• Code (for input size n)
  1. DoWork (int n)
  2. if (n == 1)
  3.    A
  4. else                             critical
  5.    DoWork(n/2)                  sections
  6.    DoWork(n/2)
• Code execution
  – A  1 times
  – DoWork(n/2)  2 times
• Time(1)  1 Time(n) = 2  Time(n/2) + 1
          Recurrence Relations
• Definition
  – Value of a function at a point is given in terms
    of its value at other points
• Examples
  –   T(n) = T(n-1) + k
  –   T(n) = T(n-1) + n
  –   T(n) = T(n-1) + T(n-2)
  –   T(n) = T(n/2) + k
  –   T(n) = 2  T(n/2) + k
          Recurrence Relations
• Base case
  – Value of function at some specified points
  – Also called boundary values / boundary
    conditions
• Base case example
  –   T(1) = 0
  –   T(1) = 1
  –   T(2) = 1
  –   T(2) = k
    Solving Recurrence Equations
• Back substitution (iteration method)
  – Iteratively substitute recurrence into formula
• Example
  – T(1) = 5
  – T(n) = T(n-1) + 1    = ( T(n-2) + 1 ) + 1
                         = ( ( T(n-3) + 1 ) + 1 ) + 1
                         = ( ( ( T(n-4) + 1 ) + 1 ) + 1 ) + 1
                         =…
                         = (…( T(1) + 1 ) + … ) + 1
                         = (…( 5 + 1 ) + … ) + 1
                         =n+4
  Example Recurrence Solutions
• Examples
  –   T(n) = T(n-1) + k         O(n)
  –   T(n) = T(n-1) + n         O(n2)
  –   T(n) = T(n-1) + T(n-2)    O(n!)
  –   T(n) = T(n/2) + k         O(log(n))
  –   T(n) = 2  T(n/2) + k     O(n)
  –   T(n) = 2  T(n-1) + k     O(2n)


• Many additional issues, solution methods
       Iterative Factorial Algorithm

int factorial (N)
{                           Time Units to Compute
  temp = 1;                 -------------------------------
  for i = 1 to N {          N loops.
     temp = temp*i;         c for the multiplication.
  }
  return(temp);
}
                                      N
                         T ( N )   c  cN
                                     i 1



              Thus this requires O(N) computation.
         Recursive Factorial Algorithm

int factorial (N)                    Time Units to Compute
{                                    -------------------------------
  if (N == 1) return 1;              c for the conditional
  else return(N * factorial(N-1));   d for the multiplication and then T(N-1).


                                     T (1)  c
                                     T ( N )  c  d  T ( N  1)

         T(N) = c+d+T(N-1) = k+T(N-1) is a recurrence relation.
         How do we solve it?
     Example: Factorial
T (1)  c
T ( N )  T ( N  1)  f

T ( N )  T ( N  2)  2 f
T ( N )  T ( N  k )  kf
T ( N )  T ( N  ( N  1))  ( N  1) f
T ( N )  c  ( N  1) f  O( N )
                     Search Algorithm
int a[N]; // An array
int value; // Value to be found
                                      Time Units to Compute
int search ( ) {                      -------------------------------
   for (i = 0; i < N; i++) {          N loops
      if (a[i] == value) return(i);   c for comparison
   }
   return(-1);
}                                                 N 1
                                      T ( N )   c  cN
                                                  i 0



                         Thus this is an O(N) algorithm.
    Recursive Binary Search Algorithm
int a[N]; // Sorted array
int x; // Value to be found
                                         Time Units to Compute
int bin_search (int left, int right) {   -------------------------------
    if (left > right) return –1;         c for comparison.

    int mid = (left+right) / 2;          d for computation of mid.
    if (x < a[mid])                      c for comparison.
           bin_search(left, mid-1);      T(N/2)
    else if (x > a[mid])                 c for comparison.
           bin_search(mid+1,right);      T(N/2)
    else return mid;
}

                   Thus T(N) = T(N/2) + 3c+d = T(N/2)+f
              Analysis
T (1)  c      (assume N  2 )          k


T(N)  T(N/ 2 )  f

T (N )  T (N / 4 )  2 f
T ( N )  T ( N / 2 k )  kf  c  kf
T ( N )  c  k log N  O(log N )

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:12
posted:11/29/2011
language:English
pages:29