CS/IS C363 Data Structures & Algorithms
8/13/2010
Sundar B.
REVIEW: EFFICIENCY & COMPLEXITY
CSIS, BITS, Pilani
Resources and Measurements
Time and Space Complexity
1
- Order Complexity and Notation
- Examples
Cost Models
RESOURCES
8/13/2010
Resources and Usage
Resource Resource Usage
Sundar B.
CPU CPU Time
Space – Main Memory and Memory Used (during
Secondary Memory computation)
I/O Devices (including I/O Time (for input/output and
networking devices) swapping)
CSIS, BITS, Pilani
Power Power consumed (for the entire
process)
Measurement
Absolute (exact) measurement
Design Time measurement (estimate) 2
ALGORITHMIC COMPLEXITY
8/13/2010
Design Time Measurement of Resource Usage
Measured and expressed in proportion to problem size (i.e.
Sundar B.
input size)
3 factors:
Time Complexity
Space Complexity
CSIS, BITS, Pilani
I/O Complexity
[Will be covered – if time permits – at the end of the course].
3
COMPLEXITY - EXAMPLE [1]
Example 1 (Y and Z are input)
X = 3 * Y + Z;
// operations: addition, multiplication, assignment
X = 2 + X;
// operations: addition, assignment
We count it in the abstract:
each statement takes 1 unit of time
under the assumption / knowledge
that the difference across instruction sets and hardware
organization is a “small constant factor” and
that the typical statement is made of a constant
number of operations
Space Complexity : 1 unit
4
COMPLEXITY - EXAMPLE [2]
// a and N are input
j = 0;
while (j 103
6.6 100 104 106 >1025
9.9 1000 106 109 >10250
13.2 10000 108 1012 >102500
MOTIVATION FOR ORDER NOTATION - BIG O
Examples
• 100 * log2N 1000
• 70 * N + 3000 100
• 105 * N2 + 106 * N 26
• Abstraction – Upper Bound
• 100*log2 N is O(log N)
• 70 * N + 3000 is O(N)
• 105 * N2 + 106 * N is O(N*N)
• 2 * 2N + 106 * N17 + 1789 is O(2N)
MOTIVATION FOR ORDER NOTATION
N N2/10 N3/10 2N/10
2 0.4 0.8 0.4
10 10 102 >102
100 103 105 >1024
1000 105 108 >10249
10000 107 1011 >102499
Compare , for example,
an O(N) Algorithm A running on a machine M1 with speed x, with
an O(N*N) Algorithm B running on a machine with speed 10x
MOTIVATION FOR ORDER NOTATION
N N2/104 N3/104 2N/104
2 0.0004 0.0008 0.0004
10 0.01 0.1 >0.1
1000 100 105 >10246
104 104 108 >102496
106 108 1020 ?!*@
Compare , for example,
an O(N) Algorithm A running on a machine M1 with speed x, with
an O(N*N) Algorithm B running on a machine with speed 10000x
ORDER NOTATION AND CONVENTIONS
Asymptotic Complexity – Upper Bound
g(n) is O(f(n))
if there is a constant c such that g(n) 0
We are often not interested in the best case.
Typical measures are for the worst case and the average
(or expected) case.
13 August 2010 TA C252 BITS, Pilani. 12
BINARY SEARCH ALGORITHM
Worst case:
low = 1; high = N; •Loop executes until low > high
while (low <= high) { i.e. until size of list becomes 0
mid = (low + high) /2;
•Size halved in each iteration
if (A[mid] = = x) return x; N, N/2, N/4, … 1
else if (A[mid] < x) low = mid +1;
• Number of steps = K
else high = mid – 1; such that 2K = N
} i.e. logN steps
return Not_Found; where N is input size
Time complexity O(logN)
TIME COMPLEXITY
8/13/2010
Polynomial Time Complexity
Time Complexity is O(Nk) for some constant k, where N is input size.
Sundar B.
Consider the following algorithm:
int fact(int N) {
j=1; prod=1;
while (j<=N) {
j=j+1; prod=prod*j;
CSIS, BITS, Pilani
}
return prod;
}
What is the time complexity?
Is this polynomial time? Why or why not?
14
TIME COMPLEXITY
8/13/2010
Uniform Cost vs. Logarithmic Cost
Uniform Cost – All basic operations cost same (constant)
Sundar B.
amount of time (irrespective of the data size)
Logarithmic Cost – Each operation has a cost that is
proportional to the size of the data
Hint: Refer to the RAM model slide for assumptions; consider
the call fact(100). End of Hint.
CSIS, BITS, Pilani
Revisit the example (factorial)
What is the time complexity?
Why?
Number of operations
Cost of individual operations.
15