# Algorithms

Document Sample

```					              Algorithms

Amortized Analysis

CS 8833   Algorithms
Amortized Analysis
   Method for analyzing algorithms that
involve a series of operations on a data
structure
– some operations are more expensive than
others
– the cost of the expensive operations is
balanced by the cost of the less expensive
operations
– worst case cost per operation is less than
the cost of the most expensive operation
CS 8833   Algorithms
 The term amortization comes from the
 Problem: I pay my bills once per month.
One particular bill is \$600 every six
months. What is the monthly cost of the
bill in the worst case?
if monthi mod 6 = 0
costi  600
else costi 0

CS 8833   Algorithms
   Worst case cost for n months?

   Is this an upper bound on the worst
case cost for n months?

   Is this a good upper bound?
CS 8833   Algorithms
Amortized Cost
   The average cost per month (amortized
cost) c  T(n)  600(n div 6)
n          n

So if n= 12
1200
T(n)        100/ month
12
   Analysis method used–aggregate
method. Amortized cost is applied to
all operations
CS 8833     Algorithms
Accounting Method
   I will pay more than I need to some months
and specify when I will use the extra money.
   Each operation has a cost (\$100 in this case).
When I pay \$100 and need 0, I will apply that
\$100 to the next month that the bill is due. I
accumulate a credit in my account.
   Constraint–the balance in the account must
never be negative.
   Must associate the extra money with a
particular part of my data structure (a
particular month in this case)
CS 8833   Algorithms
Potential Method
   Like the accounting method except that credit
is not associated with a particular part of the
data structure, but with a pool of potential
energy for the entire data structure.
   Potential energy – like an escrow account
   Each operation has a cost. If the cost of an
operation is more than the energy associated
with the operation, I draw on the potential
energy pool for the excess
   The potential can never be negative
CS 8833    Algorithms
Notation
Di         data structure after ith operation
       potential function
 (Di ) potential associated with data structure D
after ith operation
ci         actual cost of each operation
ˆ
ci              amortized cost of ith operation
ci  ci +  ( Di ) -  ( Di -1 )
ˆ
n          n

 c   (c   ( D )   ( D
ˆ
i =1
i
i 1
i       i       i 1   ))

 n 
   ci    ( Dn )   ( D0 )
 i 1 
CS 8833 Algorithms
Stack Operations
   Operations
PUSH(s,x)
POP(s)
MULTIPOP(s,k)
where s is the stack
x is the item pushed onto the stack
k is the number of items popped in a
MULTIPOP operation
CS 8833   Algorithms
MULTIPOP
MULTIPOP(s,k)
1 while not STACK-EMPTY(s) and k°0
2     do POP(s)
3        k k-1

CS 8833   Algorithms
COST OF OPERATIONS
   ACTUAL COST
– PUSH(s,x)            1
– POP(s)               1
– MULTIPOP(s,k)        min(k,s)
 For a series of n operations, the worst
case cost per operation is O(n) when all
n items are popped with a MULTIPOP
 A cost of O(n) for n operations yields a
total cost for the sequence of O(n2)
CS 8833   Algorithms
Tighter Bound
   Aggregate method
– You can’t POP more objects than you
PUSH
– So for n operations, the total cost cannot
be more than 2n
– This gives an upper bound of O(n) for the
cost of a sequence of n operations
– Amortized cost per operation
T(n)/n = 1
CS 8833   Algorithms
Accounting Method
Actual cost     Amortized cost
PUSH          1               2
POP           1               0
MULTIPOP      min(k,s)        0

CS 8833   Algorithms
Description
 Let a dollar represent each unit cost.
 When you push and object, pay \$1 for
the push and keep \$1 with the object on
the stack
 When you perform pop operations, you
can use the \$1 associated with the
object to pay the cost of popping each
object
CS 8833   Algorithms
Amortized Cost using
Accounting Method
 Sequence of n operations in the worst
case (n pushes) will cost 2n
 Amortized cost of n pushes is (n) in
worst case
 Amortized cost of a single operation is


CS 8833   Algorithms
Potential Method
 They key is to come up with a good
potential function
 For stack operations, we know that for
any object the cost of pushing it will be
1 and the cost of popping it will be one,
so for processing n items, the max cost
per item is 2

CS 8833   Algorithms
Potential Method

 potential function - number of items on stack
Empty stack  (D 0 ) = 0
Two constraint s
 (Di )  0
 (Di )   (D 0 )

CS 8833   Algorithms
Amortized Cost of Operations
Amortized cost of operations
PUSH
ci  ci   ( Di )   ( Di 1 )  1  1  2
ˆ
MULTIPOP - Let k' = min(k, s)
ci  ci   ( Di )   ( Di 1 )  k ' k '  0
ˆ
POP
ci  ci   ( Di )   ( Di 1 )  1  1  0
ˆ
Worst case cost of n operations is 2n = O(n)
CS 8833      Algorithms
Dynamic Tables
   Must enlarge the size of a table as needed.
We also want to be able to decrease the size
of the table if many elements are removed
   Dynamically expanding and contracting a
table
   Some insertions and deletions are very
expensive if they trigger an expansion or
contraction
   Will show amortized cost of insertions and
deletions is O(1)
CS 8833   Algorithms
Assumptions
 T is an object representing a table
 table[T] is a pointer to the block of
storage representing the table
 num[T] is the number of items in the
table
 size[T] is the total number of slots in the
table
 Initially, the table is empty
num[T] = size[T] = 0
CS 8833   Algorithms
TABLE-INSERT(T,x)
1 if size[T] = 0
2    then allocate table[T] with 1 slot
3          size[T]  1
4 if num[T] = size[T]
5     then allocate new-table with 2*size[T] slots
6          insert all items in table[T] into new-table
7          free table[T]
8         table[T] new-table
9         size[T] 2 * size[T]
10 insert x into table[T]
11 num[T] num[T] + 1

CS 8833       Algorithms
 We are using a common heuristic of
doubling the size of the table when we
allocate new storage. Why wouldn’t we
just increase the size of the table by 1?
 Load factor of the table (the percentage
of the allocated storage that is used)
– (T)=#of items in table/size of table

CS 8833   Algorithms
Analysis of Insertions
   Assumptions
– Running time is linear in the number of
items inserted
– During expansion, line 6 dominates
   Sequence of operations on an initially
empty table
– What is the worst case cost of inserting
items into the table
CS 8833   Algorithms
What is the cost c of the ith operation?
i

If the table is not full ic  1
else ic i             i
//Copy-1 items and insert item i
Worst case
2
n operations each requiring O(n) time resulting) in O(n
as the cost for all n operations
-
But,this is only the case if we expand the table each time
we don't
i if i- 1 is a power of 2

ci  


1         otherwise

CS 8833        Algorithms
Aggregate Analysis
n       lg n
Total cost of n inserts  c i  n   2 j
i =1     j 0

x n1  1    n

Remember that x                       k

k 0     x 1
2
lg n1
 lg n                      1  lg n1
So  2          j
2          1 2lg n2 1 2n  1
j 0                   2 1
n                    lg n
 ci  n   2 j  n  2n  3n
i =1                  j 0

Amortized cost of an insert is 3.
CS 8833                 Algorithms
Accounting Method
see why the cost per operation is 3.
 The 3 operations each insert pays for
are
– inserting the item itself
– moving itself on expansion
– moving another item that has already been
moved

CS 8833   Algorithms
Examples
   Suppose that we have just done an expansion
so we have m slots and m/2 objects.
   Now for each insert
– \$1 for insert
– \$1 on item inserted
– \$1 on item in the table
   When we are ready to expand the table again,
every item will be covered with the \$1 it will
cost to move it to the new table

CS 8833   Algorithms
A
B

Operations
Insert (C)
Insert (D)
Insert (E)

CS 8833       Algorithms
Potential Method
   Criterion for Potential Method
– If we can define a potential function  such that
 (Dn)  (D0)                    n

then the total amortized cost      ciˆ
i1

is an upper bound on the actual cost
– In other words if the potential for the ith operation is
always greater than the starting potential, we always
– It is usually convenient to require (D0) = 0 and
then show that (Di)  for all i
CS 8833    Algorithms
Potential Method for Dynamic
Tables
   Define  to:
– Be 0 immediately after an expansion
– Build to the table size by the time the table
is full
   Definition of potential function for a
dynamic table
(T) = 2 * num[T] - size[T]

CS 8833   Algorithms
Does the function meet our
criteria?
   Immediately after expansion
num[T] = size[T]/2
(T) = 2*(size[T] / 2) - size[T] = 0
   Immediately before expansion
num[T] = size[T]
(T) = 2*size[T] - size[T] = size[T] = num[T]
so enough potential built up to pay for all insertions
   Since the table is always at least 1/2 full, (T) is
always non-negative

CS 8833     Algorithms
Amortized analysis of insert
using potential method
   There are two cases that must be
considered
– An insertion that does not trigger an
expansion
– An insertion that does trigger an expansion
   We need to compute the amortized cost
for both cases

CS 8833   Algorithms
No expansion
ci  ci   i   i1
ˆ
 1 (2* num  size)  (2* num1  size1 )
i    i          i        i

 1 (2* num  size)  (2* (num  1)  size)
i     i            i          i

= 1 (2* num  2* num)  (size  size)  2
i        i       i      i

=3

CS 8833    Algorithms
With Expansion
ci  ci   i   i1
ˆ
 num  (2* num  size)  (2* num1  size )
i           i i           i       i1

of i
Put everything in termsnum
i 
Remember that size 2* num -1 = 2* (num  1)
i            i

 num  (2* num  2* (num  1))  (2* (num  1)  (num  1))
i         i         i                i           i

 num  2* num  2* num  2 (2* num  2 num  1)
i        i        i            i        i

 num  2 2num  2 num  1
i         i        i

=3

CS 8833     Algorithms
Table Insertion and Deletion
   Table insert
– As described before
   Table delete
– remove specified item from table
– may also want to contract size of table
» copy items to a smaller table
» free storage of old table

CS 8833   Algorithms
Table Insertion and Deletion
   Goals
– load factor of the table is bounded below
by a constant
– amortized cost of a table operation is
bounded above by a constant
   Assume cost is measured in terms of
– elementary insertions
– elementary deletions
CS 8833   Algorithms
Strategy for Contraction
   Consider using the same strategy as for
expansion
– whenever table is less than 1/2 full,
contract to 1/2 size (contract when the load
factor is less than 1/2)
– remember that we want the amortized cost
of deletions and insertions to be bounded
above by a constant

CS 8833   Algorithms
Example
   First n/2 operations are insertions and n
is a power of 2
num[T] = size[T] = n/2
   Here are our next n/2 operations
IDDIIDDIIDD...
What is the cost of this series of operations?

CS 8833   Algorithms
What is wrong?
 After an expansion, we don’t do enough
deletions to pay for a contraction.
 And, after a contraction, we do not do
enough insertions to pay for an
expansion

CS 8833   Algorithms
Improved Strategy
 Allow the load factor to drop below 1/2
 Specifically, use a lower bound of 1/4
– Contract when the load factor drops below
1/4
– Contract the size by 1/2, resulting in a load
factor of 1/2 after the contraction
   Consistent results of expansion and
contraction
– after expansion       =1/2
– after contraction     =1/2
CS 8833    Algorithms
Analyze Using the Potential
Method
 Let  be 0 immediately after either an
expansion or a contraction.
 Then we need to build up enough
potential before another expansion or
contraction to pay for the expensive
operation

CS 8833   Algorithms
Potential Function
2  num[T ]  size[T ]    if  [T ]  1
 (T )                                        2

 size[T ] / 2  num[T ]   if  [T ]  1
2

 Potential of an empty table is 0
 Potential is never negative

CS 8833      Algorithms
Potential Function
 At any point, we need to have enough
potential to pay for the next operation.
 How much does an operation cost?
– Assume that the cost of each insertion and
deletion is 1.
– If table expands or contracts, must also add the
cost of copying the items in the table
– So we must have (Ti) = num[T] potential built up
before we expand or contract the table

CS 8833   Algorithms
Observations continued
   When the load factor is 1/2,
(Ti)=0
   When load factor is 1
size[T]=num[T]
[T] = num[T] (enough potential for copies)
   When load factor is 1/4
size[T] = 4 num[T]
[T] = num[T] (enough potential for copies)

CS 8833   Algorithms
Cost of a sequence of inserts
and deletes
ci      Actual cost of the ith operation
ˆ
ci      Amortized cost of the ith operation
numi Number of items in the table after the ith operation
sizei   Size of the table after the ith operation
i      Load factor after the ith operation

CS 8833    Algorithms
Amortized cost of an insertion
 Case 1: i-1>= 1/2          Analysis
unchanged
 Case 2: i-1< 1/2
– Table will not expand after the next
operation
– Two instances need to be considered
»  remains less than 1/2 after insertion
»  becomes greater than or equal to 1/2 after
insertion

CS 8833   Algorithms
Insertion when
i-1 and i < 1/2
ci  ci   i   i1
ˆ
= 1 + (size /2 num)  (size1 /2 num1 )
i        i        i         i

-1
Get everything in terms of i instead of i
= 1 + (size /2 num)  (size /2 (num  1))
i        i        i         i

= 1- 1 = 0

CS 8833       Algorithms
Insertion when
i-1 < 1/2 and i >= 1/2
ci  ci   i   i 1
ˆ
= 1 + (2  num i  sizei )  ( sizei 1 / 2  numi 1 )
This time it is easier to get everything in terms of i  1
= 1 + (2  ( num i -1  1)  sizei 1 )  ( sizei 1 / 2  numi 1 )
= 1 + 2  num i -1  2  sizei 1  sizei 1 / 2  numi 1
= 3  num i -1  3 sizei 1  3
2

= 3   i 1sizei -1  3 sizei 1  3
2

< 3 sizei -1  3 sizei 1  3
2            2

<3
CS 8833         Algorithms
Table Delete
Cases
i-1 < 1/2
with contraction
without contraction
i-1 >= 1/2
i >= 1/2
i < 1/2

CS 8833   Algorithms
Deletion when
i-1 < 1/2 and no contraction

ci  ci   i   i1
ˆ
= 1 + (sizei /2 num)  (size1 /2 num )
i        i         i1

= 1 + (sizei /2 num)  (size /2 (num  1))
i       i         i

= 1 +1 = 2

CS 8833     Algorithms
Deletion when
i-1 < 1/2 and contraction

ci  ci   i   i 1
ˆ
The actual cost will be the cost of moving num i items
and deleting one item         ci  num i + 1
ci = (num i + 1) + ( size i / 2  numi )  ( sizei 1 / 2  numi 1 )
ˆ
Since sizei / 2  sizei 1 / 4  numi  1
= (num i + 1) + ((num i + 1)  numi )  ((2numi  2)  (numi  1))
=1

CS 8833          Algorithms

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 12 posted: 10/11/2011 language: English pages: 51