# Algorithms by x3V7WxJi

VIEWS: 7 PAGES: 51

• pg 1
```									              Algorithms

Amortized Analysis

CS 8833   Algorithms
Amortized Analysis
   Method for analyzing algorithms that
involve a series of operations on a data
structure
– some operations are more expensive than
others
– the cost of the expensive operations is
balanced by the cost of the less expensive
operations
– worst case cost per operation is less than
the cost of the most expensive operation
CS 8833   Algorithms
 The term amortization comes from the
 Problem: I pay my bills once per month.
One particular bill is \$600 every six
months. What is the monthly cost of the
bill in the worst case?
if monthi mod 6 = 0
costi  600
else costi 0

CS 8833   Algorithms
   Worst case cost for n months?

   Is this an upper bound on the worst
case cost for n months?

   Is this a good upper bound?
CS 8833   Algorithms
Amortized Cost
   The average cost per month (amortized
cost) c  T(n)  600(n div 6)
n          n

So if n= 12
1200
T(n)        100/ month
12
   Analysis method used–aggregate
method. Amortized cost is applied to
all operations
CS 8833     Algorithms
Accounting Method
   I will pay more than I need to some months
and specify when I will use the extra money.
   Each operation has a cost (\$100 in this case).
When I pay \$100 and need 0, I will apply that
\$100 to the next month that the bill is due. I
accumulate a credit in my account.
   Constraint–the balance in the account must
never be negative.
   Must associate the extra money with a
particular part of my data structure (a
particular month in this case)
CS 8833   Algorithms
Potential Method
   Like the accounting method except that credit
is not associated with a particular part of the
data structure, but with a pool of potential
energy for the entire data structure.
   Potential energy – like an escrow account
   Each operation has a cost. If the cost of an
operation is more than the energy associated
with the operation, I draw on the potential
energy pool for the excess
   The potential can never be negative
CS 8833    Algorithms
Notation
Di         data structure after ith operation
          potential function
 (Di ) potential associated with data structure D
after ith operation
ci         actual cost of each operation
ˆ
ci              amortized cost of ith operation
ci  ci +  ( Di ) -  ( Di -1 )
ˆ
n          n

 c   (c   ( D )   ( D
ˆ
i =1
i
i 1
i       i       i 1   ))

 n 
   ci    ( Dn )   ( D0 )
 i 1 
CS 8833 Algorithms
Stack Operations
   Operations
PUSH(s,x)
POP(s)
MULTIPOP(s,k)
where s is the stack
x is the item pushed onto the stack
k is the number of items popped in a
MULTIPOP operation
CS 8833   Algorithms
MULTIPOP
MULTIPOP(s,k)
1 while not STACK-EMPTY(s) and k°0
2     do POP(s)
3        k k-1

CS 8833   Algorithms
COST OF OPERATIONS
   ACTUAL COST
– PUSH(s,x)            1
– POP(s)               1
– MULTIPOP(s,k)        min(k,s)
 For a series of n operations, the worst
case cost per operation is O(n) when all
n items are popped with a MULTIPOP
 A cost of O(n) for n operations yields a
total cost for the sequence of O(n2)
CS 8833   Algorithms
Tighter Bound
   Aggregate method
– You can’t POP more objects than you
PUSH
– So for n operations, the total cost cannot
be more than 2n
– This gives an upper bound of O(n) for the
cost of a sequence of n operations
– Amortized cost per operation
T(n)/n = 1
CS 8833   Algorithms
Accounting Method
Actual cost     Amortized cost
PUSH          1               2
POP           1               0
MULTIPOP      min(k,s)        0

CS 8833   Algorithms
Description
 Let a dollar represent each unit cost.
 When you push and object, pay \$1 for
the push and keep \$1 with the object on
the stack
 When you perform pop operations, you
can use the \$1 associated with the
object to pay the cost of popping each
object
CS 8833   Algorithms
Amortized Cost using
Accounting Method
 Sequence of n operations in the worst
case (n pushes) will cost 2n
 Amortized cost of n pushes is (n) in
worst case
 Amortized cost of a single operation is


CS 8833   Algorithms
Potential Method
 They key is to come up with a good
potential function
 For stack operations, we know that for
any object the cost of pushing it will be
1 and the cost of popping it will be one,
so for processing n items, the max cost
per item is 2

CS 8833   Algorithms
Potential Method

 potential function - number of items on stack
Empty stack  (D 0 ) = 0
Two constraints
 (Di )  0
 (Di )   (D 0 )

CS 8833    Algorithms
Amortized Cost of Operations
Amortized cost of operations
PUSH
ci  ci   ( Di )   ( Di 1 )  1  1  2
ˆ
MULTIPOP - Let k' = min(k, s)
ci  ci   ( Di )   ( Di 1 )  k ' k '  0
ˆ
POP
ci  ci   ( Di )   ( Di 1 )  1  1  0
ˆ
Worst case cost of n operations is 2n = O(n)
CS 8833      Algorithms
Dynamic Tables
   Must enlarge the size of a table as needed.
We also want to be able to decrease the size
of the table if many elements are removed
   Dynamically expanding and contracting a
table
   Some insertions and deletions are very
expensive if they trigger an expansion or
contraction
   Will show amortized cost of insertions and
deletions is O(1)
CS 8833   Algorithms
Assumptions
 T is an object representing a table
 table[T] is a pointer to the block of
storage representing the table
 num[T] is the number of items in the
table
 size[T] is the total number of slots in the
table
 Initially, the table is empty
num[T] = size[T] = 0
CS 8833   Algorithms
TABLE-INSERT(T,x)
1 if size[T] = 0
2    then allocate table[T] with 1 slot
3          size[T]  1
4 if num[T] = size[T]
5     then allocate new-table with 2*size[T] slots
6          insert all items in table[T] into new-table
7          free table[T]
8         table[T] new-table
9         size[T] 2 * size[T]
10 insert x into table[T]
11 num[T] num[T] + 1

CS 8833       Algorithms
 We are using a common heuristic of
doubling the size of the table when we
allocate new storage. Why wouldn’t we
just increase the size of the table by 1?
 Load factor of the table (the percentage
of the allocated storage that is used)
– (T)=#of items in table/size of table

CS 8833   Algorithms
Analysis of Insertions
   Assumptions
– Running time is linear in the number of
items inserted
– During expansion, line 6 dominates
   Sequence of operations on an initially
empty table
– What is the worst case cost of inserting
items into the table
CS 8833   Algorithms
What is the cost ci of the ith operation?
If the table is not full ci  1
else ic  i             i
//Copy-1 items and insert item i
Worst case
n operations each requiring O(n) time resulting 2 ) O(n
in
as the cost for all n operations
-
But,this is only the case if we expand the table each time
we don't
i if i - 1 is a power of 2

ci  


1          otherwise

CS 8833        Algorithms
Aggregate Analysis
n        lg n
Total cost of n inserts                     ci  n   2j
i =1      j 0

x n1  1    n

Remember that x                       k

k 0     x 1
lg n1
 lg n         2           1  lg n1
So  2          j
2          1 2lg n2 1 2n  1
j 0                   2 1
n                    lg n
 ci  n   2 j  n  2n  3n
i =1                  j 0

Amortized cost of an insert is 3.
CS 8833                 Algorithms
Accounting Method
see why the cost per operation is 3.
 The 3 operations each insert pays for
are
– inserting the item itself
– moving itself on expansion
– moving another item that has already been
moved

CS 8833   Algorithms
Examples
   Suppose that we have just done an expansion
so we have m slots and m/2 objects.
   Now for each insert
– \$1 for insert
– \$1 on item inserted
– \$1 on item in the table
   When we are ready to expand the table again,
every item will be covered with the \$1 it will
cost to move it to the new table

CS 8833   Algorithms
A
B

Operations
Insert (C)
Insert (D)
Insert (E)

CS 8833       Algorithms
Potential Method
   Criterion for Potential Method
– If we can define a potential function  such that
 (Dn)  (D0)                    n

then the total amortized cost      ciˆ
i1

is an upper bound on the actual cost
– In other words if the potential for the ith operation is
always greater than the starting potential, we always
– It is usually convenient to require (D0) = 0 and
then show that (Di)  for all i
CS 8833    Algorithms
Potential Method for Dynamic
Tables
   Define  to:
– Be 0 immediately after an expansion
– Build to the table size by the time the table
is full
   Definition of potential function for a
dynamic table
(T) = 2 * num[T] - size[T]

CS 8833   Algorithms
Does the function meet our
criteria?
   Immediately after expansion
num[T] = size[T]/2
(T) = 2*(size[T] / 2) - size[T] = 0
   Immediately before expansion
num[T] = size[T]
(T) = 2*size[T] - size[T] = size[T] = num[T]
so enough potential built up to pay for all insertions
   Since the table is always at least 1/2 full, (T) is
always non-negative

CS 8833     Algorithms
Amortized analysis of insert
using potential method
   There are two cases that must be
considered
– An insertion that does not trigger an
expansion
– An insertion that does trigger an expansion
   We need to compute the amortized cost
for both cases

CS 8833   Algorithms
No expansion
ci  ci   i   i1
ˆ
 1 (2* num  size)  (2* num1  size1 )
i    i          i        i

 1 (2* num  size)  (2* (num  1)  size)
i     i            i          i

= 1 (2* num  2* num)  (size  size)  2
i        i       i      i

=3

CS 8833    Algorithms
With Expansion
ci  ci   i   i1
ˆ
 num  (2* num  size)  (2* num1  size )
i         i     i           i       i1

of i
Put everything in terms num
i 
Remember that size 2* numi -1 = 2* (numi  1)
 num  (2* num  2* (num  1))  (2* (num  1)  (num  1))
i         i         i                i           i

 num  2* num  2* num  2 (2* num  2 num  1)
i        i        i            i        i

 num  2 2num  2 num  1
i         i        i

=3

CS 8833        Algorithms
Table Insertion and Deletion
   Table insert
– As described before
   Table delete
– remove specified item from table
– may also want to contract size of table
» copy items to a smaller table
» free storage of old table

CS 8833   Algorithms
Table Insertion and Deletion
   Goals
– load factor of the table is bounded below
by a constant
– amortized cost of a table operation is
bounded above by a constant
   Assume cost is measured in terms of
– elementary insertions
– elementary deletions
CS 8833   Algorithms
Strategy for Contraction
   Consider using the same strategy as for
expansion
– whenever table is less than 1/2 full,
contract to 1/2 size (contract when the load
factor is less than 1/2)
– remember that we want the amortized cost
of deletions and insertions to be bounded
above by a constant

CS 8833   Algorithms
Example
   First n/2 operations are insertions and n
is a power of 2
num[T] = size[T] = n/2
   Here are our next n/2 operations
IDDIIDDIIDD...
What is the cost of this series of operations?

CS 8833   Algorithms
What is wrong?
 After an expansion, we don’t do enough
deletions to pay for a contraction.
 And, after a contraction, we do not do
enough insertions to pay for an
expansion

CS 8833   Algorithms
Improved Strategy
 Allow the load factor to drop below 1/2
 Specifically, use a lower bound of 1/4
– Contract when the load factor drops below
1/4
– Contract the size by 1/2, resulting in a load
factor of 1/2 after the contraction
   Consistent results of expansion and
contraction
– after expansion       =1/2
– after contraction     =1/2
CS 8833    Algorithms
Analyze Using the Potential
Method
 Let  be 0 immediately after either an
expansion or a contraction.
 Then we need to build up enough
potential before another expansion or
contraction to pay for the expensive
operation

CS 8833   Algorithms
Potential Function
2  num[T ]  size[T ]    if  [T ]  1
(T )                                        2

 size[T ] / 2  num[T ]   if  [T ]  1
2

 Potential of an empty table is 0
 Potential is never negative

CS 8833      Algorithms
Potential Function
 At any point, we need to have enough
potential to pay for the next operation.
 How much does an operation cost?
– Assume that the cost of each insertion and
deletion is 1.
– If table expands or contracts, must also add the
cost of copying the items in the table
– So we must have (Ti) = num[T] potential built up
before we expand or contract the table

CS 8833   Algorithms
Observations continued
   When the load factor is 1/2,
(Ti)=0
   When load factor is 1
size[T]=num[T]
[T] = num[T] (enough potential for copies)
   When load factor is 1/4
size[T] = 4 num[T]
[T] = num[T] (enough potential for copies)

CS 8833   Algorithms
Cost of a sequence of inserts
and deletes
ci      Actual cost of the ith operation
ˆ
ci      Amortized cost of the ith operation
num i Number of items in the table after the ith operation
sizei   Size of the table after the ith operation
i      Load factor after the ith operation

CS 8833     Algorithms
Amortized cost of an insertion
 Case 1: i-1>= 1/2          Analysis
unchanged
 Case 2: i-1< 1/2
– Table will not expand after the next
operation
– Two instances need to be considered
»  remains less than 1/2 after insertion
»  becomes greater than or equal to 1/2 after
insertion

CS 8833   Algorithms
Insertion when
i-1 and i < 1/2
ci  ci   i   i1
ˆ
= 1 + (size /2 num )  (size1 /2 num1 )
i        i         i         i

of
Get everything in terms of i instead-1 i
= 1 + (size /2 num )  (size /2 (num  1))
i        i        i         i

= 1- 1 = 0

CS 8833       Algorithms
Insertion when
i-1 < 1/2 and i >= 1/2
ci  ci   i   i 1
ˆ
= 1 + (2  numi  sizei )  ( sizei 1 / 2  numi 1 )
This time it is easier to get everything in terms of i  1
= 1 + (2  ( numi -1  1)  sizei 1 )  ( sizei 1 / 2  numi 1 )
= 1 + 2  numi -1  2  sizei 1  sizei 1 / 2  numi 1
= 3  numi -1  3 sizei 1  3
2

= 3   i 1sizei -1  3 sizei 1  3
2

< 3 sizei -1  3 sizei 1  3
2            2

<3
CS 8833         Algorithms
Table Delete
Cases
i-1 < 1/2
with contraction
without contraction
i-1 >= 1/2
i >= 1/2
i < 1/2

CS 8833   Algorithms
Deletion when
i-1 < 1/2 and no contraction

ci  ci   i   i1
ˆ
= 1 + (sizei /2 num)  (size1 /2 num )
i       i          i1

= 1 + (sizei /2 num)  (size /2 (num  1))
i       i         i

= 1 +1 = 2

CS 8833        Algorithms
Deletion when
i-1 < 1/2 and contraction

ci  ci   i   i 1
ˆ
The actual cost will be the cost of moving numi items
and deleting one item        ci  numi + 1
ci = (numi + 1) + ( size i / 2  numi )  ( sizei 1 / 2  numi 1 )
ˆ
Since sizei / 2  sizei 1 / 4  numi  1
= (numi + 1) + ((numi + 1)  numi )  ((2numi  2)  (numi  1))
=1

CS 8833          Algorithms

```
To top