Algorithms by x3V7WxJi

VIEWS: 7 PAGES: 51

									              Algorithms

           Amortized Analysis




CS 8833   Algorithms
          Amortized Analysis
   Method for analyzing algorithms that
    involve a series of operations on a data
    structure
    – some operations are more expensive than
      others
    – the cost of the expensive operations is
      balanced by the cost of the less expensive
      operations
    – worst case cost per operation is less than
      the cost of the most expensive operation
    CS 8833   Algorithms
         A Business Analogy
 The term amortization comes from the
  business world
 Problem: I pay my bills once per month.
  One particular bill is $600 every six
  months. What is the monthly cost of the
  bill in the worst case?
         if monthi mod 6 = 0
              costi  600
         else costi 0

    CS 8833   Algorithms
                    Answer
   Worst case cost for n months?



   Is this an upper bound on the worst
    case cost for n months?

   Is this a good upper bound?
    CS 8833   Algorithms
                Amortized Cost
   The average cost per month (amortized
    cost) c  T(n)  600(n div 6)
                   n          n

              So if n= 12
                        1200
                T(n)        100/ month
                         12
   Analysis method used–aggregate
    method. Amortized cost is applied to
    all operations
    CS 8833     Algorithms
           Accounting Method
   I will pay more than I need to some months
    and specify when I will use the extra money.
   Each operation has a cost ($100 in this case).
    When I pay $100 and need 0, I will apply that
    $100 to the next month that the bill is due. I
    accumulate a credit in my account.
   Constraint–the balance in the account must
    never be negative.
   Must associate the extra money with a
    particular part of my data structure (a
    particular month in this case)
    CS 8833   Algorithms
              Potential Method
   Like the accounting method except that credit
    is not associated with a particular part of the
    data structure, but with a pool of potential
    energy for the entire data structure.
   Potential energy – like an escrow account
   Each operation has a cost. If the cost of an
    operation is more than the energy associated
    with the operation, I draw on the potential
    energy pool for the excess
   The potential can never be negative
    CS 8833    Algorithms
                           Notation
    Di         data structure after ith operation
              potential function
     (Di ) potential associated with data structure D
                      after ith operation
    ci         actual cost of each operation
    ˆ
    ci              amortized cost of ith operation
    ci  ci +  ( Di ) -  ( Di -1 )
    ˆ
     n          n

     c   (c   ( D )   ( D
      ˆ
    i =1
           i
               i 1
                       i       i       i 1   ))

         n 
         ci    ( Dn )   ( D0 )
         i 1 
CS 8833 Algorithms
              Stack Operations
   Operations
    PUSH(s,x)
    POP(s)
    MULTIPOP(s,k)
     where s is the stack
           x is the item pushed onto the stack
           k is the number of items popped in a
              MULTIPOP operation
    CS 8833   Algorithms
             MULTIPOP
    MULTIPOP(s,k)
    1 while not STACK-EMPTY(s) and k°0
    2     do POP(s)
    3        k k-1




CS 8833   Algorithms
      COST OF OPERATIONS
   ACTUAL COST
    – PUSH(s,x)            1
    – POP(s)               1
    – MULTIPOP(s,k)        min(k,s)
 For a series of n operations, the worst
  case cost per operation is O(n) when all
  n items are popped with a MULTIPOP
 A cost of O(n) for n operations yields a
  total cost for the sequence of O(n2)
    CS 8833   Algorithms
               Tighter Bound
   Aggregate method
    – You can’t POP more objects than you
      PUSH
    – So for n operations, the total cost cannot
      be more than 2n
    – This gives an upper bound of O(n) for the
      cost of a sequence of n operations
    – Amortized cost per operation
        T(n)/n = 1
    CS 8833   Algorithms
       Accounting Method
         Actual cost     Amortized cost
PUSH          1               2
POP           1               0
MULTIPOP      min(k,s)        0




 CS 8833   Algorithms
                 Description
 Let a dollar represent each unit cost.
 When you push and object, pay $1 for
  the push and keep $1 with the object on
  the stack
 When you perform pop operations, you
  can use the $1 associated with the
  object to pay the cost of popping each
  object
    CS 8833   Algorithms
        Amortized Cost using
         Accounting Method
 Sequence of n operations in the worst
  case (n pushes) will cost 2n
 Amortized cost of n pushes is (n) in
  worst case
 Amortized cost of a single operation is
  


    CS 8833   Algorithms
              Potential Method
 They key is to come up with a good
  potential function
 For stack operations, we know that for
  any object the cost of pushing it will be
  1 and the cost of popping it will be one,
  so for processing n items, the max cost
  per item is 2

    CS 8833   Algorithms
            Potential Method

 potential function - number of items on stack
Empty stack  (D 0 ) = 0
Two constraints
   (Di )  0
    (Di )   (D 0 )

  CS 8833    Algorithms
Amortized Cost of Operations
Amortized cost of operations
PUSH
ci  ci   ( Di )   ( Di 1 )  1  1  2
ˆ
MULTIPOP - Let k' = min(k, s)
ci  ci   ( Di )   ( Di 1 )  k ' k '  0
ˆ
POP
ci  ci   ( Di )   ( Di 1 )  1  1  0
ˆ
Worst case cost of n operations is 2n = O(n)
 CS 8833      Algorithms
              Dynamic Tables
   Must enlarge the size of a table as needed.
    We also want to be able to decrease the size
    of the table if many elements are removed
   Dynamically expanding and contracting a
    table
   Some insertions and deletions are very
    expensive if they trigger an expansion or
    contraction
   Will show amortized cost of insertions and
    deletions is O(1)
    CS 8833   Algorithms
                Assumptions
 T is an object representing a table
 table[T] is a pointer to the block of
  storage representing the table
 num[T] is the number of items in the
  table
 size[T] is the total number of slots in the
  table
 Initially, the table is empty
    num[T] = size[T] = 0
    CS 8833   Algorithms
TABLE-INSERT(T,x)
1 if size[T] = 0
2    then allocate table[T] with 1 slot
3          size[T]  1
4 if num[T] = size[T]
5     then allocate new-table with 2*size[T] slots
6          insert all items in table[T] into new-table
7          free table[T]
8         table[T] new-table
9         size[T] 2 * size[T]
10 insert x into table[T]
11 num[T] num[T] + 1



    CS 8833       Algorithms
      Load Factor of the Table
 We are using a common heuristic of
  doubling the size of the table when we
  allocate new storage. Why wouldn’t we
  just increase the size of the table by 1?
 Load factor of the table (the percentage
  of the allocated storage that is used)
    – (T)=#of items in table/size of table


    CS 8833   Algorithms
         Analysis of Insertions
   Assumptions
    – Running time is linear in the number of
      items inserted
    – During expansion, line 6 dominates
   Sequence of operations on an initially
    empty table
    – What is the worst case cost of inserting
      items into the table
    CS 8833   Algorithms
What is the cost ci of the ith operation?
If the table is not full ci  1
  else ic  i             i
                   //Copy-1 items and insert item i
Worst case
     n operations each requiring O(n) time resulting 2 ) O(n
                                                     in
        as the cost for all n operations
                                                       -
   But,this is only the case if we expand the table each time
      we don't
     i if i - 1 is a power of 2
     
ci  
     
     
      1          otherwise



 CS 8833        Algorithms
                     Aggregate Analysis
                                            n        lg n
Total cost of n inserts                     ci  n   2j
                                           i =1      j 0

                  x n1  1    n
            
Remember that x                       k

           k 0     x 1
                             lg n1
        lg n         2           1  lg n1
So  2          j
                                      2          1 2lg n2 1 2n  1
        j 0                   2 1
 n                    lg n
 ci  n   2 j  n  2n  3n
i =1                  j 0

Amortized cost of an insert is 3.
     CS 8833                 Algorithms
          Accounting Method
 The accounting argument will help you
  see why the cost per operation is 3.
 The 3 operations each insert pays for
  are
    – inserting the item itself
    – moving itself on expansion
    – moving another item that has already been
      moved

    CS 8833   Algorithms
                  Examples
   Suppose that we have just done an expansion
    so we have m slots and m/2 objects.
   Now for each insert
     – $1 for insert
     – $1 on item inserted
     – $1 on item in the table
   When we are ready to expand the table again,
    every item will be covered with the $1 it will
    cost to move it to the new table

    CS 8833   Algorithms
          A
          B




     Operations
     Insert (C)
     Insert (D)
     Insert (E)

CS 8833       Algorithms
              Potential Method
   Criterion for Potential Method
    – If we can define a potential function  such that
         (Dn)  (D0)                    n

        then the total amortized cost      ciˆ
                                          i1



        is an upper bound on the actual cost
    – In other words if the potential for the ith operation is
      always greater than the starting potential, we always
      pay in advance
    – It is usually convenient to require (D0) = 0 and
      then show that (Di)  for all i
    CS 8833    Algorithms
Potential Method for Dynamic
           Tables
   Define  to:
    – Be 0 immediately after an expansion
    – Build to the table size by the time the table
      is full
   Definition of potential function for a
    dynamic table
    (T) = 2 * num[T] - size[T]


    CS 8833   Algorithms
    Does the function meet our
             criteria?
   Immediately after expansion
     num[T] = size[T]/2
     (T) = 2*(size[T] / 2) - size[T] = 0
   Immediately before expansion
     num[T] = size[T]
     (T) = 2*size[T] - size[T] = size[T] = num[T]
     so enough potential built up to pay for all insertions
   Since the table is always at least 1/2 full, (T) is
    always non-negative

    CS 8833     Algorithms
    Amortized analysis of insert
     using potential method
   There are two cases that must be
    considered
    – An insertion that does not trigger an
      expansion
    – An insertion that does trigger an expansion
   We need to compute the amortized cost
    for both cases

    CS 8833   Algorithms
            No expansion
  ci  ci   i   i1
  ˆ
      1 (2* num  size)  (2* num1  size1 )
                   i    i          i        i

      1 (2* num  size)  (2* (num  1)  size)
                 i     i            i          i

     = 1 (2* num  2* num)  (size  size)  2
                 i        i       i      i

     =3




CS 8833    Algorithms
                 With Expansion
ci  ci   i   i1
ˆ
    num  (2* num  size)  (2* num1  size )
        i         i     i           i       i1

                        of i
Put everything in terms num
               i 
Remember that size 2* numi -1 = 2* (numi  1)
   num  (2* num  2* (num  1))  (2* (num  1)  (num  1))
       i         i         i                i           i

   num  2* num  2* num  2 (2* num  2 num  1)
       i        i        i            i        i

   num  2 2num  2 num  1
       i         i        i

  =3

  CS 8833        Algorithms
    Table Insertion and Deletion
   Table insert
    – As described before
   Table delete
    – remove specified item from table
    – may also want to contract size of table
       » copy items to a smaller table
       » free storage of old table


    CS 8833   Algorithms
    Table Insertion and Deletion
   Goals
    – load factor of the table is bounded below
      by a constant
    – amortized cost of a table operation is
      bounded above by a constant
   Assume cost is measured in terms of
    – elementary insertions
    – elementary deletions
    CS 8833   Algorithms
       Strategy for Contraction
   Consider using the same strategy as for
    expansion
    – whenever table is less than 1/2 full,
      contract to 1/2 size (contract when the load
      factor is less than 1/2)
    – remember that we want the amortized cost
      of deletions and insertions to be bounded
      above by a constant

    CS 8833   Algorithms
                   Example
   First n/2 operations are insertions and n
    is a power of 2
    num[T] = size[T] = n/2
   Here are our next n/2 operations
    IDDIIDDIIDD...
    What is the cost of this series of operations?



    CS 8833   Algorithms
              What is wrong?
 After an expansion, we don’t do enough
  deletions to pay for a contraction.
 And, after a contraction, we do not do
  enough insertions to pay for an
  expansion



    CS 8833   Algorithms
              Improved Strategy
 Allow the load factor to drop below 1/2
 Specifically, use a lower bound of 1/4
    – Contract when the load factor drops below
      1/4
    – Contract the size by 1/2, resulting in a load
      factor of 1/2 after the contraction
   Consistent results of expansion and
    contraction
    – after expansion       =1/2
    – after contraction     =1/2
    CS 8833    Algorithms
    Analyze Using the Potential
             Method
 Let  be 0 immediately after either an
  expansion or a contraction.
 Then we need to build up enough
  potential before another expansion or
  contraction to pay for the expensive
  operation



    CS 8833   Algorithms
              Potential Function
              2  num[T ]  size[T ]    if  [T ]  1
      (T )                                        2

               size[T ] / 2  num[T ]   if  [T ]  1
                                                     2




 Potential of an empty table is 0
 Potential is never negative




    CS 8833      Algorithms
       Observations about the
         Potential Function
 At any point, we need to have enough
  potential to pay for the next operation.
 How much does an operation cost?
    – Assume that the cost of each insertion and
      deletion is 1.
    – If table expands or contracts, must also add the
      cost of copying the items in the table
    – So we must have (Ti) = num[T] potential built up
      before we expand or contract the table

    CS 8833   Algorithms
       Observations continued
   When the load factor is 1/2,
    (Ti)=0
   When load factor is 1
    size[T]=num[T]
    [T] = num[T] (enough potential for copies)
   When load factor is 1/4
    size[T] = 4 num[T]
    [T] = num[T] (enough potential for copies)

    CS 8833   Algorithms
 Cost of a sequence of inserts
          and deletes
ci      Actual cost of the ith operation
ˆ
ci      Amortized cost of the ith operation
num i Number of items in the table after the ith operation
sizei   Size of the table after the ith operation
i      Load factor after the ith operation




     CS 8833     Algorithms
Amortized cost of an insertion
 Case 1: i-1>= 1/2          Analysis
  unchanged
 Case 2: i-1< 1/2
    – Table will not expand after the next
      operation
    – Two instances need to be considered
       »  remains less than 1/2 after insertion
       »  becomes greater than or equal to 1/2 after
        insertion

    CS 8833   Algorithms
               Insertion when
               i-1 and i < 1/2
ci  ci   i   i1
ˆ
   = 1 + (size /2 num )  (size1 /2 num1 )
             i        i         i         i

                                     of
Get everything in terms of i instead-1 i
   = 1 + (size /2 num )  (size /2 (num  1))
             i        i        i         i

   = 1- 1 = 0




CS 8833       Algorithms
                      Insertion when
                 i-1 < 1/2 and i >= 1/2
ci  ci   i   i 1
ˆ
   = 1 + (2  numi  sizei )  ( sizei 1 / 2  numi 1 )
This time it is easier to get everything in terms of i  1
  instead of i
  = 1 + (2  ( numi -1  1)  sizei 1 )  ( sizei 1 / 2  numi 1 )
   = 1 + 2  numi -1  2  sizei 1  sizei 1 / 2  numi 1
    = 3  numi -1  3 sizei 1  3
                    2

    = 3   i 1sizei -1  3 sizei 1  3
                           2

    < 3 sizei -1  3 sizei 1  3
      2            2

   <3
  CS 8833         Algorithms
             Table Delete
Cases
 i-1 < 1/2
     with contraction
     without contraction
 i-1 >= 1/2
     i >= 1/2
     i < 1/2


 CS 8833   Algorithms
                   Deletion when
           i-1 < 1/2 and no contraction

ci  ci   i   i1
ˆ
   = 1 + (sizei /2 num)  (size1 /2 num )
                       i       i          i1

   = 1 + (sizei /2 num)  (size /2 (num  1))
                       i       i         i

   = 1 +1 = 2




 CS 8833        Algorithms
                       Deletion when
                 i-1 < 1/2 and contraction

ci  ci   i   i 1
ˆ
The actual cost will be the cost of moving numi items
and deleting one item        ci  numi + 1
 ci = (numi + 1) + ( size i / 2  numi )  ( sizei 1 / 2  numi 1 )
 ˆ
Since sizei / 2  sizei 1 / 4  numi  1
   = (numi + 1) + ((numi + 1)  numi )  ((2numi  2)  (numi  1))
   =1



   CS 8833          Algorithms

								
To top