Algorithms

Document Sample
Algorithms Powered By Docstoc
					              Algorithms

           Amortized Analysis




CS 8833   Algorithms
          Amortized Analysis
   Method for analyzing algorithms that
    involve a series of operations on a data
    structure
    – some operations are more expensive than
      others
    – the cost of the expensive operations is
      balanced by the cost of the less expensive
      operations
    – worst case cost per operation is less than
      the cost of the most expensive operation
    CS 8833   Algorithms
         A Business Analogy
 The term amortization comes from the
  business world
 Problem: I pay my bills once per month.
  One particular bill is $600 every six
  months. What is the monthly cost of the
  bill in the worst case?
         if monthi mod 6 = 0
              costi  600
         else costi 0

    CS 8833   Algorithms
                    Answer
   Worst case cost for n months?



   Is this an upper bound on the worst
    case cost for n months?

   Is this a good upper bound?
    CS 8833   Algorithms
                Amortized Cost
   The average cost per month (amortized
    cost) c  T(n)  600(n div 6)
                   n          n

              So if n= 12
                        1200
                T(n)        100/ month
                         12
   Analysis method used–aggregate
    method. Amortized cost is applied to
    all operations
    CS 8833     Algorithms
           Accounting Method
   I will pay more than I need to some months
    and specify when I will use the extra money.
   Each operation has a cost ($100 in this case).
    When I pay $100 and need 0, I will apply that
    $100 to the next month that the bill is due. I
    accumulate a credit in my account.
   Constraint–the balance in the account must
    never be negative.
   Must associate the extra money with a
    particular part of my data structure (a
    particular month in this case)
    CS 8833   Algorithms
              Potential Method
   Like the accounting method except that credit
    is not associated with a particular part of the
    data structure, but with a pool of potential
    energy for the entire data structure.
   Potential energy – like an escrow account
   Each operation has a cost. If the cost of an
    operation is more than the energy associated
    with the operation, I draw on the potential
    energy pool for the excess
   The potential can never be negative
    CS 8833    Algorithms
                           Notation
    Di         data structure after ith operation
           potential function
     (Di ) potential associated with data structure D
                   after ith operation
    ci         actual cost of each operation
    ˆ
    ci              amortized cost of ith operation
    ci  ci +  ( Di ) -  ( Di -1 )
    ˆ
     n          n

     c   (c   ( D )   ( D
      ˆ
    i =1
           i
               i 1
                       i       i       i 1   ))

         n 
         ci    ( Dn )   ( D0 )
         i 1 
CS 8833 Algorithms
              Stack Operations
   Operations
    PUSH(s,x)
    POP(s)
    MULTIPOP(s,k)
     where s is the stack
           x is the item pushed onto the stack
           k is the number of items popped in a
              MULTIPOP operation
    CS 8833   Algorithms
             MULTIPOP
    MULTIPOP(s,k)
    1 while not STACK-EMPTY(s) and k°0
    2     do POP(s)
    3        k k-1




CS 8833   Algorithms
      COST OF OPERATIONS
   ACTUAL COST
    – PUSH(s,x)            1
    – POP(s)               1
    – MULTIPOP(s,k)        min(k,s)
 For a series of n operations, the worst
  case cost per operation is O(n) when all
  n items are popped with a MULTIPOP
 A cost of O(n) for n operations yields a
  total cost for the sequence of O(n2)
    CS 8833   Algorithms
               Tighter Bound
   Aggregate method
    – You can’t POP more objects than you
      PUSH
    – So for n operations, the total cost cannot
      be more than 2n
    – This gives an upper bound of O(n) for the
      cost of a sequence of n operations
    – Amortized cost per operation
        T(n)/n = 1
    CS 8833   Algorithms
       Accounting Method
         Actual cost     Amortized cost
PUSH          1               2
POP           1               0
MULTIPOP      min(k,s)        0




 CS 8833   Algorithms
                 Description
 Let a dollar represent each unit cost.
 When you push and object, pay $1 for
  the push and keep $1 with the object on
  the stack
 When you perform pop operations, you
  can use the $1 associated with the
  object to pay the cost of popping each
  object
    CS 8833   Algorithms
        Amortized Cost using
         Accounting Method
 Sequence of n operations in the worst
  case (n pushes) will cost 2n
 Amortized cost of n pushes is (n) in
  worst case
 Amortized cost of a single operation is
  


    CS 8833   Algorithms
              Potential Method
 They key is to come up with a good
  potential function
 For stack operations, we know that for
  any object the cost of pushing it will be
  1 and the cost of popping it will be one,
  so for processing n items, the max cost
  per item is 2

    CS 8833   Algorithms
             Potential Method

 potential function - number of items on stack
Empty stack  (D 0 ) = 0
Two constraint s
   (Di )  0
    (Di )   (D 0 )

   CS 8833   Algorithms
Amortized Cost of Operations
Amortized cost of operations
PUSH
ci  ci   ( Di )   ( Di 1 )  1  1  2
ˆ
MULTIPOP - Let k' = min(k, s)
ci  ci   ( Di )   ( Di 1 )  k ' k '  0
ˆ
POP
ci  ci   ( Di )   ( Di 1 )  1  1  0
 ˆ
Worst case cost of n operations is 2n = O(n)
 CS 8833      Algorithms
              Dynamic Tables
   Must enlarge the size of a table as needed.
    We also want to be able to decrease the size
    of the table if many elements are removed
   Dynamically expanding and contracting a
    table
   Some insertions and deletions are very
    expensive if they trigger an expansion or
    contraction
   Will show amortized cost of insertions and
    deletions is O(1)
    CS 8833   Algorithms
                Assumptions
 T is an object representing a table
 table[T] is a pointer to the block of
  storage representing the table
 num[T] is the number of items in the
  table
 size[T] is the total number of slots in the
  table
 Initially, the table is empty
    num[T] = size[T] = 0
    CS 8833   Algorithms
TABLE-INSERT(T,x)
1 if size[T] = 0
2    then allocate table[T] with 1 slot
3          size[T]  1
4 if num[T] = size[T]
5     then allocate new-table with 2*size[T] slots
6          insert all items in table[T] into new-table
7          free table[T]
8         table[T] new-table
9         size[T] 2 * size[T]
10 insert x into table[T]
11 num[T] num[T] + 1



    CS 8833       Algorithms
      Load Factor of the Table
 We are using a common heuristic of
  doubling the size of the table when we
  allocate new storage. Why wouldn’t we
  just increase the size of the table by 1?
 Load factor of the table (the percentage
  of the allocated storage that is used)
    – (T)=#of items in table/size of table


    CS 8833   Algorithms
         Analysis of Insertions
   Assumptions
    – Running time is linear in the number of
      items inserted
    – During expansion, line 6 dominates
   Sequence of operations on an initially
    empty table
    – What is the worst case cost of inserting
      items into the table
    CS 8833   Algorithms
What is the cost c of the ith operation?
                 i

If the table is not full ic  1
  else ic i             i
                  //Copy-1 items and insert item i
Worst case
                                                   2
     n operations each requiring O(n) time resulting) in O(n
         as the cost for all n operations
                                                     -
   But,this is only the case if we expand the table each time
      we don't
     i if i- 1 is a power of 2
     
ci  
     
     
      1         otherwise



 CS 8833        Algorithms
                     Aggregate Analysis
                                            n       lg n
Total cost of n inserts  c i  n   2 j
                                           i =1     j 0

                  x n1  1    n
           
Remember that x                       k

           k 0     x 1
                       2
                             lg n1
        lg n                      1  lg n1
So  2          j
                                      2          1 2lg n2 1 2n  1
        j 0                   2 1
 n                    lg n
 ci  n   2 j  n  2n  3n
i =1                  j 0

Amortized cost of an insert is 3.
     CS 8833                 Algorithms
          Accounting Method
 The accounting argument will help you
  see why the cost per operation is 3.
 The 3 operations each insert pays for
  are
    – inserting the item itself
    – moving itself on expansion
    – moving another item that has already been
      moved

    CS 8833   Algorithms
                  Examples
   Suppose that we have just done an expansion
    so we have m slots and m/2 objects.
   Now for each insert
     – $1 for insert
     – $1 on item inserted
     – $1 on item in the table
   When we are ready to expand the table again,
    every item will be covered with the $1 it will
    cost to move it to the new table

    CS 8833   Algorithms
          A
          B




     Operations
     Insert (C)
     Insert (D)
     Insert (E)

CS 8833       Algorithms
              Potential Method
   Criterion for Potential Method
    – If we can define a potential function  such that
         (Dn)  (D0)                    n

        then the total amortized cost      ciˆ
                                          i1



        is an upper bound on the actual cost
    – In other words if the potential for the ith operation is
      always greater than the starting potential, we always
      pay in advance
    – It is usually convenient to require (D0) = 0 and
      then show that (Di)  for all i
    CS 8833    Algorithms
Potential Method for Dynamic
           Tables
   Define  to:
    – Be 0 immediately after an expansion
    – Build to the table size by the time the table
      is full
   Definition of potential function for a
    dynamic table
    (T) = 2 * num[T] - size[T]


    CS 8833   Algorithms
    Does the function meet our
             criteria?
   Immediately after expansion
     num[T] = size[T]/2
     (T) = 2*(size[T] / 2) - size[T] = 0
   Immediately before expansion
     num[T] = size[T]
     (T) = 2*size[T] - size[T] = size[T] = num[T]
     so enough potential built up to pay for all insertions
   Since the table is always at least 1/2 full, (T) is
    always non-negative

    CS 8833     Algorithms
    Amortized analysis of insert
     using potential method
   There are two cases that must be
    considered
    – An insertion that does not trigger an
      expansion
    – An insertion that does trigger an expansion
   We need to compute the amortized cost
    for both cases

    CS 8833   Algorithms
            No expansion
  ci  ci   i   i1
  ˆ
      1 (2* num  size)  (2* num1  size1 )
                   i    i          i        i

      1 (2* num  size)  (2* (num  1)  size)
                 i     i            i          i

     = 1 (2* num  2* num)  (size  size)  2
                 i        i       i      i

     =3




CS 8833    Algorithms
              With Expansion
ci  ci   i   i1
ˆ
    num  (2* num  size)  (2* num1  size )
          i           i i           i       i1

                        of i
Put everything in termsnum
               i 
Remember that size 2* num -1 = 2* (num  1)
                         i            i

   num  (2* num  2* (num  1))  (2* (num  1)  (num  1))
       i         i         i                i           i

   num  2* num  2* num  2 (2* num  2 num  1)
       i        i        i            i        i

  num  2 2num  2 num  1
      i         i        i

 =3

  CS 8833     Algorithms
    Table Insertion and Deletion
   Table insert
    – As described before
   Table delete
    – remove specified item from table
    – may also want to contract size of table
       » copy items to a smaller table
       » free storage of old table


    CS 8833   Algorithms
    Table Insertion and Deletion
   Goals
    – load factor of the table is bounded below
      by a constant
    – amortized cost of a table operation is
      bounded above by a constant
   Assume cost is measured in terms of
    – elementary insertions
    – elementary deletions
    CS 8833   Algorithms
       Strategy for Contraction
   Consider using the same strategy as for
    expansion
    – whenever table is less than 1/2 full,
      contract to 1/2 size (contract when the load
      factor is less than 1/2)
    – remember that we want the amortized cost
      of deletions and insertions to be bounded
      above by a constant

    CS 8833   Algorithms
                   Example
   First n/2 operations are insertions and n
    is a power of 2
    num[T] = size[T] = n/2
   Here are our next n/2 operations
    IDDIIDDIIDD...
    What is the cost of this series of operations?



    CS 8833   Algorithms
              What is wrong?
 After an expansion, we don’t do enough
  deletions to pay for a contraction.
 And, after a contraction, we do not do
  enough insertions to pay for an
  expansion



    CS 8833   Algorithms
              Improved Strategy
 Allow the load factor to drop below 1/2
 Specifically, use a lower bound of 1/4
    – Contract when the load factor drops below
      1/4
    – Contract the size by 1/2, resulting in a load
      factor of 1/2 after the contraction
   Consistent results of expansion and
    contraction
    – after expansion       =1/2
    – after contraction     =1/2
    CS 8833    Algorithms
    Analyze Using the Potential
             Method
 Let  be 0 immediately after either an
  expansion or a contraction.
 Then we need to build up enough
  potential before another expansion or
  contraction to pay for the expensive
  operation



    CS 8833   Algorithms
              Potential Function
               2  num[T ]  size[T ]    if  [T ]  1
       (T )                                        2

                size[T ] / 2  num[T ]   if  [T ]  1
                                                      2




 Potential of an empty table is 0
 Potential is never negative




    CS 8833      Algorithms
       Observations about the
         Potential Function
 At any point, we need to have enough
  potential to pay for the next operation.
 How much does an operation cost?
    – Assume that the cost of each insertion and
      deletion is 1.
    – If table expands or contracts, must also add the
      cost of copying the items in the table
    – So we must have (Ti) = num[T] potential built up
      before we expand or contract the table

    CS 8833   Algorithms
       Observations continued
   When the load factor is 1/2,
    (Ti)=0
   When load factor is 1
    size[T]=num[T]
    [T] = num[T] (enough potential for copies)
   When load factor is 1/4
    size[T] = 4 num[T]
    [T] = num[T] (enough potential for copies)

    CS 8833   Algorithms
 Cost of a sequence of inserts
          and deletes
ci      Actual cost of the ith operation
ˆ
ci      Amortized cost of the ith operation
numi Number of items in the table after the ith operation
sizei   Size of the table after the ith operation
i      Load factor after the ith operation




     CS 8833    Algorithms
Amortized cost of an insertion
 Case 1: i-1>= 1/2          Analysis
  unchanged
 Case 2: i-1< 1/2
    – Table will not expand after the next
      operation
    – Two instances need to be considered
       »  remains less than 1/2 after insertion
       »  becomes greater than or equal to 1/2 after
        insertion

    CS 8833   Algorithms
               Insertion when
               i-1 and i < 1/2
ci  ci   i   i1
ˆ
   = 1 + (size /2 num)  (size1 /2 num1 )
             i        i        i         i

                                   -1
Get everything in terms of i instead of i
  = 1 + (size /2 num)  (size /2 (num  1))
              i        i        i         i

   = 1- 1 = 0




CS 8833       Algorithms
                      Insertion when
                 i-1 < 1/2 and i >= 1/2
ci  ci   i   i 1
ˆ
   = 1 + (2  num i  sizei )  ( sizei 1 / 2  numi 1 )
This time it is easier to get everything in terms of i  1
   instead of i
   = 1 + (2  ( num i -1  1)  sizei 1 )  ( sizei 1 / 2  numi 1 )
   = 1 + 2  num i -1  2  sizei 1  sizei 1 / 2  numi 1
    = 3  num i -1  3 sizei 1  3
                     2

    = 3   i 1sizei -1  3 sizei 1  3
                           2

    < 3 sizei -1  3 sizei 1  3
      2            2

   <3
  CS 8833         Algorithms
             Table Delete
Cases
 i-1 < 1/2
     with contraction
     without contraction
 i-1 >= 1/2
     i >= 1/2
     i < 1/2


 CS 8833   Algorithms
                 Deletion when
         i-1 < 1/2 and no contraction

ci  ci   i   i1
ˆ
   = 1 + (sizei /2 num)  (size1 /2 num )
                       i        i         i1

  = 1 + (sizei /2 num)  (size /2 (num  1))
                      i       i         i

  = 1 +1 = 2




 CS 8833     Algorithms
                       Deletion when
                 i-1 < 1/2 and contraction

ci  ci   i   i 1
ˆ
The actual cost will be the cost of moving num i items
and deleting one item         ci  num i + 1
 ci = (num i + 1) + ( size i / 2  numi )  ( sizei 1 / 2  numi 1 )
 ˆ
Since sizei / 2  sizei 1 / 4  numi  1
   = (num i + 1) + ((num i + 1)  numi )  ((2numi  2)  (numi  1))
   =1



   CS 8833          Algorithms

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:12
posted:10/11/2011
language:English
pages:51