VIEWS: 7 PAGES: 51 POSTED ON: 10/3/2012 Public Domain
Algorithms Amortized Analysis CS 8833 Algorithms Amortized Analysis Method for analyzing algorithms that involve a series of operations on a data structure – some operations are more expensive than others – the cost of the expensive operations is balanced by the cost of the less expensive operations – worst case cost per operation is less than the cost of the most expensive operation CS 8833 Algorithms A Business Analogy The term amortization comes from the business world Problem: I pay my bills once per month. One particular bill is $600 every six months. What is the monthly cost of the bill in the worst case? if monthi mod 6 = 0 costi 600 else costi 0 CS 8833 Algorithms Answer Worst case cost for n months? Is this an upper bound on the worst case cost for n months? Is this a good upper bound? CS 8833 Algorithms Amortized Cost The average cost per month (amortized cost) c T(n) 600(n div 6) n n So if n= 12 1200 T(n) 100/ month 12 Analysis method used–aggregate method. Amortized cost is applied to all operations CS 8833 Algorithms Accounting Method I will pay more than I need to some months and specify when I will use the extra money. Each operation has a cost ($100 in this case). When I pay $100 and need 0, I will apply that $100 to the next month that the bill is due. I accumulate a credit in my account. Constraint–the balance in the account must never be negative. Must associate the extra money with a particular part of my data structure (a particular month in this case) CS 8833 Algorithms Potential Method Like the accounting method except that credit is not associated with a particular part of the data structure, but with a pool of potential energy for the entire data structure. Potential energy – like an escrow account Each operation has a cost. If the cost of an operation is more than the energy associated with the operation, I draw on the potential energy pool for the excess The potential can never be negative CS 8833 Algorithms Notation Di data structure after ith operation potential function (Di ) potential associated with data structure D after ith operation ci actual cost of each operation ˆ ci amortized cost of ith operation ci ci + ( Di ) - ( Di -1 ) ˆ n n c (c ( D ) ( D ˆ i =1 i i 1 i i i 1 )) n ci ( Dn ) ( D0 ) i 1 CS 8833 Algorithms Stack Operations Operations PUSH(s,x) POP(s) MULTIPOP(s,k) where s is the stack x is the item pushed onto the stack k is the number of items popped in a MULTIPOP operation CS 8833 Algorithms MULTIPOP MULTIPOP(s,k) 1 while not STACK-EMPTY(s) and k°0 2 do POP(s) 3 k k-1 CS 8833 Algorithms COST OF OPERATIONS ACTUAL COST – PUSH(s,x) 1 – POP(s) 1 – MULTIPOP(s,k) min(k,s) For a series of n operations, the worst case cost per operation is O(n) when all n items are popped with a MULTIPOP A cost of O(n) for n operations yields a total cost for the sequence of O(n2) CS 8833 Algorithms Tighter Bound Aggregate method – You can’t POP more objects than you PUSH – So for n operations, the total cost cannot be more than 2n – This gives an upper bound of O(n) for the cost of a sequence of n operations – Amortized cost per operation T(n)/n = 1 CS 8833 Algorithms Accounting Method Actual cost Amortized cost PUSH 1 2 POP 1 0 MULTIPOP min(k,s) 0 CS 8833 Algorithms Description Let a dollar represent each unit cost. When you push and object, pay $1 for the push and keep $1 with the object on the stack When you perform pop operations, you can use the $1 associated with the object to pay the cost of popping each object CS 8833 Algorithms Amortized Cost using Accounting Method Sequence of n operations in the worst case (n pushes) will cost 2n Amortized cost of n pushes is (n) in worst case Amortized cost of a single operation is CS 8833 Algorithms Potential Method They key is to come up with a good potential function For stack operations, we know that for any object the cost of pushing it will be 1 and the cost of popping it will be one, so for processing n items, the max cost per item is 2 CS 8833 Algorithms Potential Method potential function - number of items on stack Empty stack (D 0 ) = 0 Two constraints (Di ) 0 (Di ) (D 0 ) CS 8833 Algorithms Amortized Cost of Operations Amortized cost of operations PUSH ci ci ( Di ) ( Di 1 ) 1 1 2 ˆ MULTIPOP - Let k' = min(k, s) ci ci ( Di ) ( Di 1 ) k ' k ' 0 ˆ POP ci ci ( Di ) ( Di 1 ) 1 1 0 ˆ Worst case cost of n operations is 2n = O(n) CS 8833 Algorithms Dynamic Tables Must enlarge the size of a table as needed. We also want to be able to decrease the size of the table if many elements are removed Dynamically expanding and contracting a table Some insertions and deletions are very expensive if they trigger an expansion or contraction Will show amortized cost of insertions and deletions is O(1) CS 8833 Algorithms Assumptions T is an object representing a table table[T] is a pointer to the block of storage representing the table num[T] is the number of items in the table size[T] is the total number of slots in the table Initially, the table is empty num[T] = size[T] = 0 CS 8833 Algorithms TABLE-INSERT(T,x) 1 if size[T] = 0 2 then allocate table[T] with 1 slot 3 size[T] 1 4 if num[T] = size[T] 5 then allocate new-table with 2*size[T] slots 6 insert all items in table[T] into new-table 7 free table[T] 8 table[T] new-table 9 size[T] 2 * size[T] 10 insert x into table[T] 11 num[T] num[T] + 1 CS 8833 Algorithms Load Factor of the Table We are using a common heuristic of doubling the size of the table when we allocate new storage. Why wouldn’t we just increase the size of the table by 1? Load factor of the table (the percentage of the allocated storage that is used) – (T)=#of items in table/size of table CS 8833 Algorithms Analysis of Insertions Assumptions – Running time is linear in the number of items inserted – During expansion, line 6 dominates Sequence of operations on an initially empty table – What is the worst case cost of inserting items into the table CS 8833 Algorithms What is the cost ci of the ith operation? If the table is not full ci 1 else ic i i //Copy-1 items and insert item i Worst case n operations each requiring O(n) time resulting 2 ) O(n in as the cost for all n operations - But,this is only the case if we expand the table each time we don't i if i - 1 is a power of 2 ci 1 otherwise CS 8833 Algorithms Aggregate Analysis n lg n Total cost of n inserts ci n 2j i =1 j 0 x n1 1 n Remember that x k k 0 x 1 lg n1 lg n 2 1 lg n1 So 2 j 2 1 2lg n2 1 2n 1 j 0 2 1 n lg n ci n 2 j n 2n 3n i =1 j 0 Amortized cost of an insert is 3. CS 8833 Algorithms Accounting Method The accounting argument will help you see why the cost per operation is 3. The 3 operations each insert pays for are – inserting the item itself – moving itself on expansion – moving another item that has already been moved CS 8833 Algorithms Examples Suppose that we have just done an expansion so we have m slots and m/2 objects. Now for each insert – $1 for insert – $1 on item inserted – $1 on item in the table When we are ready to expand the table again, every item will be covered with the $1 it will cost to move it to the new table CS 8833 Algorithms A B Operations Insert (C) Insert (D) Insert (E) CS 8833 Algorithms Potential Method Criterion for Potential Method – If we can define a potential function such that (Dn) (D0) n then the total amortized cost ciˆ i1 is an upper bound on the actual cost – In other words if the potential for the ith operation is always greater than the starting potential, we always pay in advance – It is usually convenient to require (D0) = 0 and then show that (Di) for all i CS 8833 Algorithms Potential Method for Dynamic Tables Define to: – Be 0 immediately after an expansion – Build to the table size by the time the table is full Definition of potential function for a dynamic table (T) = 2 * num[T] - size[T] CS 8833 Algorithms Does the function meet our criteria? Immediately after expansion num[T] = size[T]/2 (T) = 2*(size[T] / 2) - size[T] = 0 Immediately before expansion num[T] = size[T] (T) = 2*size[T] - size[T] = size[T] = num[T] so enough potential built up to pay for all insertions Since the table is always at least 1/2 full, (T) is always non-negative CS 8833 Algorithms Amortized analysis of insert using potential method There are two cases that must be considered – An insertion that does not trigger an expansion – An insertion that does trigger an expansion We need to compute the amortized cost for both cases CS 8833 Algorithms No expansion ci ci i i1 ˆ 1 (2* num size) (2* num1 size1 ) i i i i 1 (2* num size) (2* (num 1) size) i i i i = 1 (2* num 2* num) (size size) 2 i i i i =3 CS 8833 Algorithms With Expansion ci ci i i1 ˆ num (2* num size) (2* num1 size ) i i i i i1 of i Put everything in terms num i Remember that size 2* numi -1 = 2* (numi 1) num (2* num 2* (num 1)) (2* (num 1) (num 1)) i i i i i num 2* num 2* num 2 (2* num 2 num 1) i i i i i num 2 2num 2 num 1 i i i =3 CS 8833 Algorithms Table Insertion and Deletion Table insert – As described before Table delete – remove specified item from table – may also want to contract size of table » copy items to a smaller table » free storage of old table CS 8833 Algorithms Table Insertion and Deletion Goals – load factor of the table is bounded below by a constant – amortized cost of a table operation is bounded above by a constant Assume cost is measured in terms of – elementary insertions – elementary deletions CS 8833 Algorithms Strategy for Contraction Consider using the same strategy as for expansion – whenever table is less than 1/2 full, contract to 1/2 size (contract when the load factor is less than 1/2) – remember that we want the amortized cost of deletions and insertions to be bounded above by a constant CS 8833 Algorithms Example First n/2 operations are insertions and n is a power of 2 num[T] = size[T] = n/2 Here are our next n/2 operations IDDIIDDIIDD... What is the cost of this series of operations? CS 8833 Algorithms What is wrong? After an expansion, we don’t do enough deletions to pay for a contraction. And, after a contraction, we do not do enough insertions to pay for an expansion CS 8833 Algorithms Improved Strategy Allow the load factor to drop below 1/2 Specifically, use a lower bound of 1/4 – Contract when the load factor drops below 1/4 – Contract the size by 1/2, resulting in a load factor of 1/2 after the contraction Consistent results of expansion and contraction – after expansion =1/2 – after contraction =1/2 CS 8833 Algorithms Analyze Using the Potential Method Let be 0 immediately after either an expansion or a contraction. Then we need to build up enough potential before another expansion or contraction to pay for the expensive operation CS 8833 Algorithms Potential Function 2 num[T ] size[T ] if [T ] 1 (T ) 2 size[T ] / 2 num[T ] if [T ] 1 2 Potential of an empty table is 0 Potential is never negative CS 8833 Algorithms Observations about the Potential Function At any point, we need to have enough potential to pay for the next operation. How much does an operation cost? – Assume that the cost of each insertion and deletion is 1. – If table expands or contracts, must also add the cost of copying the items in the table – So we must have (Ti) = num[T] potential built up before we expand or contract the table CS 8833 Algorithms Observations continued When the load factor is 1/2, (Ti)=0 When load factor is 1 size[T]=num[T] [T] = num[T] (enough potential for copies) When load factor is 1/4 size[T] = 4 num[T] [T] = num[T] (enough potential for copies) CS 8833 Algorithms Cost of a sequence of inserts and deletes ci Actual cost of the ith operation ˆ ci Amortized cost of the ith operation num i Number of items in the table after the ith operation sizei Size of the table after the ith operation i Load factor after the ith operation CS 8833 Algorithms Amortized cost of an insertion Case 1: i-1>= 1/2 Analysis unchanged Case 2: i-1< 1/2 – Table will not expand after the next operation – Two instances need to be considered » remains less than 1/2 after insertion » becomes greater than or equal to 1/2 after insertion CS 8833 Algorithms Insertion when i-1 and i < 1/2 ci ci i i1 ˆ = 1 + (size /2 num ) (size1 /2 num1 ) i i i i of Get everything in terms of i instead-1 i = 1 + (size /2 num ) (size /2 (num 1)) i i i i = 1- 1 = 0 CS 8833 Algorithms Insertion when i-1 < 1/2 and i >= 1/2 ci ci i i 1 ˆ = 1 + (2 numi sizei ) ( sizei 1 / 2 numi 1 ) This time it is easier to get everything in terms of i 1 instead of i = 1 + (2 ( numi -1 1) sizei 1 ) ( sizei 1 / 2 numi 1 ) = 1 + 2 numi -1 2 sizei 1 sizei 1 / 2 numi 1 = 3 numi -1 3 sizei 1 3 2 = 3 i 1sizei -1 3 sizei 1 3 2 < 3 sizei -1 3 sizei 1 3 2 2 <3 CS 8833 Algorithms Table Delete Cases i-1 < 1/2 with contraction without contraction i-1 >= 1/2 i >= 1/2 i < 1/2 CS 8833 Algorithms Deletion when i-1 < 1/2 and no contraction ci ci i i1 ˆ = 1 + (sizei /2 num) (size1 /2 num ) i i i1 = 1 + (sizei /2 num) (size /2 (num 1)) i i i = 1 +1 = 2 CS 8833 Algorithms Deletion when i-1 < 1/2 and contraction ci ci i i 1 ˆ The actual cost will be the cost of moving numi items and deleting one item ci numi + 1 ci = (numi + 1) + ( size i / 2 numi ) ( sizei 1 / 2 numi 1 ) ˆ Since sizei / 2 sizei 1 / 4 numi 1 = (numi + 1) + ((numi + 1) numi ) ((2numi 2) (numi 1)) =1 CS 8833 Algorithms