VIEWS: 7 PAGES: 23 CATEGORY: Childrens Literature POSTED ON: 12/20/2009 Public Domain
COMP670K Online Algorithm Memory Allocation Hung Kam Po 00030134 tommy@ust.hk Content Content...............................................................................................................2 1........................................................................................................Introduction ............................................................................................................................4 2......................................................................................Multiple Stack System ............................................................................................................................5 2.1.....................................................................................................Introduction ............................................................................................................................5 2.2..........................................................................................Knuth’s Algorithm ............................................................................................................................5 2.2.1.....................................................................................................Algorithm ............................................................................................................................5 2.2.2........................................................................Competitive Ratio Analysis ............................................................................................................................7 2.3.......................................................................................Garwick’s algorithm ............................................................................................................................8 2.3.1.....................................................................................................Algorithm ............................................................................................................................8 2.3.2....................................................................................Worst Case Analysis ..........................................................................................................................10 3..................................................................Classical Dynamic Storage System ..........................................................................................................................14 3.1.....................................................................................................Introduction ..........................................................................................................................14 3.2..........................................................................................First Fit Algorithm ..........................................................................................................................14 3.2.1.....................................................................................................Algorithm ..........................................................................................................................14 3.2.2................................................................First Competitive Ratio Analysis ..........................................................................................................................15 3.2.3...........................................................Second Competitive Ratio Analysis ..........................................................................................................................16 3.3........................................................................................Coloring Algorithm ..........................................................................................................................18 3.3.1.....................................................................................................Algorithm COMP670K Online Algorithm – Memory Allocation 2 ..........................................................................................................................18 3.3.2................................................................First Competitive Ratio Analysis ..........................................................................................................................19 3.3.3...........................................................Second Competitive Ratio Analysis ..........................................................................................................................20 4.........................................................................................................Conclusion ..........................................................................................................................22 5..........................................................................................................References ..........................................................................................................................23 COMP670K Online Algorithm – Memory Allocation 3 1....................................................Introduction Memory allocation is the technique to serve memory requests from different processes to the memory space. A good algorithm can handle requests in the way that it uses the limited memory space efficiently. The cost of the algorithm determines its efficiency, in which memory allocation algorithms focus on two areas: Runtime and Space used. Runtime involves the time for adding items to memory, plus any processes required in memory in handling the new items. Some algorithms, on the other hand, may focus on the total space used for serving same amount of memory requests. The reason of making memory allocation online is that all algorithms must handle sequences of requests without the knowledge of future requests. All decisions made for allocating memory spaces for the new requests are based on the current states, such as the distribution of the occupied blocks. Instead, the offline algorithm mentioned in memory allocation assumes the knowledge of the whole memory request sequence, and then calculates the memory location for each item, and finally allocates the items accordingly. The cost of offline algorithm is the optimal cost and the competitive ratio for each online algorithm is calculated by comparing its cost with the optimal one. The storage system discussed involves two commonly used allocation strategies. They are multiple stacks system and classical dynamic storage system. The ideas of these systems are completely different. Stack system builds up the stacks by moving items in the memory. Classical system fixes the items position in the memory and all new items are places in the remaining empty spaces. For each, two algorithms are introduced with the main steps of proofs of the competitive ratios. Multiple stacks system includes Knuth’ s algorithm [1] and Garwick’ s algorithm [1], while classical dynamic storage system involves First Fit algorithm [2][3][4] and Coloring algorithm [3]. COMP670K Online Algorithm – Memory Allocation 4 2. Multiple Stack System 2.1 Introduction Stack is a simple and useful data structure to allocate items in sequential order. However, one problem must be solved when more than one stack are implemented – overflow. Since the size of each stack is unpredictable, overflow occurs when one of the stacks is full and new items are needed to push in. The solution for this is to reallocate memory and give more empty spaces to the overflowed stack. This action requires the move of items in the memory properly and extra cost is made in addition to the original push operations. Therefore, the objective of this system is to minimize the number of item movements in the memory. As mentioned before, the ideal case of stack system requires the knowledge of the actual push sequence of items. The offline algorithm would first compute total number of items for each stack, and then set each stack accordingly to avoid overflow and finish by pushing items to stacks. This gives the optimal cost as O(m) for m push operations. 2.2 Knuth’s Algorithm 2.2.1 Algorithm Knuth’ s algorithm is a simple algorithm in stack system. In every stack system, each stack i is pointed by BASE[i] and TOP[i] as base and top pointers respectively. Before talking about the algorithm of the Knuth’ s algorithm, assume: n: number of stacks; P: the push sequence; Ij: the item for stack j; xj: the number of items for stack j in P; m: the total items in P. When stack i overflows, the strategy finds the smallest k where i < k < n or the largest COMP670K Online Algorithm – Memory Allocation 5 k where 1 < k < i. If smallest k is chosen, it then moves all items between stack i + 1 to stack k by one entry to the right. Otherwise, it moves items from stack k +1 to stack i by one to left if largest k is chosen. COMP670K Online Algorithm – Memory Allocation 6 Example: Initially, all memory spaces are empty. 1 2 3 4 5 6 7 8 BASE[i] TOP[i] If P = I2 I3 I3 I3 I3 I4, the memory becomes 1 2 3 4 5 6 7 8 2 3 3 3 3 4 TOP[4] BASE[2] TOP[2] BASE[3] TOP[3] BASE[4] After adding I1, overflow occurs and item movements are required. In this case, it has 6 item movements. 1 2 3 4 5 6 7 8 1 2 3 3 3 3 4 TOP[4] BASE[1] TOP[1] TOP[2] BASE[2] BASE[3] TOP[3] BASE[4] 2.2.2 Competitive Ratio Analysis The cost of stack system due to P can be calculated by summing up the number of pushes and the total item movements due to P. When considering the worst case in Knuth’ s algorithm, the sequence P would be pushing items in reverse order of the stack number. That means, if there are totally m items, pushing items to stack n first and then to stack n – 1 and so on. This can be shown as: P = In…In… In-1… In-1……I2…I1…I1 After adding items to stack n, each item for stack n-1 requires xn number of item movements. When adding items for stack n-2, it requires total xn + xn-1 item moves. By continuing this pattern, maximum number of item movements can be calculated as: COMP670K Online Algorithm – Memory Allocation 7 It gives the result that the number of item movements in Knuth’ s algorithm is mproportional to the number of push operations for P. By comparing the optimal cost as O(m) for stack system, it shows that Knuth’ s algorithm is not a competitive algorithm. 2.3 Garwick’s algorithm 2.3.1 Algorithm Garwick’ s algorithm starts with allocating each stack with same initial sizes, except for the last stack if the total size is not divisible by n. After a series of push operations, overflow may occur and, again, reallocation of memory is required. For each overflow, it first calculates the total amount of available space left, and then allocates empty spaces for each stack proportional to its growth in last overflow. Assume: M: the total memory size; P: the push sequence; n: the total number of stack; xkj: the number of items added to stack j in kth overflow; Vk ; The relation between vk and xkj pair can be represented as: n The available space for each stack can be calculated by: remained space remained space x kj n vk n vk COMP670K Online Algorithm – Memory Allocation 8 i i 1 x kj ¥ M v 1 i M k k vi vk j 1 x kj M n Vk M vk vk: the number of items added in kth overflow; Vk: the total number of item movements up to kth overflow, i.e. m: the total items in P. £ ¡ © © 1 1 1 m2 2 n i 1 j § n 1 n i x 1 j xi £ £ x n x n- 1 xn x n- 1 x n- 2 ¤ ¡ ¤ ¨¥ §¦¥ xn ¢ ¡ x 2 x1 k v j 1 j © Vk x kj where 0 , 1, The first term allocates each stack with certain amount of space, regardless the growth of the stack in the kth overflow. The second term gives extra space to each stack, which is proportional to the amount of item xkj added. and are used to control the proportion of these two terms. COMP670K Online Algorithm – Memory Allocation + = 1. 9 Example: After k-1 th overflow, all memory spaces are empty. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 BASE[1] TOP[1] BASE[2] TOP[2] BASE[3] TOP[3] BASE[4] TOP[4] If adding P = I2 I3 I3 I3 I3 I4 I1, the memory becomes 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 BASE[1] 2 TOP[1] BASE[2] 3 TOP[2] BASE[3] 3 3 3 4 TOP[4] TOP[3] BASE[4] After adding I3, kth overflow occurs and item movements are required. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 2 3 3 3 3 3 TOP[3] 4 BASE[4] TOP[4] BASE[1] TOP[1] TOP[2] BASE[3] BASE[2] In this case, it is assumed and are set as 0 and 1 respectively. By equation, remainded space 8 empty space allocated for stack 3 is xk 5 5 3 vk 8 2.3.2 Worst Case Analysis Assume there are only two stacks in the memory, the worst case of Garwick’ s algorithm requires two criteria: 1. After each overflow, remained available spaces are allocated evenly to each stack (by setting = 0) 2. All items are added to 2nd stack, as items in 1st stack do not contribute the item movements. This point explains the first point as setting = 0 gives the smallest amount of empty space to stack 2. This makes the largest chance of having overflows. This means P can be written as: P = I2…I2 I2…I2 I2……I2…I2 COMP670K Online Algorithm – Memory Allocation " ! " ! 10 M v 1 and Since each stack has initially M spaces, v 1 M and similarly, v k 2 2 2 v M 2 i . 1 i proof of the worst case cost requires a lemma shown below. The vk 2 COMP670K Online Algorithm – Memory Allocation $ # # ' &$ % # 11 Lemma Proof 1 1 M 1 2 The cost of the algorithm can be proved by achieving the objective below: Objective: Proof: The total number of movements up to kth stack would be M(k+1/2k) According to above criteria, total number of movements up to kth stack would be: k 1 k k j 1 k 1 2 1 M k 2k Total cost of Garwick’ s algorithm is then summing the amount of push operations and 1 the item movements within the kth overflow, which is m M k . k 2 For finding values of k, it can be done by the equation: M Vk M Vk 0 m V m min 2 vk x kj COMP670K Online Algorithm – Memory Allocation ( ( 7 4 ( 4 4 4 M k 1 1 2 k by sum of G . P . 4 ( ( 4 ( 4 M 1 4 M 1 1 2 3 20 1 V 1 V 2 ... V k i Vk by lemma 1 2 2 1 ( ( v 1 + v 1 + v 2 + ... v 1 + v 2 ...+ v k ... 1 1 2 4 ( 4 8 ( 8 4 8 1 2 3 ... 1 k 8 M 2k k 2k 1 ( ( ( ( ( 4 ( 6 ( 0 Vk M 2 M 22 ... M 2k k 2k 2 k 2k 3 0 31 0 vk 1 4 4 M 2 Vk M 2 M 1 8 9 4 0 8 8 0 8 0 8 1 1 1 2 k 1 4 Assume v k M 2k and V k M 1 1 2 M 2k 8 0 M 1 vk and V k M 1 k 2 2 The case holds for k 1, as v 1 V k k 0 4 0 0 0 M 2 k 1 4 ) ) ) ) ) ) ) 5 k 1 12 1 The Garwick’ s algorithm can be proved as competitive for two stacks only. The reason is that the worst case considered in the proof is valid for two stacks environment only. Systems with more than two stacks possibly exist other push sequences whose total push costs are worse than that calculated here. Therefore, the competitive ratio obtained here is the lower bound for the Garwick’ s algorithm. COMP670K Online Algorithm – Memory Allocation G where k B B @ F @ F G By letting c 1 m 1 , competitive ratio would be k M c 2k log M M m log n n 1 G Combining two values of k, k @ F log M M log 2 m . 1 13 @ A C B min @ @ @ B 1 m M 1 2 log M M m k log 2 M 2 Vk M vk Vk x kj M 1 1 2 D k C B @ F A @ @ Case 2: For m V m min 2 vk @ @ A I k @ F log M M log 2 m @ @ G m Vk m M 1 1 2 M Vk H Case 1: For m - Vk > 0, k 0 @ M Vk , x kj k 1 E 3. Classical Dynamic Storage System 3.1 Introduction Classical Dynamic Storage System refers to those systems that process allocated in memory cannot be moved to a different location. After many insertions and removals of process in memory, there are many holes generated. They are mostly too small to house normal processes and so most of them are wasted. Therefore, the objective of this system is to minimize the wasted space in memory. The cost of this system focuses on the space required for some processes in a period of time. After knowing the whole memory request sequence, the memory space required would be the maximum total sizes of concurrent processes among all time period. If assuming the maximum size of current processes as M, the optimal cost in offline case would be O(M). 3.2 First Fit Algorithm 3.2.1 Algorithm This algorithm serves each memory request by finding the first free area in the memory that can fit the memory requirement. Assume the request requires x blocks of memory, the algorithm serves it by using the first x words of the first gap of x or more words. In other words, the item is allocated in the area which has the lowest possible address. If extending this idea, the locations with high addresses possibly do not contain small items. This idea can be used in analyzing worst case of this algorithm. Small items 0 … … Large items COMP670K Online Algorithm – Memory Allocation 14 3.2.2 First Competitive Ratio Analysis Before starting the proof, three variables are needed to be defined: M: total occupied size in the memory; n: size of inserted item; D(j): distance from start of memory to the end of the last block with size j. The idea of the first lower bound is summarized as the following objective: n Proof: The situation can be considered oppositely that a new block of size n cannot be allocated in the memory. It implies that the size of all holes n M in the first cannot be greater than n. Assuming that all holes have i 1 i ln 2 size n – 1. The situation is shown as follows: Block Size = j Block Size = 1 Block Size = j+1 Block Size = n-1 … 0 … Holes Size = n-1 D(j) Area 1 Area 3 The total occupied sizes can be calculated by considering three areas separately. D 1 1 In area 1, all the space occupied = ; n n In area 3, for each value of j, the space occupied = So, the space occupied for all possible values of n 1 D j 1 .D j j j 1 n j j 1 COMP670K Online Algorithm – Memory Allocation 15 b a ` e b ` D j 1 D j n j C a c db a ` s r q u s q V In area 2, the space occupied = i 1 2 n- 1 ; Y X W M i ln 2 T S R F Q Objective: All blocks with size n can be allocated in first i P 1 M i ln 2 U r t ds r q i p g hf Q P Block Size = n Area 2 D n- 1 n j 1 This shows that the new income block of size n is ruled out by the limitation of maximum M occupied blocks. Therefore, new items with size not larger than n must n M , under the limitation that the total occupied sizes be able to insert in first i 1 i ln 2 in memory is less than M. As , it gives the first competitive ratio of First Fit algorithm as O(log n). n M M 1 log n i 1 i ln 2 3.2.3 Second Competitive Ratio Analysis This analysis requires the use of Interval Graph. A process in the memory can be seen as an interval in the interval graph, with the storage size as the interval weight. The chromatic number of the interval graph, which is equal to the clique number as interval graph is perfect, is the maximum number of concurrent active processes. By using the representation of Interval Graph, memory space can be viewed as: Time 2 3 4 4 Memory Space According to the above graph, the definition of the variables is summarized as follows: M: maximum total occupied size in the memory; n: maximum item size inserted in the memory; k: maximum number of concurrent processes in the memory; COMP670K Online Algorithm – Memory Allocation 16 w j 1 - Number of concurrent processes k = 3 - Maximum total occupied size M = 9 y x j 1 2n 1 i 1 y x y x y x w D 1 n M n v n 1 D j 1 n j D j n M i ln 2 w By summing up all the values, total space occupied in first becomes: j i v 1 M i ln 2 D n 1 y x n W: maximum item size inserted / maximum number of concurrent process (n/k). The proof of the second lower bound requires a lemma. Lemma: Items with size 4W can be inserted in first 5M Proof: As the total concurrent processes in first 5M of memory must not be more than k, the number of holes in first 5M is also not greater than k. Also, the occupied size in first 5M must be, by definition, less than M. Therefore, Total empty size in first 5 M 5M M 4M M n At least one of the holes has size 4 4 4W k k COMP670K Online Algorithm – Memory Allocation 17 The main proof of the second competitive ratio is done by proofing the following statement: Proof: For items with size j < W + 1, it holds by using the lemma. For items with size j W + 1, it can be done by using Proof by Contradiction j M 1 Let L j 5M 0 .3 l W 1 l Assume an item with size j+1 W + 1 cannot be added to first Lj+1. Again, it implies that all holes cannot be greater than size j. Consider for each slot between Li-1 and Li, all items within this slot have the size L Li 1 i. The occupied area in the slot is . Wheni summing all i i j possible values ofj i (between W + 1 and j + 1), total occupied area 1 Li Li 1 , which is proved i be greater than M. It leads to to i j i W 1 contradiction as it means that no more items can be added in the first Lj memory space, but not only items with size j + 1. As the maximum item size is limited to be n, the total space required in First Fit is at n M 1 most 5M .When analyzing this value with the properties 0 .3 l W 1 l n This gives the competitive ratio as O(log k). Combining with previous result, the First Fit algorithm has competitive ratio min[O(log n), O(log k)]. 3.3 Coloring Algorithm 3.3.1 Algorithm It also makes use of the Interval Graph Representation in memory allocation. It includes the new concepts of using Sets. An item with size l is assigned to set r where COMP670K Online Algorithm – Memory Allocation 18 w ~ ~ w | w 5M ln W 1 M w M 1 ln n 0 .3 y w z y l W w { w 5M M 0 .3 1 1 l 5M M 0.3 i 1 1 i y | zx x x f l 1 n ln n 1 l ln n 1 , it gives: n W 1 j 1 1 j M 9 4 ln k 2 . 5 ln k 0.3 m k j d l i h Objective: Item with size j can be inserted in first 5M M 0 .3 l j W 1 1 l v t s u r q g f p o pn e d } 2r-1 l < 2r. Each set is assigned zero or more slots with size 2r each. Once the memory space is assigned to a specific set, the space then belongs to the set forever. So, a location is never allocated to items belonging to different sets. Time As now it is focused in the set instead of the whole memory space, the corresponding variable is defined as: kr: maximum number of concurrent processes in set r; 2M: total occupied size of all sets in the memory; 2n: maximum size occupied for each set; W: n/k. The total occupied size in memory and the maximum size of each set is at most twice the optimal amount of space used in First Fit algorithm. It is because each item added is rounded up to the nearest power of 2 and so, in the worst case, doubles amount of space required. 3.3.2 The calculation of the first competitive ratio involves the proof of the following objective. Objective: The total space used in Coloring Algorithm is at most 2(log(n) + 1)M Proof: The number of distinct set r: 2n = kr * 2r 2r log(2n) log(2r) COMP670K Online Algorithm – Memory Allocation 2r 2r 2 r Set r - Number of concurrent processes kr = 3 - Total occupied = kr * 2 r 2r+1 2r+1 Memory Space First Competitive Ratio Analysis 19 r log(2n) The amount of space used by set r: kr * 2r 2M (a poor bound) The overall space used is at most: 2M * log(2n) = 2 * (log(n) + 1) * M This objective is proved by replacing the space for set r with the maximum possible value. Obviously, size of a whole set is at most the total occupied size of the memory, which is 2M in this case. This gives the first competitive ratio of competitive ratio as log(n). 3.3.3 Second Competitive Ratio Analysis The calculation of the second competitive ratio involves the proof of the following objective: Objective: The total space used in Coloring Algorithm is at most 2log(2k)M COMP670K Online Algorithm – Memory Allocation For 2r W, Number of distinct sets containing items of size log 2n – log W = log 2n – log n/k = log 2k The amount of space used by set r: kr * 2r 2M Total amount of space allocated: 2M * log 2k = 2 * (log (k) + 1) * M Case 2: W: 2k 2n 2 log 2k M 2 log k 1 M n r 0 n k log W k 2r Proof: Case 1: For 2r W, Size for each set r: kr * 2r k * 2r Total amount of space allocated: 2k W M 20 The first case of this proof is done by replacing the number of slots in each set with the maximum possible value. The second case, again, uses the same technique as the first proof of competitive ratio. This gives the second competitive ratio as log(k). Combining two results, the Coloring algorithm has competitive ratio min[O(log n), O(log k)]. COMP670K Online Algorithm – Memory Allocation 21 4. Conclusion This report summarized two main techniques used in memory allocation, with two algorithms introduced for each technique. The main steps of proving the competitive ratio for each algorithm were highlighted. The multiple stack system focuses on the movement of the items in the memory to maintain the stacks. Knuth’ s algorithm performs badly as its item movements in worst push sequence is in proportion to the number of push operations in the push sequence. This makes the Knuth’ s algorithm not be a competitive algorithm. Garwick’ s algorithm was proved to be a competitive algorithm. However, the competitive ratio calculated is only for the system with only two stacks. It may be worse for system with more than two stacks. The classical dynamic storage system concentrates on the efficiency of memory space usage. According to this report, both First Fit and Coloring algorithms gave the same competitive ratio. Indeed, Coloring algorithm performs better than First Fit algorithm. The scenario considered for the calculation of competitive ratio is the worst for the Coloring algorithm but not for the First Fit algorithm. This reports only extracted some of the techniques and there are much more algorithms about memory allocation, such as Best Fit algorithm and even Randomized online algorithm. And, comparing the performance in deterministic and randomized online algorithm in memory allocation makes this topic more interesting. COMP670K Online Algorithm – Memory Allocation 22 5. References [1] B.C. Chien, R.J. Chen, Competitive Analysis of the Online Algorithm for Multiple Stacks Systems, 1992 [2] J.M. Robson, Worst case fragmentation of first fit and best fit storage allocation strategies, 1975 [3] M.G. Luby, J. Naor, A. Orda, Tight Bounds for Dynamic Storage Allocation, 1994 [4] D.R. Woodall, The Bay Restaurant – A Linear Storage Problem, 1974 COMP670K Online Algorithm – Memory Allocation 23