VIEWS: 3 PAGES: 32 POSTED ON: 4/14/2012
UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Lecture 9 Tuesday, 11/20/01 Parallel Algorithms Chapters 28, 30 Relevant Sections of Chapters 28-30 You’re responsible for material in Ch28 this chapter that we discuss in Sorting Networks lecture. (Note that this includes all sections 28.1 - 28.5.) You’re not responsible for any of Ch29 the material in this chapter. We Arithmetic Circuits will not be discussing it in lecture. You’re responsible for material in Ch30 this chapter that we discuss in Algorithms for Parallel lecture. (Note that this includes Computers only sections 30.1 - 30.2.) Overview Sorting Networks Comparison Networks 0-1 Principle Bitonic Sorting Network Merging Network Sorting Network Algorithms for Parallel Computers PRAM Model Pointer Jumping CRCW Algorithms vs. EREW Algorithms Sorting Networks Chapter 28 Comparison Networks 0-1 Principle Bitonic Sorting Network Merging Network Sorting Network Comparison Networks: Definition Comparison Network only performs comparisons. Comparisons may occur in parallel. Comparison Network contains only comparators & wires. 2-input comparator: input wires output wires source: 91.503 textbook Cormen et al. Comparison Networks: Definition (continued) Running Time: Comparator uses Q(1) time. Define time using wire depth. Graph of interconnections Input wire has must be acyclic. depth = 0. Comparator with input wire depths dx, dy has output wire depths = max( d x , d y ) 1 Depth of comparison network = max depth of a comparator. source: 91.503 textbook Cormen et al. Sorting Network: Definition Sorting Network: Comparison Network for which output sequence is monotonically increasing Example: source: 91.503 textbook Cormen et al. Sorting Network: Structure Families of COMPARATORs Comparison Networks HALF-CLEANERs Recursive Bitonic Structure Sorting “Parallel Networks BITONIC-SORTERs MergeSort” Sorting Merging Strategy Networks Networks Sort n values in MERGERs O( lg2n ) time SORTERs 0-1 Principle If sorting network works correctly for {0,1} inputs, it works correctly on arbitrary input numbers. allows us to limit attention to {0,1} inputs Proof relies on function monotonicity: source: 91.503 textbook Cormen et al. 0-1 Principle (continued) Proof of Lemma 28.1: f monotonically increasing comparator with inputs f(x), f(y) produces outputs f(min(x,y)), f(max(x,y)) Induction on wire depth source: 91.503 textbook Cormen et al. 0-1 Principle (continued) x Example applying Lemma 28.1: f ( x) f 2 source: 91.503 textbook Cormen et al. 0-1 Principle (continued) If sorting network works correctly for {0,1} inputs, it works correctly on arbitrary input numbers. allows us to limit attention to {0,1} inputs source: 91.503 textbook Cormen et al. Sorting Network: Structure Families of COMPARATORs Comparison Networks HALF-CLEANERs Recursive Bitonic Structure Sorting “Parallel Networks BITONIC-SORTERs MergeSort” Sorting Merging Strategy Networks Networks Sort n values in MERGERs O( lg2n ) time SORTERs Bitonic Sorting Network Bitonic Sequence monotonically increases then monotonically decreases or can be circularly shifted to conform to this Example: < 1, 4, 6, 8, 3, 2 > {0,1} bitonic sequence has structure: 0i 1j 0k or 1i 0j 1k i, j , k 0 Bitonic Sorter: comparison network that sorts bitonic {0,1} sequences will be used to construct Sorting Network source: 91.503 textbook Cormen et al. Bitonic Sorting Network Bitonic Sorter uses HALF-CLEANERs HALF-CLEANER: - comparison network of depth 1 - input line i compared with line i + n/2 for i = 1,2,…,n/2 Sample inputs & outputs: source: 91.503 textbook Cormen et al. Bitonic Sorting Network source: 91.503 textbook Cormen et al. Bitonic Sorting Network source: 91.503 textbook Cormen et al. Bitonic Sorting Network BITONIC- SORTER[n/2] HALF- CLEANER[n] BITONIC- SORTER[n/2] source: 91.503 textbook Cormen et al. Recurrence for depth of BITONIC-SORTER[n] Sorting Network: Structure Families of COMPARATORs Comparison Networks HALF-CLEANERs Recursive Bitonic Structure Sorting “Parallel Networks BITONIC-SORTERs MergeSort” Sorting Merging Strategy Networks Networks Sort n values in MERGERs O( lg2n ) time SORTERs Merging Network Merge 2 sorted input sequences into 1 sorted output sequence. use modification of BITONIC-SORTER KEY IDEA: For sorted input sequences X, Y: XYR is bitonic can merge X, Y, using BITONIC-SORTER(XYR) challenge: perform reversal implicitly source: 91.503 textbook Cormen et al. Merging Network BITONIC- SORTER[n/2] BITONIC- SORTER[n/2] source: 91.503 textbook Cormen et al. Sorting Network: Structure Families of COMPARATORs Comparison Networks HALF-CLEANERs Recursive Bitonic Structure Sorting “Parallel Networks BITONIC-SORTERs MergeSort” Sorting Merging Strategy Networks Networks Sort n values in MERGERs O( lg2n ) time SORTERs Sorting Network MERGER[2] SORTER[n/2] MERGER[4] MERGER[2] MERGER[n] MERGER[8] MERGER[2] SORTER[n/2] MERGER[4] MERGER[2] Recurrence for depth of SORTER[n] 0 if n 1 D(n) D(n / 2) lg n if n 2k and k 1 source: 91.503 textbook Cormen et al. Algorithms for Parallel Computers Chapter 30 PRAM Model Pointer Jumping CRCW Algorithms vs. EREW Algorithms PRAM Model Need a model for parallel computing RAM model is serial Sorting network (Ch28) too restrictive Popular model: PRAM Parallel Random Access Machine source: 91.503 textbook Cormen et al. PRAM Model Memory Access Policies: Section 30.1 Section 30.2 Common-CRCW model: When processors write “simultaneously” to same memory location, they write same value Alternatives: source: 91.503 textbook Cormen et al. Pointer Jumping: List Ranking List Ranking Problem: Given singly-linked list of n objects, compute, for each object, its distance from end of list: Correctness Invariant: At start of each iteration of while loop, for each object i, sum of d values for sublist headed by i = correct d[i] Running-Time Invariant: Each step of pointer jumping transforms each list into 2 interleaved lists (even, odd). O( lgn ) time Work = time x #processors Q( nlgn ) work source: 91.503 textbook Cormen et al. Pointer Jumping: Prefix start with x[i]=xk in each object i of the list Correctness Invariant: At end of tth iteration of while loop, kth processor stores [max(1,k-2t +1),k] Differences from LIST-RANK O( lgn ) time At each, if we perform prefix computation on each existing list, each object obtains correct value. source: 91.503 textbook Cormen et al. Pointer Jumping: Euler Tour Problem: Compute depth of each 1) Construct Euler Tour of a graph: (cycle O(1) node in n-node binary tree. traversing each edge exactly once.) time 3 processors per node: 2) Initialize values for each of processor O(lgn) 3) Parallel Prefix computation using + time source: 91.503 textbook Cormen et al. CRCW vs. EREW Algorithms Problem where concurrent reads help: Find identities of tree roots in a forest source: 91.503 textbook Cormen et al. CRCW vs. EREW Algorithms Problem where concurrent writes help: Find maximum element in array of real numbers source: 91.503 textbook Cormen et al. CRCW vs. EREW Algorithms source: 91.503 textbook Cormen et al.