Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out

COMP 150

VIEWS: 3 PAGES: 32

									UMass Lowell Computer Science 91.503
     Analysis of Algorithms
          Prof. Karen Daniels
               Fall, 2001


           Lecture 9
        Tuesday, 11/20/01
     Parallel Algorithms
        Chapters 28, 30
       Relevant Sections of Chapters
       28-30
                          You’re responsible for material in
Ch28                      this chapter that we discuss in
Sorting Networks          lecture. (Note that this includes
                          all sections 28.1 - 28.5.)


                          You’re not responsible for any of
Ch29                      the material in this chapter. We
Arithmetic Circuits       will not be discussing it in
                          lecture.

                          You’re responsible for material in
Ch30                      this chapter that we discuss in
Algorithms for Parallel   lecture. (Note that this includes
Computers                 only sections 30.1 - 30.2.)
          Overview

   Sorting Networks
       Comparison Networks
       0-1 Principle
       Bitonic Sorting Network
       Merging Network
       Sorting Network
   Algorithms for Parallel Computers
       PRAM Model
       Pointer Jumping
       CRCW Algorithms vs. EREW Algorithms
  Sorting Networks
     Chapter 28
 Comparison Networks
     0-1 Principle
Bitonic Sorting Network
   Merging Network
    Sorting Network
        Comparison Networks:
        Definition
Comparison Network only performs comparisons.
      Comparisons may occur in parallel.
Comparison Network contains only comparators & wires.

    2-input comparator:

 input wires




                          output wires



                                    source: 91.503 textbook Cormen et al.
                         Comparison Networks:
                         Definition (continued)
Running Time:
 Comparator
uses Q(1) time.
Define time using
   wire depth.                         Graph of
                                   interconnections
Input wire has                     must be acyclic.
  depth = 0.
 Comparator
with input wire
 depths dx, dy
  has output
wire depths =
  max( d x , d y )  1

    Depth of
  comparison
 network = max
   depth of a
  comparator.                                     source: 91.503 textbook Cormen et al.
       Sorting Network:
       Definition

   Sorting Network:
     Comparison  Network for which output
      sequence is monotonically increasing



     Example:




                                 source: 91.503 textbook Cormen et al.
              Sorting Network:
              Structure
Families of                                            COMPARATORs
Comparison
 Networks
                                                      HALF-CLEANERs
  Recursive                                 Bitonic
  Structure                                Sorting
  “Parallel                                Networks   BITONIC-SORTERs
 MergeSort”          Sorting    Merging
  Strategy                      Networks
                     Networks



  Sort n values in                                       MERGERs
   O( lg2n ) time

                                                         SORTERs
          0-1 Principle
    If sorting network works correctly for {0,1} inputs,
    it works correctly on arbitrary input numbers.
            allows us to limit attention to {0,1} inputs
Proof relies on function monotonicity:




                                              source: 91.503 textbook Cormen et al.
               0-1 Principle (continued)

Proof of Lemma 28.1:
  f monotonically increasing
       comparator with
   inputs f(x), f(y) produces
outputs f(min(x,y)), f(max(x,y))




     Induction on wire depth




source: 91.503 textbook Cormen et al.
       0-1 Principle (continued)

                                                   x
   Example applying Lemma 28.1:       f ( x)  f    
                                                   2 
                                                   




                            source: 91.503 textbook Cormen et al.
      0-1 Principle (continued)
If sorting network works correctly for {0,1} inputs,
it works correctly on arbitrary input numbers.



                                                   allows us
                                                     to limit
                                                  attention to
                                                  {0,1} inputs




     source: 91.503 textbook Cormen et al.
              Sorting Network:
              Structure
Families of                                            COMPARATORs
Comparison
 Networks
                                                      HALF-CLEANERs
  Recursive                                 Bitonic
  Structure                                Sorting
  “Parallel                                Networks   BITONIC-SORTERs
 MergeSort”          Sorting    Merging
  Strategy                      Networks
                     Networks



  Sort n values in                                       MERGERs
   O( lg2n ) time

                                                         SORTERs
            Bitonic Sorting Network

   Bitonic Sequence
     monotonically      increases then monotonically decreases
          or can be circularly shifted to conform to this
     Example:   < 1, 4, 6, 8, 3, 2 >
     {0,1} bitonic sequence has structure:
          0i 1j 0k     or      1i 0j 1k     i, j , k  0


   Bitonic Sorter:
     comparison  network that sorts bitonic {0,1} sequences
     will be used to construct Sorting Network

                                              source: 91.503 textbook Cormen et al.
            Bitonic Sorting Network

       Bitonic Sorter uses HALF-CLEANERs
   HALF-CLEANER: - comparison network of depth 1
                   - input line i compared with line i + n/2 for i = 1,2,…,n/2



 Sample
inputs &
outputs:




                                                   source: 91.503 textbook Cormen et al.
Bitonic Sorting Network




                source: 91.503 textbook Cormen et al.
Bitonic Sorting Network




                source: 91.503 textbook Cormen et al.
             Bitonic Sorting Network

              BITONIC-
              SORTER[n/2]
HALF-
CLEANER[n]

              BITONIC-
              SORTER[n/2]




                               source: 91.503 textbook Cormen et al.

                            Recurrence for depth of
                             BITONIC-SORTER[n]
              Sorting Network:
              Structure
Families of                                            COMPARATORs
Comparison
 Networks
                                                      HALF-CLEANERs
  Recursive                                 Bitonic
  Structure                                Sorting
  “Parallel                                Networks   BITONIC-SORTERs
 MergeSort”          Sorting    Merging
  Strategy                      Networks
                     Networks



  Sort n values in                                       MERGERs
   O( lg2n ) time

                                                         SORTERs
      Merging Network
 Merge 2 sorted input sequences into 1 sorted output sequence.
          use modification of BITONIC-SORTER
KEY IDEA: For sorted input sequences X, Y: XYR is bitonic
        can merge X, Y, using BITONIC-SORTER(XYR)
             challenge: perform reversal implicitly




                                      source: 91.503 textbook Cormen et al.
Merging Network


BITONIC-
SORTER[n/2]




BITONIC-
SORTER[n/2]




              source: 91.503 textbook Cormen et al.
              Sorting Network:
              Structure
Families of                                            COMPARATORs
Comparison
 Networks
                                                      HALF-CLEANERs
  Recursive                                 Bitonic
  Structure                                Sorting
  “Parallel                                Networks   BITONIC-SORTERs
 MergeSort”          Sorting    Merging
  Strategy                      Networks
                     Networks



  Sort n values in                                       MERGERs
   O( lg2n ) time

                                                         SORTERs
       Sorting Network
                                        MERGER[2]
      SORTER[n/2]                                   MERGER[4]
                                        MERGER[2]
                          MERGER[n]
                                                                 MERGER[8]
                                        MERGER[2]
       SORTER[n/2]
                                                    MERGER[4]
                                        MERGER[2]




                                               Recurrence for depth of
                                                       SORTER[n]
                                                      0               if n  1     
                                        D(n)                                      
                                                D(n / 2)  lg n if n  2k and k  1




source: 91.503 textbook Cormen et al.
            Algorithms
                 for
        Parallel Computers
            Chapter 30

         PRAM Model
         Pointer Jumping
CRCW Algorithms vs. EREW Algorithms
          PRAM Model

 Need a model for parallel
  computing
 RAM model is serial
 Sorting network (Ch28) too
  restrictive
 Popular model: PRAM
     Parallel   Random Access Machine

                                    source: 91.503 textbook Cormen et al.
           PRAM Model

   Memory Access Policies:
                                                          Section 30.1



                                                             Section 30.2

   Common-CRCW model:
       When processors write “simultaneously” to same
        memory location, they write same value
   Alternatives:


                                            source: 91.503 textbook Cormen et al.
                   Pointer Jumping: List Ranking
                                                    List Ranking Problem: Given singly-linked
                                                    list of n objects, compute, for each object,
                                                             its distance from end of list:




Correctness Invariant: At start of each iteration
of while loop, for each object i, sum of d values
for sublist headed by i = correct d[i]


  Running-Time Invariant: Each step of
  pointer jumping transforms each list into 2
  interleaved lists (even, odd).


              O( lgn ) time
      Work = time x #processors
           Q( nlgn ) work                                           source: 91.503 textbook Cormen et al.
                    Pointer Jumping: Prefix




                 start with x[i]=xk in each object i of the list




                                                                   Correctness Invariant: At end of tth iteration of while
                                                                   loop, kth processor stores [max(1,k-2t +1),k]

  Differences from LIST-RANK
                                        O( lgn ) time              At each, if we perform prefix computation on each
                                                                   existing list, each object obtains correct value.
source: 91.503 textbook Cormen et al.
                   Pointer Jumping: Euler Tour
  Problem: Compute depth of each            1) Construct Euler Tour of a graph: (cycle     O(1)
     node in n-node binary tree.               traversing each edge exactly once.)         time
3 processors per node:
                                            2) Initialize values for each of processor
                                                                                         O(lgn)
                                            3) Parallel Prefix computation using +        time




    source: 91.503 textbook Cormen et al.
               CRCW vs. EREW Algorithms
   Problem where
    concurrent
    reads help:
     Find   identities
       of tree roots in
       a forest




source: 91.503 textbook Cormen et al.
             CRCW vs. EREW Algorithms
   Problem where concurrent writes help:
     Find   maximum element in array of real numbers




                                           source: 91.503 textbook Cormen et al.
CRCW vs. EREW Algorithms




             source: 91.503 textbook Cormen et al.

								
To top