Learning Center
Plans & pricing Sign in
Sign Out

Sorting Insertion Sort Merge Sort Quick Sort Heap Sort Sorting


									                                                                       Insertion Sort
Sorting                                                                s   Corresponds to how most
                                                                           people sort cards
                                                                       s   Invariant: everything to left           // Code for sorting a[ ] an array of int
                                                                           is already sorted                       for (int i = 1; i < a.length; i++) {
                                                                       s   Works especially well when                   int temp = a[ i ];
                                                                                                                        int k = i;
                                                                           input is nearly sorted
                                                                                                                        for (; k > 0 && a[k–1] > temp; k – –)
                                                             CS211     s   Runtime                                              a[k] = a[k–1];
                                                                            q Worst-case                                a[k] = temp;
                                                           Fall 2000                                               }
                                                                                 v   O(n2)
                                                                                 v   Consider reverse-
                                                                                     sorted input
                                                                            q   Best-case
                                                                                 v   O(n)
                                                                                 v   Consider sorted input

 Merge Sort                                                            Quick Sort
                                                                       s   Also uses recursion (Divide &           s    Runtime analysis (worst-case)
 s   Uses recursion (Divide &         s   Runtime recurrence               Conquer)                                      q Partition can work badly
     Conquer)                              q Let T(n) be the time to   s   Outline                                         producing this:
 s   Outline (text has detailed              sort an array of size n        q Partition the array
                                                                                                                             p          >p
     code)                                                                  q Recursively sort each piece
                                           q T(n) = 2T(n/2) + O(n)
      q Split array into two halves                                             of the partition                         q   Runtime recurrence
                                           q T(1) = O(1)               s   Partition = divide the array like                  T(n) = T(n–1) + O(n)
      q Recursively sort each half
                                           q Can show by induction         this      <p      p    >p                     q  This can be solved by
      q Merge the two halves
                                             that T(n) = O(n log n)                                                         induction to show T(n) =
                                                                       s   p is the pivot item                              O(n2)
 s   Merge = combine two sorted                                        s   Best pivot choices                      s    Runtime analysis (expected-
     arrays to make a single sorted   s   Alternately, can show T(n)        q middle item
     array                                = O(n log n) by looking at                                                     q More complex recurrence
                                                                            q random item
      q Rule: Always choose the           tree of recursive calls           q median of leftmost, rightmost,
                                                                                                                         q Can solve by induction to
         smallest item                                                          and middle items                            show
                                                                                                                            expected T(n) = O(n log n)
      q Time: O(n)
                                                                                                                   s    Can improve constant factor by
                                                                                                                        avoiding QSort on small sets
 3                                                                     4

 Heap Sort                                                             Sorting Algorithm Summary
 s   Not recursive                    s   Runtime analysis (worst-     s   The ones we have discussed          s       Why so many? Do Computer
 s   Outline                              case)                             q Insertion Sort                           Scientists have some kind of
      q Build heap                                                          q Merge Sort
                                                                                                                       sorting fetish or what?
                                           q O(n) time to build heap
                                                                                                                        q Stable sorts: Ins, Mer
      q Perform removeMax on                 (using bottom-up               q Quick Sort
         heap until empty                                                                                               q Worst-case O(n log n): Mer,
                                             approach)                      q Heap Sort
      q Note that items are
                                           q O(log n) time (worst-
         removed from heap in                                                                                           q Expected-case O(n log n):
                                             case) for each removal    s   Other sorting algorithms
         sorted order                                                                                                      Mer, Hea, Qui
                                           q Total time: O(n log n)         q Selection Sort
 s   Heap Sort is the only                                                                                              q Best for nearly-sorted sets:
                                                                            q Shell Sort (in text)                         Ins
     O(n log n) sort that uses no
     extra space                                                            q Bubble Sort                               q No extra space needed: Ins,

      q Merge Sort uses extra                                               q Radix Sort                                   Hea
         array during merge                                                 q Bin Sort                                  q Fastest in practice: Qui

      q Quick Sort uses recursive                                           q Counting Sort                             q Least data movement: Sel
 5                                                                     6

Lower Bounds on Sorting: Goals                                                     Comparison Trees
s    Goal: Determine the            s    But how can we prove                      s Any algorithm can be                      s   In general, you get a
     minimum time required to            anything about the best                     “unrolled” to show the                        comparison tree
     sort n items                        possible algorithm?                         comparisons that are
                                                                                                                               s   If the algorithm fails to
                                                                                     (potentially) performed
s    Note: we want worst-case                                                                                                      terminate for some input
     not best-case time                      q    We want to find                                                                  then the comparison tree is
                                                                                          for (int i = 0; i < x.length; i++)
      q Best-case doesn’t tell us                 characteristics that are                                                         infinite
                                                                                             if (x[i] < 0) x[i] = – x[i];
        much; for example, we                     common to all sorting                                                        s   The height of the
        know Insertion Sort                       algorithms                                                                       comparison tree represents
        takes O(n) time on                                                              0 < length                x[1] < 0         the worst-case number of
        already-sorted input                 q    Let’s try looking at                                                             comparisons for that
      q We want to determine                      comparisons                           x[0] < 0                 2 < length        algorithm
        the worst-case time for
        the best-possible                                                               1 < length                x[2] < 0
 7                                                                                  8

Lower Bounds on Sorting: Notation                                                  The Answer to a Sorting Problem
s    Suppose we want to sort the items in the array B[ ]                           s     An answer for a sorting problem tells where each of the ai
                                                                                         resides when the algorithm finishes
                                                                                   s     How many answers are possible?
s    Let’s name the items
      q a1 is the item initially residing in B[1], a2 is the                       s     The correct answer depends on the actual values
        item initially residing in B[2], etc.                                            represented by each ai
      q In general, ai is the item initially stored in B[i]                        s     Since we don’t know what the ai are going to be, it has to be
                                                                                         possible to produce each permutation of the ai

s    Rule: an item keeps its name forever, but it can                              s     For a sorting algorithm to be valid it must be possible for that
     change its location                                                                 algorithm to give any of n! potential answers
      q Example: after swap(B,1,5), a1 is stored in B[5]
        and a5 is stored in B[1]
 9                                                                                 10

Comparison Tree for Sorting                                                        Time vs. Height
s    Every sorting algorithm has     s   Comparison tree for sorting               s     The worst-case time for a             s   What is the minimum possible
     a corresponding                     n items:                                        sorting method must be ≥                  height for a binary tree with n!
     comparison tree                                                                     the height of its comparison              leaves?
      q Note that other stuff                                                            tree                                       Height ≥ log(n!) = Θ(n log n)
        happens during the                                                                q The height corresponds
        sorting algorithm, we                                                                to the worst-case                 s   This implies that any
        just aren’t showing it in                                                            number of comparisons                 comparison-based sorting
        the tree                                           comparison                                                              algorithm must have a worst-
                                                                                          q Each comparison takes
                                                              tree                                                                 case time of Ω(n log n)
s    The comparison tree must                                                                Θ(1) time
                                                                                                                                    q Note: this is a lower
     have n! (or more) leaves       abc...       bacd...                 cabd...          q The algorithm is doing                    bound; thus, the use of
     because a valid sorting                                                                 more than just                           big-Omega instead of
     algorithm must be able to                                                               comparisons                              big-O
     get any of n! possible                                 n! leaves
11                                                                                 12

Using the Lower Bound on Sorting                                     Sorting in Linear Time
Claim: I have a PQ               Claim: I have a PQ                  There are several sorting        s   How do these methods get
    q Insert time: O(1)              q Insert time: O(loglog n)        methods that take linear           around the Ω(n log n) lower
    q GetMax time: O(1)              q GetMax time: O(loglog
                                                                       time                               bound?
                                       n)                                                                  q They don’t use
s True or false?
                                 s True or false?                    s    Counting Sort                      comparisons
                                                                           q sorts integers from a
False (for general sets)
  because if such a PQ           False (for general sets)                    small range: [0..k]      s   What sorting method works
  existed, it could be used to     because it could be used to               where k = O(n)               best?
  sort in time O(n)                sort in time O(n loglog n)        s    Radix Sort                       q QuickSort is best

                                 True for items with priorities in         q the method used by the          general-purpose sort
                                    range 1..n [van Emde Boas]               old card-sorters              q Counting Sort or Radix
                                    (Note: such a set can be               q sorting time O(dn)              Sort can be best for
                                    sorted in O(n) time)                     where d is the number           some kinds of data
                                                                             of “digits”
13                                                                   14


To top