# quicks sort

Document Sample

```					                          Chapter 7: Quicksort
Quicksort(A, p, r)                                Partition(A, p, r)
if p < r then                                      x, i := A[r], p – 1;
q := Partition(A, p, r);                      for j := p to r – 1 do
Quicksort(A, p, q – 1);                            if A[j]  x then
Quicksort(A, q + 1, r)                                  i := i + 1;
fi                                                           A[i]  A[j]
fi
A[p..r]                                                       od;
A[i + 1]  A[r];
5                                               return i + 1

A[p..q – 1] A[q+1..r]

Partition                        5

5         5

Jim Anderson                               Comp 750, Fall 2009                          Quicksort - 1
Example
p                  r
initially:          2 5 8 3 9 4 1 7 10 6         note: pivot (x) = 6
i j

next iteration:    2 5 8 3 9 4 1 7 10 6
i j

next iteration:    2 5 8 3 9 4 1 7 10 6
i j

next iteration:    2 5 8 3 9 4 1 7 10 6
i   j

next iteration:    2 5 3 8 9 4 1 7 10 6
i   j

Jim Anderson                     Comp 750, Fall 2009                    Quicksort - 2
Example (Continued)
next iteration:      2 5 3 8 9 4 1 7 10 6
i   j

next iteration:      2 5 3 8 9 4 1 7 10 6
i     j

next iteration:      2 5 3 4 9 8 1 7 10 6
i     j

next iteration:      2 5 3 4 1 8 9 7 10 6
i     j

next iteration:      2 5 3 4 1 8 9 7 10 6
i        j

next iteration:      2 5 3 4 1 8 9 7 10 6
i          j

after final swap:    2 5 3 4 1 6 9 7 10 8
i          j
Jim Anderson                     Comp 750, Fall 2009   Quicksort - 3
Time Analysis
Worst-Case: Always get completely unbalanced partition.
T(n)  T(n  1)  Θ(n)
n
  Θ(k)
k 1

 n 
 Θ  k 
 k 1 
 Θ n2   
Happens on sorted list!!

Best-Case: Always get perfectly balanced partition.
T(n)  2T(n/2) + (n) = (n lg n)
Jim Anderson                       Comp 750, Fall 2009           Quicksort - 4
Average-Case Time Analysis
What happens if we get sort-of-balanced partitions, e.g., something like:
T(n)  T(9n/10) + T(n/10) + (n)?
Still get (n lg n)!!

Intuition: Can divide n by c > 1 only (lg n) times before getting 1.
n

n/c

n/c2     roughly, logc n steps



1
Intuition holds even if c is very close to 1, e.g., 100/99.
(Remember: Different base logs are related by a constant.)
Jim Anderson                    Comp 750, Fall 2009             Quicksort - 5
Randomized Version
Want to make running time independent of input ordering.

Randomized-Partition(A, p, r)       Randomized-Quicksort(A, p, r)
i := Random(p, r);                  if p < r then
A[r]  A[i];                             q := Randomized-Partition(A, p, r);
Partition(A, p, r)                       Randomized-Quicksort(A, p, q – 1);
Randomized-Quicksort(A, q + 1, r)
fi

Jim Anderson                      Comp 750, Fall 2009                   Quicksort - 6
Avg. Case Analysis of Randomized
Quicksort
Let RV X = number of comparisons over all calls to Partition.

Suffices to compute E[X].        Why?

Notation:
• Let z1, z2, …, zn denote the list items (in sorted order).
• Let Zij = {zi, zi+1, …, zj}.
1 if zi is compared to zj           This is an example of an
Let RV Xij =
0 otherwise                         indicator random variable.
n 1    n
Thus, X     X .
i 1 j i 1
ij

Jim Anderson                        Comp 750, Fall 2009                    Quicksort - 7
Analysis (Continued)
We have:
Note:
         n 1  n
E[X]  E   X ij                      E[Xij] = 0·P[Xij=0] + 1·P[Xij=1]
 i 1 ji 1                         = P[Xij=1]
n 1     n
This is a nice property of
   E[X ij ]                 indicator RVs.
i 1 ji 1
n 1     n
   P[zi is compared to z j ]
i 1 ji 1

So, all we need to do is to compute P[zi is compared to zj].

Jim Anderson                       Comp 750, Fall 2009                 Quicksort - 8
Analysis (Continued)
zi and zj are compared iff the first element to be chosen as a pivot
from Zij is either zi or zj.

Exercise: Prove this.

So, P[zi is compared to z j ]  P[zi or z j is first pivot from Zij ]
 P[zi is first pivot from Zij ]
 P[z j is first pivot from Zij ]
1        1
          
j  i 1 j  i 1
2

j  i 1
Jim Anderson                     Comp 750, Fall 2009                  Quicksort - 9
Analysis (Continued)
Therefore,
n -1   n
2
E[X]   
i 1 ji 1 j  i  1
n -1 n -i
2
 
i 1 k 1 k  1
n -1   n
2
 
i 1 k 1 k                 n
1
n -1
  O(lg n)
k
k 1
 H n (n th Harmonic number)
i 1
H n  ln n  O(1)

 O(n lg n).
Jim Anderson                           Comp 750, Fall 2009          Quicksort - 10
Harmonic Numbers
What’s the asymptotic behavior of Hn?
Use approximation by integral (CLRS, page 1067).
(Applies to monotonic functions.)
n 1                    n              n
f(x)
 f(x)dx  f(k)   f(x)dx
m                     k m           m 1


x
m–1 m                                         n n+1

Jim Anderson                    Comp 750, Fall 2009                         Quicksort - 11
Harmonic Numbers (Continued)

n 1
dx n 1
So,             , or H n  ln(n  1)
1
x k 1 k
n           n
1   dx
 k   x  ln n, or H n  1  ln n
k 2   1

ln(n  1)  H n  1  ln n

Jim Anderson                      Comp 750, Fall 2009    Quicksort - 12
Other Variants
Lots of other variants of Quicksort have been
proposed.
We now analyze (in a different way) another
well-known variant and briefly comment on yet
another.

Jim Anderson          Comp 750, Fall 2009   Quicksort - 13
Second Version
Quicksort(A, p, r)                                   A[0] := ;
if r  p then                                     Quicksort(A, 1, n)
x, i, j := A[r], p – 1, r;
repeat
Differences from
repeat i := i + 1 until A[i]  x;
previous algorithm:
repeat j := j  1 until A[j]  x;      • i and j move in opposite
t := A[i];                               directions
A[i]  A[j]                            • new t variable
until j  i;
A[j] := A[i]; A[i] := A[r]; A[r] := t;
Quicksort(A, p, i–1);
Quicksort(A, i+1, r)
fi

Jim Anderson                    Comp 750, Fall 2009                   Quicksort - 14
Example
initially:          2 5 8 3 9 4 1 7 10 6           note: pivot (x) = 6
i                    j

after inner repeats: 2 5 8 3 9 4 1 7 10 6
i       j

after swap:        2 5 1 3 9 4 8 7 10 6            t = 8 (will always be the bigger
i       j                          of the two swapped)

after inner repeats: 2 5 1 3 9 4 8 7 10 6
i j

after swap:        2 5 1 3 4 9 8 7 10 6            t=9
i j

after inner repeats: 2 5 1 3 4 9 8 7 10 6          if all values are distinct,
j i                   will always cross like this and
then do an unnecessary swap
Jim Anderson                    Comp 750, Fall 2009                         Quicksort - 15
Example
after inner repeats: 2 5 1 3 4 9 8 7 10 6            from previous slide
j i

after swap:           2 5 1 3 9 4 8 7 10 6           t=9
j i

after outer repeat:   2 5 1 3 4 6 8 7 10 9           “correcting step” after main
j i                    loop -- note: 6 is in its proper
place now

Assuming distinct values, the correcting step always does this:

x x x x x x x x x x x                 pivot ends up here
j i
Question: What prevents the two inner repeats from causing an array
bounds error?
Jim Anderson                      Comp 750, Fall 2009                         Quicksort - 16
Average-Case Analysis
We want to compute the expected number of
comparisons in the average case.
How do we know what the “average case” is?
Need an input distribution. We assume:
 Some permutation of {1, …, n} is to be sorted.
 Each permutation is equally likely.
 Is this reasonable?
Note that the analysis here is different from before.
 We are doing average-case analysis of a deterministic
(not random) algorithm.
Jim Anderson                Comp 750, Fall 2009           Quicksort - 17
Digression
Question: How do you generate a random permutation of {1, …, n}?

A := {1, …, n}; /* identity permutation */
for i := 1 to n do
A[i]  A[random(i,n)]
od

Exercise: Prove that this routine generates all n! permutations with
equal probability.

Jim Anderson                   Comp 750, Fall 2009              Quicksort - 18
Recurrence to Solve
Note: When partitioning an array of length n, n+1 data comparisons
are always done (assuming the array is some permutation of {1, …, n}).
• the pivot is compared to each of the other n–1 elements once, and
• we do one additional “wasted” swap that entails two more
comparisons.
Note: The CLRS Partition does only n–1 comparisons.

Let C(n) = the number of data comparisons required to sort inputs of
size n.

We have:      C(0)  0
1 n
C(n)  n  1   C(i  1)  C(n  i)        ,n  0
n i 1
Jim Anderson                   Comp 750, Fall 2009            Quicksort - 19
Recurrence (Continued)
C(0)  0
1 n
C(n)  n  1   C(i  1)  C(n  i)      ,n  0
n i 1

Note 1: Each element is equally likely to be the pivot, hence 1/n.

Note 2: The subproblems are random if the original problem is
random, hence we can use the same C on both sides.
Exercise: Prove this.
Hint: The partition routine works in a very predictable way.

Jim Anderson                       Comp 750, Fall 2009           Quicksort - 20
More on Note 2 (A Third Version)
Not all Quicksort variants satisfy Note 2. Consider Hoare’s original
Partition, covered in the old CLR:
Quicksort(A, p, r)                             Partition(A, p, r)
if p < r then                                   x, i, j := A[p], p – 1, r + 1;
q := Partition(A, p, r);                   while true do
Quicksort(A, p, q);                              repeat j := j – 1 until A[j]  x;
Quicksort(A, q + 1, r)                           repeat i := i + 1 until A[i]  x;
fi                                                    if i < j then
A[i]  A[j]
A[p..r]                                                  else
return j
5                                                             fi
od
A[p..q] A[q+1..r]

Partition

5           5
Jim Anderson                               Comp 750, Fall 2009                        Quicksort - 21
Example of Hoare’s Partition
initially:        5 3 2 6 4 1 3 7
i                     j

after repeats: 5 3 2 6 4 1 3 7
i           j
Differences from
after swap:   3 3 2 6 4 1 5 7
previous Partition:
i           j
• pivot is on the left
after repeats: 3 3 2 6 4 1 5 7                    • inner repeats are reversed
i j                         • pivot ends up in one of the
subarrays
after swap:       3 3 2 1 4 6 5 7
i   j

after repeats: 3 3 2 1 4 6 5 7
j i

A[p..q] A[q+1..r]
Jim Anderson                          Comp 750, Fall 2009                        Quicksort - 22
More on Note 2
Hoare’s Partition is not random. Consider {1,…,n} = {1,…,4}.

list     resulting sublists     list        resulting sublists
1234     1 | 234                2134        1 | 234
1243     1 | 243                2143        1 | 243
1324     1 | 324                2314        1 | 324
1342     1 | 342                2341        1 | 342
1423     1 | 423                2413        1 | 423       A second sublist
1432     1 | 432                2431        1 | 432       of 43 is twice as
4123     312 | 4                3124        21 | 34       likely as a second
4132     213 | 4                3142        21 | 43       sublist of 34.
4213     321 | 4                3214        12 | 34
4231     123 | 4                3241        12 | 43
4312     231 | 4                3412        21 | 43
4321     132 | 4                3421        12 | 43
Jim Anderson                        Comp 750, Fall 2009                  Quicksort - 23
Note 2 (Continued)
Now, consider the Partition routine from the second version.
list   resulting sublists    list    resulting sublists
1234 123 |4|                 2134 213 |4|
1243 12 |3| 4                2143 21 |3| 4        First sublist:
123, 132, 213, 231,
1324 132 |4|                 2314 231 |4|
312, 321 each occur
1342 1 |2| 43                2341 |1| 342         once.
1423 12 |3| 4                2413 21 |3| 4
12 and 21 each occur
1432 1 |2| 34                2431 |1| 432         three times.
4123 21 |3| 4                3124 312 |4|
1 and  each occur
4132 1 |2| 34                3142 1 |2| 43
six times.
4213 12 |3| 4                3214 321 |4|
4231 |1| 234                 3241 |1| 243         Second sublist is
4312 1 |2| 43                3412 1 |2| 34        similar.
4321 |1| 324                 3421 |1| 423
Question: What about the CLRS Partition routine?
Jim Anderson                      Comp 750, Fall 2009               Quicksort - 24
Solving the Recurrence
1 n
C(n)  n  1   C(i  1)  C(n  i)         ,n  0
n i 1
2 n
C(n)  n  1   C(i  1)                      ,n  0
n i 1
n
nC(n)  n(n  1)  2 C(i  1)                 ,n  0
i 1
n -1
(n  1)C(n  1)  (n  1)n  2 C(i  1)       , n 1  0
i 1

Subtracting,
nC(n)  (n  1)C(n  1)  2n  2C(n  1)       , n 1

Jim Anderson                       Comp 750, Fall 2009            Quicksort - 25
Solving (Continued)
nC(n)  (n  1)C(n  1)  2n                   , n 1
Dividing by n(n  1),
C(n) C(n  1)       2
          
n 1       n      n 1
2     2 C(n  2)
        
n 1 n         n 1

n 1
1 C(1)
 2 
k 3 k  2
1 1
C(1)  (1  1)   (C(i  1)  C(1  i))  2
1 i 1
Jim Anderson                       Comp 750, Fall 2009            Quicksort - 26
Solving (Continued)
So,
n 1
C(n)         1 C(1)
 2 
n 1    k 3 k  2
         1 1 2
 2 H n 1    
         1 2 2
 2H n 1  2
Note: This is an exact
derivation  no
C(n)  2(n  1)(Hn 1  1)
asymptotic sloppiness.

Thus, C(n)  θ(n lg n)
Jim Anderson                 Comp 750, Fall 2009                Quicksort - 27
Stack Space
Algorithm as shown sorts sub-problems in a fixed
order. This can lead to deep O(n) call stacks.

Fix: Stack larger sub-problem, solve smaller sub-
problem first.
 Stack space becomes O(lg n).
 Recurrence is:
• S(n) = 0,                n  2.
• S(n) = 1 + S((n–1)/2), n > 2.

Jim Anderson                       Comp 750, Fall 2009   Quicksort - 28
Variations
Quicksort is not very efficient on small lists.

This is a problem because Quicksort will be
called on lots of small lists.

Fix 1: Use Insertion Sort on small problems.

Fix 2: Leave small problems unsorted. Fix with
one final Insertion Sort at end.
 Note: Insertion Sort is very fast on almost-sorted lists.

Jim Anderson                   Comp 750, Fall 2009         Quicksort - 29
Variations (Continued)
Input distribution may not be uniformly random.
Fix 1: Use “randomly” selected pivot.
 We’ve already analyzed this for a slightly different
algorithm.

Fix 2: Median-of-three Quicksort.
 Use median of three fixed elements (say, the first,
middle, and last) as the pivot.
 To get O(n2) behavior, we must continually get unlucky
and see that two out of the three elements examined are
among the largest or smallest.
Jim Anderson                Comp 750, Fall 2009         Quicksort - 30

```
DOCUMENT INFO
Shared By:
Categories:
Stats:
 views: 15 posted: 1/26/2011 language: English pages: 30
Description: Analysis and Design of Algorithms
About if any file u wil find copyright contact me it will be remove in 3 to 4 buisnees days. add me on sanjaydudeja007@gmail.com or visit http://www.ohotech.com/