# Lecture 6

Document Sample

Algorithms and Complexity Theory                                                                     UKZN - Computer Science - 2010   Algorithms and Complexity Theory                                           UKZN - Computer Science - 2010

Introduction

• Sorting is the process of arranging the elements of a set in order.
ALGORITHMS and COMPLEXITY THEORY                                                                                 • Sorting was one of the ﬁrst problem intensively studied by computer
Lecture 6                                                                                       scientists.
Sorting Algorithms 1                                                                                 • There are several methods of sorting , and most of them use the
Divide and Conquer( to be studied later) techniques.

• some sorting algorithms: Insertion sort, selection sort, Mergesort,
Bubble sort, Radix sort , Quicksort.

Lecture 6: Sorting Algorithms 1                                                                                                  1    Lecture 6: Sorting Algorithms 1                                                                        2

Algorithms and Complexity Theory                                                                     UKZN - Computer Science - 2010   Algorithms and Complexity Theory                                           UKZN - Computer Science - 2010

Selection Sort
If we want to sort a list of n elements, swap the smallest element with one at the ﬁrst position. Apply this                                                     Selection Sort Algorithm
algorithm recursively to the n-1 remaining elements(placed from the 2nd to the nth positions). repeat
the process until the list is reduce to a single element.
SelectionSort(int[] t, int n)
int i,j,min,q
L       I      C      E   N   C    E
begin
We will have the following sequence of execution                                                                                            for i = 1 to n-1
min = i
gives                                                                                               for j = i+1 to n
L       I      C      E   N   C    E           C     I     L     E     N     C     E
if t[j] < t[min]
then min = j
C       I      L      E   N   C    E   gives   C     C      L     E     N     I    E                                                          endif
endfor
C       C       L     E   N   I    E   gives   C     C      E     L     N     I    E                                                       q = t[min]
t[min]= t[i]
C       C       E     L   N   I    E   gives   C     C      E     E     N     I    L                                                       t[i] = q
end

C       C       E     E   N   I    L   gives   C     C      E     E     I    N     L
The maximum complexity of this algorithm is O(n2 ).

C       C       E     E   I   N    L   gives   C     C      E     E     I    L     N

Lecture 6: Sorting Algorithms 1                                                                                                  3    Lecture 6: Sorting Algorithms 1                                                                        4
Algorithms and Complexity Theory                                                                      UKZN - Computer Science - 2010   Algorithms and Complexity Theory                                                                      UKZN - Computer Science - 2010

Bubble Sort
If we want to sort a list of n elements, Place the biggest element at the end. Apply the algorithm
recursively to the n-1 ﬁrst elements. Repeat the process until the list is reduce to a single element. For                                                                           Bubble sort algorithm
instance, sort the table:

We want to sort the list   C     K     G     L     B                                                                                   BUBBLESORT(int [] t, int n)
int i,j,p
step           array    operation                   Resulting array                                                                     for i = n downto 2 do
1rst       CKGLB        No change                   CKGLB                                                                                   for j = 1 to i-1 do
if t[j] > t[j + 1] then
CKGLB        K > G , swap K and G        CGKLB
p= t[j]
CGKLB        No change                   CGKLB                                                                                            t[j]= t[j+1]
CGKLB        L > B , swap L and B        CGKBL                                                                                            t[j+1]= p
2nd        CGKBL        No change                   CGKBL                                                                                        end if
CGKBL        No Change                   CGKBL                                                                                   end for
end for
CGKBL        K > B , swap K and B        CGBKL
3rd       CGBKL        No change                   CGBKL
Bubblesort is an O(n2 ) algorithm.
CGBKL        G > B , swap G and B        CBGKL
4th       CBGKL        C > B , swap C and B        BCGKL
BCGKL       end

Lecture 6: Sorting Algorithms 1                                                                                                   5    Lecture 6: Sorting Algorithms 1                                                                                                   6

Algorithms and Complexity Theory                                                                      UKZN - Computer Science - 2010   Algorithms and Complexity Theory                                                                      UKZN - Computer Science - 2010

sequential Insertion sort

We want to sort the following list:   C       K       G          A    B

step      array             Operation                                  resulting array
Insertion sort                                                                                 1rst      CKGAB             Initialization, No Change                   CKGAB
2nd       CKGAB             Save the value(K) at the 2nd position
Compare K to C, No change                   CKGAB
• more natural way to sort.                                                                                                            3rd      CKGAB             Save the value(G) at the 3rd position
Compare G to K, Move K to the right         CKKAB
• Insertion sort inserts each item into its proper place in the ﬁnal list.                                                                      CKKAB             compare G to C, No change                   CKKAB
CKKAB             insert G at the second position             CGKAB
Save the value(A) at the 4th position
• The simplest implementation of this requires two list structures: the                                                                4th      CGKAB
compare A to K, move K to the right         CGKKB
source list and the ﬁnal list, into which the sorted items are inserted.                                                                      CGKKB             compare A to G, move G to the right         CGGKB
CGGKB             compare A to C, move C to the right         CCGKB

• But we can also work "in place" with one list.                                                                                                CCGKB             insert A at the ﬁrst position               ACGKB
5th      ACGKB             Save the value(B) at the 5th position
compare B and K, move K to the right        ACGKK
ACGKK             compare B to G, move G to the right         ACGGK
ACGGK             compare B to C, move C to the right         ACCGK
ACCGK             compare B to A, No change                   ACCGK
ACCGK             insert B at the second position             ABCGK

Lecture 6: Sorting Algorithms 1                                                                                                   7    Lecture 6: Sorting Algorithms 1                                                                                                   8
Algorithms and Complexity Theory                                       UKZN - Computer Science - 2010   Algorithms and Complexity Theory                                                                                                        UKZN - Computer Science - 2010

Binary Insertion sort
We want to use binary insertion sort to sort the following array   C       K       G       A       B
sequential Insertion sort(cont’d)                                                  (1) A single element C is already sorted

(2) We will then try to insert K in the array which consists of a single element C, as K is greater than C , we will then have the array C K as
SEQINSERTIONSORT(int [] t;int n)                                                                                result, this will be done after one comparison , to know if we will insert K before or after C.

int i,j,q                                                                                                 (3) We want then to insert G in the array C K ; instead of trying to ﬁnd the place where to insert from C, we will

BEGIN                                                                                                              – split the array into two parts C and K ,

FOR i := 2 TO n DO                                                                                                – compare G to C , as G is greater to C , G can only be inserted in the second part of the list, which consists of a single element K
which is greater than G ,
j := i-1; q = t[i];
– then G will be inserted between C and K, which results in the array C G K.
WHILE((j >= 0) AND ( t[j] >q )) DO
(4) We then have to insert A in the array C G K:
t[j+1] = t[j]
– split C G K into two parts C G and K,
j := j - 1
– compare A to the last element (G) of C G, as G is greater than A, we will then recursively , using the same method, search for the
END WHILE                                                                                                       position where to insert A in C G. split the list into two parts C and G , compare A to C , as A is smaller than C , A will be inserted
t[j+1] = q                                                                                                    in the ﬁrst part of the array, which consists of a single element C ( and C is greater than A) , means A will be inserted at the
END FOR                                                                                                          beginning of the array C G, which gives to the array A C G.
– We will then have at this step the list A C G K
END
(5) We then have to insert B in the array A C G K:
– split A C G K into two parts A C and G K,
This algorithm is an O(n2 )
– compare B to the last element (C) of A C, as C is greater than B, we will then recursively , search for the position where to insert B
in A C. split the array into two parts A and C , compare B to A, as B is greater than A , B will be inserted between A and C, which
gives the array A B C.
– We will then have as result the array A B C G K

Lecture 6: Sorting Algorithms 1                                                                    9    Lecture 6: Sorting Algorithms 1                                                                                                                                    10

Algorithms and Complexity Theory                                       UKZN - Computer Science - 2010   Algorithms and Complexity Theory                                                                                                        UKZN - Computer Science - 2010

Binary search
Binary Insertion sort
int binsearch (int [] t, int Key ,int istart, int iend)
SORTINSERTBIN (int [] t, int n)                                                                           int j,k,position;
int i,j,Key ,pos;                                                                                        BEGIN
BEGIN                                                                                                     position = istart; j=iend
FOR i= 2 to n DO                                                                                        WHILE j > position DO
Key = t[i]                                                                                             k = (position+j) div 2
pos = binsearch (t,Key,1,i-1)                                                                          if Key <= t[k] then j:= k
FOR j =i downto pos+ 1 DO t[j] := t[j-1]                                                                               else position := k + 1
t[pos]= Key                                                                                          END WHILE
END FOR                                                                                                  if (Key > t[position]) then position := position + 1
END                                                                                                        return(position)
end

Lecture 6: Sorting Algorithms 1                                                                   11    Lecture 6: Sorting Algorithms 1                                                                                                                                    12
Algorithms and Complexity Theory                                                               UKZN - Computer Science - 2010   Algorithms and Complexity Theory                                                           UKZN - Computer Science - 2010

Complexity of binary sort                                                                                                complexity of Binary sort : worst case

suppose we need m iterations to ﬁnd the right position for the element we                                                       α(i) = (i-1)(e+a) +a where a is time spent arithmetic operation and e for
want to insert:                                                                                                                 store then
n
2m−1 ≤ n − 1 < 2m then m = log2 (n-1) if n is the size of the array                                                             T(n) =                  (A1 i + Blog2 (i − 1) + C1 )
The running time of binary insertion sort is:                                                                                                 i=2
n                                                                                                                                                        n                                  n

T(n) =                  (A + Blog2 (i − 1) + C + α(i))                                                                          but log2 i < i then T (n) ≤                   (A1 i + B(i − 1) + C1 ) =          (A2 i + B2 )
i=2                                                                                                                                                       i=2                                i=2
T(n) ≤ A2 (n − 1) − B2 + A2 n(n+1)
2
Were α(n) is the time spent to insert new item.                                                                                 thus T(n) = O(n2 ).

Lecture 6: Sorting Algorithms 1                                                                                           13    Lecture 6: Sorting Algorithms 1                                                                                       14

Algorithms and Complexity Theory                                                               UKZN - Computer Science - 2010   Algorithms and Complexity Theory                                                           UKZN - Computer Science - 2010

complexity of Binary sort : best case                                                                                         complexity of Binary sort : average case

α(i) = 0                                                                                                                        we have:
n                               n−1                                                                                                                  i(i−1)
0, 1, 2 , 3, . . . i-1 shifts                 and we have i type of shifts ( from 0 to i-1)
T(n) =                  (A + Blog2 (i − 1)) =         (A + Blog2 (i))                                                                                                   2
i(i−1)
i=2                               i=1                                                                             then in average we have αm (i) =       = i−1
2         2i
n                                       n−1                                                                   n
i−1
T(n)=A(n − 1) + B                      log2 (i) = A(n − 1) + Blog2 (            i)                                              T(n) =      (A + Blog2 (i − 1) + C +     )
i=2
2
i=1                                      i=1
n−1                            √                                                                                 as log2 n < n then
but , i=1 i =(n − 1)! and n! =                    2πn( n n [1 + O( n )]
e           1

Thus T(n) = O(nlogn).                                                                                                           T(n) = O(n2 )

Lecture 6: Sorting Algorithms 1                                                                                           15    Lecture 6: Sorting Algorithms 1                                                                                       16
Algorithms and Complexity Theory                                      UKZN - Computer Science - 2010