# 2IL65 2011 1 Introduction by B8RgL7F

VIEWS: 27 PAGES: 46

• pg 1
```									2IL65 Algorithms

Fall 2011

Lecture 1: Introduction
Algorithms
Algorithm
a well-defined computational procedure that takes some value, or a
set of values, as input and produces some value, or a set of values,
as output.

Data Structure
a way to store and organize data to facilitate access and
modifications.

Algorithms research
design and analysis of algorithms and data structures for
computational problems.
The course
 Design and analysis of efficient algorithms for some basic
computational problems.

 Basic algorithm design techniques and paradigms

 Algorithms analysis: O-notation, recursions, …

 Basic data structures

 Basic graph algorithms

before we really get started …
Organization
Lecturer:   Kevin Buchin, HG 7.32,
k.a.buchin@tue.nl

Web page:   http://www.win.tue.nl/~kbuchin/ teaching/2IL65/

Book:       T.H. Cormen, C.E. Leiserson, R.L. Rivest and C. Stein.
Introduction to Algorithms (2nd or 3rd edition)
mandatory
Prerequisites
 Being able to work with basic programming constructs such as

 Being able to apply standard proving techniques such as proof by

 Being familiar with sums and logarithms such as discussed in
Chapter 3 and Appendix A of the textbook.

 If you think you might lack any of this knowledge, please come and
talk to me immediately so we can get you caught up.
1. 7 homework assignments, the best 5 of which each count for 10%
2. A written exam (closed book) which counts for the remaining 50%

 If you reach less than 50% of the possible points on the homework
assignments, then you are not allowed to participate in the final exam nor
in the second chance exam. You will fail the course and your next chance
achieved. If you reach less than 50% of the points on the final exam, then
you will fail the course, regardless of the points you collected with the
homework assignments. However, you are allowed to participate in the
second chance exam. The grade of the second chance exam replaces the
grade for the first exam, that is, your homework assignments always count
Do the homework assignments!

Aim at at least 60% in the homework (45 pts on 5; 9 in average)!!!
Homework Assignments
 Posted on web-page on Wednesdays before the lecture.

 Due Wednesday (one week later) before the lecture as paper copy.

 Late assignments will not be accepted.
Only 5 out of 7 assignments count, hence there are no exceptions.

 Must be handed in by each student separately.

 Must be typeset in English.

Any questions: Stop by my office
(send email if you want to make sure that I have time)
Homework Assignments
All class work has to be done independently. You are of course
allowed to discuss the material presented in class, homework
assignments, or general solution strategies with me or your
classmates, but you have to formulate and write up your solutions
by yourself. You must not copy from the internet, your friends, or
other textbooks. Problem solving is an important component of this
course so it is really in your best interest to try and solve all
problems by yourself. If you represent other people's work as your
own then that constitutes fraud and will be dealt with accordingly.

• You should formulate and write down your solution by yourself!
• Handing in a rewording of the same solution is an obvious violation!
Organization
Components:

1. Lecture            Wednesday 13:45 - 15:30     auditorium 16

2. “Big” tutorial     Thursday 10:45 - 12:30      auditorium 12
During this tutorial you have the opportunity to work on this
week's home work assignment. I will be present to answer
questions.

3. “Small” tutorial   Friday 13:45 - 15:15        auditorium 15
During these tutorials I will explain the solutions to the
homework assignments of the previous week and answer any
questions that arise.

 Check web-page for details
Schedule
The course again …
 Design and analysis of efficient algorithms for some basic
computational problems.

 Basic algorithm design techniques and paradigms

 Algorithms analysis: O-notation, recursions, …

 Basic data structures

 Basic graph algorithms

Why efficiency?
Sorting
The sorting problem
Input: a sequence of n numbers ‹a1, a2, …, an›
Output: a permutation of the input such that ‹ai1 ≤ … ≤ ain›

 The input is typically stored in arrays

 Numbers ≈ Keys

 Additional information (satellite data) may be stored with keys

 We will study several solutions ≈ algorithms for this problem
Describing algorithms
 A complete description of an algorithm consists of three parts:

1. the algorithm
(expressed in whatever way is clearest and most concise,
can be English and / or pseudocode)

2. a proof of the algorithm’s correctness

3. a derivation of the algorithm’s running time
InsertionSort
 Like sorting a hand of playing cards:
 remove cards one by one, insert into
correct position
 to find position, compare to cards in hand from right to left
 cards in hand are always sorted

InsertionSort is
 a good algorithm to sort a small number of elements
 an incremental algorithm

Incremental algorithms
process the input elements one-by-one and maintain the solution
for the elements processed so far.
Incremental algorithms
Incremental algorithms
process the input elements one-by-one and maintain the solution
for the elements processed so far.

Check book for more pseudocode conventions
 In pseudocode:

IncAlg(A)
// incremental algorithm which computes the solution of a problem
with input A = {x1,…,xn}
1. initialize: compute the solution for {x1}
2. for j = 2 to n
3.      do compute the solution for {x1,…,xj} using the (already
computed) solution for {x1,…,xj-1}
no “begin - end”, just indentation
InsertionSort
InsertionSort(A)
// incremental algorithm that sorts array A[1..n] in non-decreasing
order
1. initialize: sort A[1]
2. for j = 2 to A.length
3.      do sort A[1..j] using the fact that A[1.. j-1] is already sorted
InsertionSort
InsertionSort(A)
// incremental algorithm that sorts array A[1..n] in non-decreasing
order
1. initialize: sort A[1]          InsertionSort is an in place algorithm:
2. for j = 2 to A.length          the numbers are rearranged within the
3.      do key = A[j]             array with only constant extra space.
4.          i = j -1
5.          while i > 0 and A[i] > key
6.             do A[i+1] = A[i]
7.                   i ← i -1
8.          A[i +1] = key

1               j              n
1 3 14 17 28 6        …
Correctness proof
 Use a loop invariant to understand why an algorithm gives the

Loop invariant (for InsertionSort)
At the start of each iteration of the “outer” for loop (indexed by j) the
subarray A[1..j-1] consists of the elements originally in A[1..j-1] but
in sorted order.
Correctness proof
 To proof correctness with a loop invariant we need to show three
things:

Initialization
Invariant is true prior to the first iteration of the loop.

Maintenance
If the invariant is true before an iteration of the loop, it remains true
before the next iteration.

Termination
When the loop terminates, the invariant (usually along with the
reason that the loop terminated) gives us a useful property that
helps show that the algorithm is correct.
Correctness proof
InsertionSort(A)                            Loop invariant
1. initialize: sort A[1]                    At the start of each iteration of the
2. for j = 2 to A.length                    “outer” for loop (indexed by j) the
3.      do key = A[j]                       subarray A[1..j-1] consists of the
4.          i = j -1                        elements originally in A[1..j-1] but
5.          while i > 0 and A[i] > key      in sorted order.
6.             do A[i+1] = A[i]
7.                   i = i -1
8.          A[i +1] = key

Initialization
Just before the first iteration, j = 2 ➨ A[1..j-1] = A[1], which is the
element originally in A[1], and it is trivially sorted.
Correctness proof
InsertionSort(A)                         Loop invariant
1. initialize: sort A[1]                 At the start of each iteration of the
2. for j = 2 to A.length                 “outer” for loop (indexed by j) the
3.      do key = A[j]                    subarray A[1..j-1] consists of the
4.          i = j -1                     elements originally in A[1..j-1] but
5.          while i > 0 and A[i] > key   in sorted order.
6.             do A[i+1] = A[i]
7.                   i = i -1
8.          A[i +1] = key

Maintenance
Strictly speaking need to prove loop invariant for “inner” while loop.
Instead, note that body of while loop moves A[j-1], A[j-2], A[j-3],
and so on, by one position to the right until proper position of key is
found (which has value of A[j]) ➨ invariant maintained.
Correctness proof
InsertionSort(A)                         Loop invariant
1. initialize: sort A[1]                 At the start of each iteration of the
2. for j = 2 to A.length                 “outer” for loop (indexed by j) the
3.      do key = A[j]                    subarray A[1..j-1] consists of the
4.          i = j -1                     elements originally in A[1..j-1] but
5.          while i > 0 and A[i] > key   in sorted order.
6.             do A[i+1] = A[i]
7.                   i = i -1
8.          A[i +1] = key

Termination
The outer for loop ends when j > n; this is when j = n+1 ➨ j-1 = n.
Plug n for j-1 in the loop invariant ➨ the subarray A[1..n] consists
of the elements originally in A[1..n] in sorted order.
Another sorting algorithm

MergeSort
 A divide-and-conquer sorting algorithm.

Divide-and-conquer
break the problem into two or more subproblems, solve the
subproblems recursively, and then combine these solutions to
create a solution to the original problem.
Divide-and-conquer

D&CAlg(A)
// divide-and-conquer algorithm that computes the solution of a
problem with input A = {x1,…,xn}
1. if # elements of A is small enough (for example 1)
2.     then compute Sol (the solution for A) brute-force
3.     else
4.        split A in, for example, 2 non-empty subsets A1 and A2
5.        Sol1 = D&CAlg(A1)
6.        Sol2 = D&CAlg(A2)
7.        compute Sol (the solution for A) from Sol1 and Sol2
8. return Sol
MergeSort

MergeSort(A)
// divide-and-conquer algorithm that sorts array A[1..n]
1. if A.length == 1
2.     then compute Sol (the solution for A) brute-force
3.     else
4.        split A in 2 non-empty subsets A1 and A2
5.        Sol1 = MergeSort(A1)
6.        Sol2 = MergeSort(A2)
7.        compute Sol (the solution for A) from Sol1 en Sol2
MergeSort

MergeSort(A)
// divide-and-conquer algorithm that sorts array A[1..n]
1. if length[A] = 1
2.     then skip
3.     else
4.        n = A.length ; n1 = n/2 ; n2 = n/2 ;
copy A[1.. n1] to auxiliary array A1[1.. n1]
copy A[n1+1..n] to auxiliary array A2[1.. n2]
5.        MergeSort(A1)
6.        MergeSort(A2)
7.        Merge(A, A1, A2)
MergeSort
3 14 1 28 17 8 21 7 4 35

1 3 4 7 8 14 17 21 28 35

3 14 1 28 17              8 21 7 4 35

1 3 14 17 28              4 7 8 21 35

3 14           1 28 17

3 14           1 17 28

3          14
MergeSort
 Merging

A1    1 3 14 17 28     A2   4 7 8 21 35

A   1 3 4 7 8 14 17 21 28 35
MergeSort: correctness proof
Induction on n (# of input elements)

   proof that the base case (n small) is solved correctly

   proof that if all subproblems are solved correctly, then the
complete problem is solved correctly
MergeSort: correctness proof
MergeSort(A)
1. if length[A] = 1
2.      then skip
3.      else
4.         n = A.length ; n1 = n/2 ; n2 = n/2 ;
copy A[1.. n1] to auxiliary array A1[1.. n1]
copy A[n1+1..n] to auxiliary array A2[1.. n2]
5.         MergeSort(A1)
6.         MergeSort(A2)              Lemma
7.         Merge(A, A1, A2)              MergeSort         sorts the array A[1..n] correctly.

Proof (by induction on n)

Base case: n = 1, trivial ✔

Inductive step: assume n > 1. Note that n1 < n and n2 < n.
Inductive hypothesis ➨ arrays A1 and A2 are sorted correctly
Remains to show: Merge(A, A1, A2) correctly constructs a sorted
array A out of the sorted arrays A1 and A2 …      etc.    ■
QuickSort
another divide-and-conquer sorting
algorithm…
QuickSort
QuickSort(A)
// divide-and-conquer algorithm that sorts array A[1..n]
1. if length[A] ≤ 1
2.     then skip
3.     else
4.        pivot = A[1]
5.        move all A[i] with A[i] < pivot into auxiliary array A1
6.        move all A[i] with A[i] > pivot into auxiliary array A2
7.        move all A[i] with A[i] = pivot into auxiliary array A3
8.        QuickSort(A1)
9.        QuickSort(A2)
10.       A = “A1 followed by A3 followed by A2”
Analysis of algorithms

some informal thoughts – for now …
Analysis of algorithms
 Can we say something about the running time of an algorithm
without implementing and testing it?

InsertionSort(A)
1. initialize: sort A[1]
2. for j = 2 to A.length
3.     do key = A[j]
4.          i = j -1
5.          while i > 0 and A[i] > key
6.             do A[i+1] = A[i]
7.                   i = i -1
8.          A[i +1] = key
Analysis of algorithms
 Analyze the running time as a function of n (# of input elements)

 best case
 average case
 worst case

An algorithm has worst case running time T(n) if for any
input of size n the maximal number of elementary operations
executed is T(n).

elementary operations
unconditional branch, return …
Analysis of algorithms: example
n=10       n=100         n=1000
InsertionSort:   15 n2 + 7n – 2        1568      150698        1.5 x 107
MergeSort:       300 n lg n + 50 n    10466      204316        3.0 x 106

InsertionSort         InsertionSort
6 x faster            1.35 x faster

MergeSort
5 x faster

n = 1,000,000       InsertionSort 1.5 x 1013
MergeSort        6 x 109    2500 x faster !
Analysis of algorithms
 It is extremely important to have efficient algorithms for large inputs

 The rate of growth (or order of growth) of the running time is far
more important than constants

InsertionSort: Θ(n2)
MergeSort: Θ(n log n)
Θ-notation
Intuition: concentrate on the leading term, ignore constants

19 n3 + 17 n2 - 3n       becomes       Θ(n3)

2 n lg n + 5 n1.1 - 5    becomes       Θ(n1.1)

n - ¾ n √n               becomes         ---

(precise definition next lecture …)
Some rules and notation
   log n denotes log2 n

   We have for a, b, c > 0 :

1. logc (ab) =
logc a + logc b

2. logc (ab) =
b logc a

3. loga b =
logc b / logc a
 lg35n vs. √n ?

 logarithmic functions grow slower than polynomial functions
 lga n grows slower than nb for all constants a > 0 and b > 0

 n100 vs. 2 n ?

 polynomial functions grow slower than exponential functions
 na grows slower than bn for all constants a > 0 and b > 1
Some rules and notation
   log n denotes log2 n

   We have for a, b, c > 0 :

1. logc (ab) = logc a + logc b

2. logc (ab) = b logc a

3. loga b =    logc b / logc a
Announcements

This week
 Lecture 2 on Friday
 no “small” tutorial

Register on education.tue.nl!

```
To top