Document Sample

461191 Discrete Mathematics Lecture 3: Algorithms, The Integers, and Matrices San Ratanasanya CS, KMUTNB Adapted from several sources: Michael P. Frank, University of Florida, University of Nevada, and University of Maryland at Baltimore Today‟s Topics Review Administrivia Algorithms Algorithmic and Problem Complexity Introduction to Number Theory Applications of Number Theory Matrices …from last week… Sets and Set operations Set is an unordered collection of objects Operations on Set: union, intersection, difference, complement, symmetric difference Venn Diagram and equivalence Functions Domain, Codomain, Range One-to-one, onto, bijection, inverse, and composite functions Sequences Represents ordered list of elements a1,a,2a3,…,an or {an} It is convenient to describe Sequence using formula an = 2an-1 Summations Example: Set Prove that A A = U is true A = {x | x A} A = {x | x A} A A = {x | x A x A} A A = {x | x U} AA=U Example: Function Let f(x) = x2/3. Find f(S) if S = {x | xZ |x| 3} f(3) = 32/3 = 3, f(2) = 22/3 = 1, f(1) = 12/3 = 0, f(0) = 0 f(S) = {0, 1, 3} Suppose that g is a function from A to B and f is a function form B to C. Show that if both f and g are one-to-one functions, then fg is also one-to-one. We need to show that x A y A (x y f(g(x)) f(g(y))). Or we need to show that x A y A (x y g(x) g(y)) x B y B (x y f(x) f(y)). Given that g is one-to-one, therefore g(x) g(y) in B. Given that f is one-to-one, therefore f(x) f(y) in C. We can now conclude that f(g(x)) f(g(y)). Example: Sequence Find a simple formula for an if the first 10 terms of the sequence {an} are 1, 7, 25, 79, 241, 727, 2185, 6559, 19681, 59047 a1 = 1 an = 3n - 2 Example: Summations n 3 3 a a i 1 ? i ? i i 1 a1 a0 a2 a1 i 1 j 0 3 3 a3 a2 an 1 an 2 i 1 i 1 j 0 an an 1 3 a n a0 4i i 1 n 4 i 3 (2k 1) ? k 1 i 1 n n n 3 * (3 1) 2 k 1 2 k n 4* k 1 k 1 k 1 2 (n)(n 1) 24 2* n n2 n n 2 n2 Administrivia Homeworks due today Homework 1.1 – 1.4 Homework 2 Programs used in this class cannot be downloaded from class webpage they are TOO BIG… There are only links to download those programs on class webpage Prolog, ISETLW Python, Scheme Information on what to download is also on the webpage Algorithms The foundation of computer programming. Most generally, an algorithm just means a definite procedure for performing some sort of task. A computer program is simply a description of an algorithm, in a language precise enough for a computer to understand, requiring only operations that the computer already knows how to do. We say that a program implements (or “is an implementation of”) its algorithm. Algorithms You Already Know Grade-school arithmetic algorithms: How to add any two natural numbers written in decimal on paper, using carries. Similar: Subtraction using borrowing. Multiplication & long division. Your favorite cooking recipe. How to register for classes at KMUTNB. Programming Languages Some common programming languages: Newer: Java, C, C++, C#, Visual Basic, JavaScript, Perl, Tcl, Pascal, many others… Older: Fortran, Cobol, Lisp, Basic Assembly languages, for low-level coding. In this class we will use an informal, Pascal- like “pseudo-code” language. You should know at least 1 real language! Algorithm Example (English) Task: Given a sequence {ai}=a1,…,an, aiN, say what its largest element is. One algorithm for doing this, in English: Set the value of a temporary variable v (largest element seen so far) to a1’s value. Look at the next element ai in the sequence. If ai>v, then re-assign v to the number ai. Repeat then previous 2 steps until there are no more elements in the sequence, & return v. Executing an Algorithm When you start up a piece of software, we say the program or its algorithm are being run or executed by the computer. Given a description of an algorithm, you can also execute it by hand, by working through all of its steps with pencil & paper. Before ~1940, “computer” meant a person whose job was to execute algorithms! Executing the Max algorithm Let {ai}=7,12,3,15,8. Find its maximum… Set v = a1 = 7. Look at next element: a2 = 12. Is a2>v? Yes, so change v to 12. Look at next element: a2 = 3. Is 3>12? No, leave v alone…. Is 15>12? Yes, v=15… Algorithm Characteristics Some important general features of algorithms: Input. Information or data that comes in. Output. Information or data that goes out. Definiteness. Algorithm is precisely defined. Correctness. Outputs correctly relate to inputs. Finiteness. Won’t take forever to describe or run. Effectiveness. Individual steps are all do-able. Generality. Works for many possible inputs. Efficiency. Takes little time & memory to run. Our Pseudocode Language procedure for variable := initial name(argument: type) value to final value variable := expression statement informal statement while condition begin statements end statement {comment} procname(arguments) if condition then Not defined in book: statement [else return expression statement] What is Pseudocode Pseudocode is an intermediate step between language description, normally English, of the steps of a procedure and a specification of this procedure using an actual programming language. ่ ั เป็ นการอธิบายโปรแกรมด้วยภาษาทีใช้กนทั่วไป ด้วยลักษณะที่ ่ คล้ายภาษาทีใช้ในการเขียนโปรแกรม Human Programming Pseudocode Languages Languages What is Program Program = Algorithm + Data Structure Each step of a program may make changes to variables Trace = a sequence of changes in variables as each step of program is executed Trace is a simple model of program executions State = variable procedure procname(arg: type) Declares that the following text defines a procedure named procname that takes inputs (arguments) named arg which are data objects of the type type. Example: procedure maximum(L: list of integers) [statements defining maximum…] variable := expression An assignment statement evaluates the expression expression, then reassigns the variable variable to the value that results. Example assignment statement: v := 3x+7 (If x is 2, changes v to 13.) In pseudocode (but not real code), the expression might be informally stated: x := the largest integer in the list L Informal statement Sometimes we may write a statement as an informal English imperative, if the meaning is still clear and precise: e.g.,“swap x and y” Keep in mind that real programming languages never allow this. When we ask for an algorithm to do so- and-so, writing “Do so-and-so” isn’t enough! Break down algorithm into detailed steps. begin statements end Groups a sequence of Allows the sequence statements together: to be used just like a begin single statement. statement 1 Might be used: statement 2 After a procedure … declaration. statement n In an if statement after end then or else. In the body of a for or Curly braces {} are used instead while loop. in many languages. {comment} Not executed (does nothing). Natural-language text explaining some aspect of the procedure to human readers. Also called a remark in some real programming languages, e.g. BASIC. Example, might appear in a max program: {Note that v is the largest integer seen so far.} Conditional Constructions if condition then statement if condition then statement 1 else statement 2 if condition then begin Block of if condition 1 then statement 1 statements else if condition 2 then statement 2 end if condition then else if condition 2 then statement 2 begin ……………………………………………………… statement 1 else if condition n then statement n end else statement n+1 else begin statement 2 end if condition then statement Evaluate the propositional expression condition. If the resulting truth value is True, then execute the statement statement; otherwise, just skip on ahead to the next statement after the if statement. Variant: if cond then stmt1 else stmt2 Like before, but iff truth value is False, executes stmt2. Loop Constructions for variable := initial value to final value statement for all element with certain property for variable := initial value to final value while condition while condition begin statement begin Block of statements end Block of statements 1 end sum := 0 sum := 0 while n > 0 for i := 1 to n begin sum := sum + i sum := sum + n n := n - 1 end while condition statement Evaluate the propositional (Boolean) expression condition. If the resulting value is True, then execute statement. Continue repeating the above two actions over and over until finally the condition evaluates to False; then proceed to the next statement. while condition statement Also equivalent to infinite nested ifs, like so: if condition begin statement if condition begin statement …(continue infinite nested if’s) end end for var := initial to final stmt Initial is an integer expression. Final is another integer expression. Semantics: Repeatedly execute stmt, first with variable var := initial, then with var := initial+1, then with var := initial+2, etc., then finally with var := final. Question: What happens if stmt changes the value of var, or the value that initial or final evaluates to? for var := initial to final stmt For can be exactly defined in terms of while, like so: begin var := initial while var final begin stmt var := var + 1 end end procedure(argument) A procedure call statement invokes the named procedure, giving it as its input the value of the argument expression. Various real programming languages refer to procedures as functions (since the procedure call notation works similarly to function application f(x)), or as subroutines, subprograms, or methods. Nested Block of statements Conditional Loop Procedure – a call to procedure within procedure Procedure statistic (L:list of integers) begin max(L) min(L) average(L) ……….. end Max procedure in pseudocode procedure max(a1, a2, …, an: integers) v := a1 {largest element so far} for i := 2 to n {go thru rest of elems} if ai > v then v := ai {found bigger?} {at this point v’s value is the same as the largest integer in the list} return v Inventing an Algorithm Requires a lot of creativity and intuition Like writing proofs. Unfortunately, we can’t give you an algorithm for inventing algorithms. Just look at lots of examples… And practice (preferably, on a computer) And look at more examples… And practice some more… etc., etc. Algorithm-Inventing Example Suppose we ask you to write an algorithm to compute the predicate: IsPrime:N→{T,F} Computes whether a given natural number is a prime number. First, start with a correct predicate-logic definition of the desired function: n: IsPrime(n) ¬1<d<n: d|n Means d divides n evenly (without remainder) IsPrime example, cont. Notice that the negated exponential can be rewritten as a universal: Means d does not divide n evenly ¬1<d<n: d|n 1<d<n: d | n (the remainder is ≠0) 2≤ d ≤ n−1: d | n This universal can then be translated directly into a corresponding for loop: for d := 2 to n−1 { Try all potential divisors >1 & <n } if d|n then return F { n has divisor d; not prime } return T { no divisors were found; n must be prime} Optimizing IsPrime The IsPrime algorithm can be further optimized: for d := 2 to n1/2 Note smaller range of search. if d|n then return F return T This works because of this theorem: If n has any (integer) divisors, it must have one less than n1/2. Proof: Suppose n’s smallest divisor >1 is a, and let b :≡ n/a. Then n = ab, but if a > n1/2 then b > n1/2 (since a is n’s smallest divisor) and so n = ab > (n1/2)2 = n, an absurdity. Further optimizations are possible: - E.g., only try divisors that are primes less than n1/2. Another example task Problem of searching an ordered list. Given a list L of n elements that are sorted into a definite order (e.g., numeric, alphabetical), And given a particular element x, Determine whether x appears in the list, and if so, return its index (position) in the list. Problem occurs often in many contexts. Let’s find an efficient algorithm! Search alg. #1: Linear Search procedure linear search (x: integer, a1, a2, …, an: distinct integers) i := 1 {start at beginning of list} while (i n x ai) {not done, not found} i := i + 1 {go to the next position} if i n then location := i {it was found} else location := 0 {it wasn’t found} return location {index or 0 if not found} Search alg. #2: Binary Search Basic idea: On each step, look at the middle element of the remaining list to eliminate half of it, and quickly zero in on the desired element. <x <x <x >x Search alg. #2: Binary Search procedure binary search (x:integer, a1, a2, …, an: distinct integers) i := 1 {left endpoint of search interval} j := n {right endpoint of search interval} while i<j begin {while interval has >1 item} m := (i+j)/2 {midpoint} if x>am then i := m+1 else j := m end if x = ai then location := i else location := 0 return location binary search for the letter „j‟ search interval a c d f g h j l m o p r s u v x z center element search interval a c d f g h j l m o p r s u v x z center element binary search for the letter „j‟ search interval a c d f g h j l m o p r s u v x z center element search interval a c d f g h j l m o p r s u v x z center element binary search for the letter „j‟ search interval a c d f g h j l m o p r s u v x z center element found ! Practice exercises Devise an algorithm that finds the sum of all the integers in a list. [2 min] procedure sum(a1, a2, …, an: integers) s := 0 {sum of elems so far} for i := 1 to n {go thru all elems} s := s + ai {add current item} {at this point s is the sum of all items} return s Sorting Algorithms Sorting is a common operation in many applications. E.g. spreadsheets and databases It is also widely used as a subroutine in other data-processing algorithms. Two sorting algorithms shown in textbook: Bubble sort However, these are not very efficient, and you should Insertion sort not use them on large data sets! We’ll see some more efficient algorithms later in the course. Bubble Sort Smallest elements “float” up to the top of the list, like bubbles in a container of liquid. 30 30 1 1 1 1 31 1 30 30 2 2 1 31 31 2 30 3 32 32 2 31 3 30 33 2 32 3 31 31 2 33 3 32 32 32 34 3 33 33 33 33 3 34 34 34 34 34 Insertion Sort Algorithm English description of algorithm: For each item in the input list, “Insert” it into the correct place in the sorted output list generated so far. Like so: Use linear or binary search to find the location where the new item should be inserted. Then, shift the items from that position onwards down by one position. Put the new item in the hole remaining. Pseudocodes Procedure insertionsort(a1,…,an: real numbers with n 2) for j := 2 to n begin Procedure bubblesort(a1,…,an: real numbers with n 2) i := 1 for i := 1 to n-1 while aj > ai for j := 1 to n-1 i := i +1 if aj > aj+1 then interchange aj and aj+1 m : = aj {a1,…,an is in increasing order} for k := 0 to j – i – 1 aj-k := aj-k-1 ai := am end {a1, a2,…,an are sorted} Greedy Algorithms Any algorithms selecting the „best‟ choice at each step are called Greedy Can solve optimization problem Procedure change(c1, c2,…,cr: values of denomination of coins, where c1 > c2 >… > cr; n: a positive integer) for i := 1 to r while n ci begin add a coin with value ci to the change n := n - ci end Analysis of Algorithms What is the goal of analysis of algorithms? To compare algorithms mainly in terms of running time but also in terms of other factors (e.g., memory requirements, programmer's effort etc.) What do we mean by running time analysis? Determine how running time increases as the size of the problem increases. Is Binary Search more efficient? Number of iterations: For a list of n elements, Binary Search can execute at most log2 n times!! Linear Search, on the other hand, can execute up to n times !! Average Number of Iterations Length Linear Search Binary Search 10 5.5 2.9 100 50.5 5.8 1,000 500.5 9.0 10,000 5000.5 12.0 Is Binary Search more efficient? Number of computations per iteration: Binary search does more computations than Linear Search per iteration. Overall: If the number of components is small (say, less than 20), then Linear Search is faster. If the number of components is large, then Binary Search is faster. How do we analyze algorithms? We need to define a number of objective measures. (1) Compare execution times? Not good: times are specific to a particular computer !! (2) Count the number of statements executed? Not good: number of statements vary with the programming language as well as the style of the individual programmer. Example (# of statements) Algorithm 1 Algorithm 2 arr[0] = 0; for(i=0; i<N; i++) arr[1] = 0; arr[i] = 0; arr[2] = 0; ... arr[N-1] = 0; How do we analyze algorithms? (3) Express running time as a function of the input size n (i.e., f(n)). To compare two algorithms with running times f(n) and g(n), we need a rough measure of how fast a function grows. Such an analysis is independent of machine time, programming style, etc. Computing running time Associate a "cost" with each statement and find the "total cost“ by finding the total number of times each statement is executed. Express running time in terms of the size of the problem. Computing running time (contd.) Algorithm 1 Algorithm 2 Cost Cost arr[0] = 0; c1 for(i=0; i<N; i++) c2 arr[1] = 0; c1 arr[i] = 0; c1 arr[2] = 0; c1 ... arr[N-1] = 0; c1 ----------- ------------- c1+c1+...+c1 = c1 x N (N+1) x c2 + N x c1 = (c2 + c1) x N + c2 Computing running time (cont.) Cost sum = 0; c1 for(i=0; i<N; i++) c2 for(j=0; j<N; j++) c2 sum += arr[i][j]; c3 ------------ c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N x N Comparing Functions Using Rate of Growth Consider the example of buying elephants and goldfish: Cost: cost_of_elephants + cost_of_goldfish Cost ~ cost_of_elephants (approximation) The low order terms in a function are relatively insignificant for large n n4 + 100n2 + 10n + 50 ~ n4 i.e., n4 + 100n2 + 10n + 50 and n4 have the same rate of growth Rate of Growth ≡Asymptotic Analysis Using rate of growth as a measure to compare different functions implies comparing them asymptotically. If f(x) is faster growing than g(x), then f(x) always eventually becomes larger than g(x) in the limit (for large enough values of x). Example Suppose you are designing a web site to process user data (e.g., financial records). Suppose program A takes fA(n)=30n+8 microseconds to process any n records, while program B takes fB(n)=n2+1 microseconds to process the n records. Which program would you choose, knowing you’ll want to support millions of users? Visualizing Orders of Growth On a graph, as you go to the Value of function fA(n)=30n+8 right, a faster growing function fB(n)=n2+1 eventually becomes larger... Increasing n Big-O Notation We say fA(n)=30n+8 is order n, or O(n). It is, at most, roughly proportional to n. fB(n)=n2+1 is order n2, or O(n2). It is, at most, roughly proportional to n2. In general, an O(n2) algorithm will be slower than O(n) algorithm. Warning: an O(n2) function will grow faster than an O(n) function. More Examples … We say that n4 + 100n2 + 10n + 50 is of the order of n4 or O(n4) We say that 10n3 + 2n2 is O(n3) We say that n3 - n2 is O(n3) We say that 10 is O(1), We say that 1273 is O(1) Big-O Visualization Computing running time Algorithm 1 Algorithm 2 Cost Cost arr[0] = 0; c1 for(i=0; i<N; i++) c2 arr[1] = 0; c1 arr[i] = 0; c1 arr[2] = 0; c1 ... arr[N-1] = 0; c1 ----------- ------------- c1+c1+...+c1 = c1 x N (N+1) x c2 + N x c1 = (c2 + c1) x N + c2 O(n) Computing running time (cont.) Cost sum = 0; c1 for(i=0; i<N; i++) c2 for(j=0; j<N; j++) c2 sum += arr[i][j]; c3 ------------ c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N x N O(n2) Running time of various statements while-loop for-loop Examples i = 0; while (i<N) { X=X+Y; // O(1) result = mystery(X); // O(N), just an example... i++; // O(1) } The body of the while loop: O(N) Loop is executed: N times N x O(N) = O(N2) Examples (cont.’d) if (i<j) for ( i=0; i<N; i++ ) O(N) X = X+i; else O(1) X=0; Max ( O(N), O(1) ) = O (N) The Growth of Functions Example: Show that f(x) = x2 + 2x + 1 is O(x2). For x > 1 we have: x + 2x + 1 x + 2x + x 2 2 2 2 x + 2x + 1 4x 2 2 Therefore, for C = 4 and k = 1: f(x) Cx2 whenever x > k. f(x) is O(x2). The Growth of Functions Question: If f(x) is O(x2), is it also O(x3)? Yes. x3 grows faster than x2, so x3 grows also faster than f(x). Therefore, we always have to find the smallest simple function g(x) for which f(x) is O(g(x)). The Growth of Functions “Popular” functions g(n) are n log n, 1, 2n, n2, n!, n, n3, log n Listed from slowest to fastest growth: 1 log n n n log n n 2 n 3 2n n! Complexity Examples What does the following algorithm compute? procedure who_knows(a1, a2, …, an: integers) m := 0 for i := 1 to n-1 for j := i + 1 to n if |ai – aj| > m then m := |ai – aj| {m is the maximum difference between any two numbers in the input sequence} Comparisons: n-1 + n-2 + n-3 + … + 1 = (n – 1)n/2 = 0.5n2 – 0.5n Time complexity is O(n2). Complexity Examples Another algorithm solving the same problem: procedure max_diff(a1, a2, …, an: integers) min := a1 max := a1 for i := 2 to n if ai < min then min := ai else if ai > max then max := ai m := max - min Comparisons: 2n - 2 Time complexity is O(n). Asymptotic Notation O notation: asymptotic “less than”: f(n)=O(g(n)) implies: f(n) “≤” g(n) notation: asymptotic “greater than”: f(n)= (g(n)) implies: f(n) “≥” g(n) notation: asymptotic “equality”: f(n)= (g(n)) implies: f(n) “=” g(n) Definition: O(g), at most order g Let f,g are functions RR. We say that “f is at most order g”, if: c,k: f(x) cg(x), x>k “Beyond some point k, function f is at most a constant c times g (i.e., proportional to g).” “f is at most order g”, or “f is O(g)”, or “f=O(g)” all just mean that fO(g). Sometimes the phrase “at most” is omitted. Big-O Visualization k Points about the definition Note that f is O(g) as long as any values of c and k exist that satisfy the definition. But: The particular c, k, values that make the statement true are not unique: Any larger value of c and/or k will also work. You are not required to find the smallest c and k values that work. (Indeed, in some cases, there may be no smallest values!) However, you should prove that the values you choose do work. “Big-O” Proof Examples Show that 30n+8 is O(n). Show c,k: 30n+8 cn, n>k . Let c=31, k=8. Assume n>k=8. Then cn = 31n = 30n + n > 30n+8, so 30n+8 < cn. Show that n2+1 is O(n2). Show c,k: n2+1 cn2, n>k: . Let c=2, k=1. Assume n>1. Then cn2 = 2n2 = n2+n2 > n2+1, or n2+1< cn2. Big-O example, graphically Note 30n+8 isn’t cn = less than n 31n 30n+8 Value of function anywhere (n>0). It isn’t even less than 31n 30n+8 everywhere. n O(n) But it is less than n>k=8 31n everywhere to Increasing n the right of n=8. Common orders of magnitude Order-of-Growth in Expressions “O(f)” can be used as a term in an arithmetic expression . E.g.: we can write “x2+x+1” as “x2+O(x)” meaning “x2 plus some function that is O(x)”. Formally, you can think of any such expression as denoting a set of functions: “x2+O(x)” : {g | fO(x): g(x)= x2+f(x)} Useful Facts about Big O Constants ... c>0, O(cf)=O(f+c)=O(fc)=O(f) Sums: - If gO(f) and hO(f), then g+hO(f). - If gO(f1) and hO(f2), then g+hO(f1+f2) =O(max(f1,f2)) (Very useful!) More Big-O facts Products: If gO(f1) and hO(f2), then ghO(f1f2) Big O, as a relation, is transitive: fO(g) gO(h) fO(h) More Big O facts f,g & constants a,bR, with b0, af = O(f) (e.g. 3x2 = O(x2)) f+O(f) = O(f) (e.g. x2+x = O(x2)) |f|1-b = O(f) (e.g. x1 = O(x)) (logb |f|)a = O(f) (e.g. log x = O(x)) g=O(fg) (e.g. x = O(x log x)) fg O(g) (e.g. x log x O(x)) a=O(f) (e.g. 3 = O(x)) Definition: (g), at least order g Let f,g be any function RR. We say that “f is at least order g”, written (g), if c,k: f(x) cg(x), x>k “Beyond some point k, function f is at least a constant c times g (i.e., proportional to g).” Often, one deals only with positive functions and can ignore absolute value symbols. “f is at least order g”, or “f is (g)”, or “f= (g)” all just mean that f (g). Big- Visualization Definition: (g), exactly order g If fO(g) and gO(f) then we say “g and f are of the same order” or “f is (exactly) order g” and write f(g). Another equivalent definition: c1c2,k: c1g(x)f(x)c2g(x), x>k “Everywhere beyond some point k, f(x) lies in between two multiples of g(x).” (g) O(g) (g) (i.e., fO(g) and f(g) ) Big- Visualization Rules for Mostly like rules for O( ), except: f,g>0 & constants a,bR, with b>0, af (f) Same as with O. f (fg) unless g=(1) Unlike O. |f| 1-b (f), and Unlike with O. (logb |f|)c (f). Unlike with O. The functions in the latter two cases we say are strictly of lower order than (f). example n ? Determine whether: i (n 2 ) Quick solution: i 1 n i n(n 1) / 2 i 1 n ( n ) / 2 n ( n ) ( n 2 ) Other Order-of-Growth Relations o(g) = {f | c k: f(x) < cg(x), x>k} “The functions that are strictly lower order than g.” o(g) O(g) (g). (g) = {f | c k: cg(x) < f(x), x>k } “The functions that are strictly higher order than g.” (g) (g) (g). Relations Between the Relations Subset relations between order-of-growth sets. RR O( f ) ( f ) •f o( f ) ( f ) ( f ) Strict Ordering of Functions Temporarily let’s write fg to mean fo(g), f~g to mean f(g) Note that f ( x) f g lim 0. x g ( x ) Let k>1. Then the following are true: 1 log log n log n ~ logk n logk n n1/k n n log n nk kn n! nn … Common orders of magnitude Review: Orders of Growth Definitions of order-of-growth sets, g:RR O(g) {f | c,k: f(x) cg(x), x>k } o(g) {f | c k: f(x) < cg(x),x>k } (g) {f| | c,k : f(x) cg(x),x>k } (g) {f | c k: f(x) >cg(x), x>k} (g) {f | c1c2,k: c1g(x)f(x)|c2g(x), x>k} Examples Algorithmic Complexity The algorithmic complexity of a computation is some measure of how difficult it is to perform the computation. Measures some aspect of cost of computation (in a general sense of cost). Time complexity Best-case time = minimum time needed to execute the algorithm for inputs of size n Worst-case time = maximum time needed to execute the algorithm for inputs of size n Average-case time = average time needed focus on best and worst time Problem Complexity The complexity of a computational problem or task is the complexity of the algorithm with the lowest order of growth of complexity for solving that problem or performing that task. E.g. the problem of searching an ordered list has at most logarithmic time complexity. (Complexity is O(log n).) Tractable vs. Intractable Problems A problem or algorithm with at most polynomial time complexity is considered tractable (or feasible). P is the set of all tractable problems. A problem or algorithm that has more than polynomial complexity is considered intractable (or infeasible). Note n1,000,000 is technically tractable, but really impossible. nlog log log n is technically intractable, but easy. Such cases are rare though. Dealing with Intractable Problems Many times, a problem is intractable for a small number of input cases that do not arise in practice very often. Average running time is a better measure of problem complexity in this case. Find approximate solutions instead of exact solutions. Unsolvable problems It can be shown that there exist problems that no algorithm exists for solving them. Turing discovered in the 1930’s that there are problems unsolvable by any algorithm. Example: the halting problem (see page 176) Given an arbitrary algorithm and its input, will that algorithm eventually halt, or will it continue forever in an “infinite loop?” NP and NP-complete NP is the set of problems for which there exists a tractable algorithm for checking solutions to see if they are correct. NP-complete is a class of problems with the property that if any one of them can be solved by a polynomial worst-case algorithm, then all of them can be solved by polynomial worst-case algorithms. Satisfiability problem: find an assignment of truth values that makes a compound proposition true. P vs. NP We know PNP, but the most famous unproven conjecture in computer science is that this inclusion is proper (i.e., that PNP rather than P=NP). It is generally accepted that no NP- complete problem can be solved in polynomial time. Whoever first proves it will be famous! Questions Find the best big-O notation to describe the complexity of following algorithms: A binary search of n elements A linear search to find the smallest number in a list of n numbers An algorithm that lists all ways to put the numbers 1,2,3,…,n in a row. Questions (cont’d) An algorithm that prints all bit strings of length n An iterative algorithm to compute n! An algorithm that finds the average of n numbers by adding them and dividing by n An algorithm that prints all subsets of size three of the set {1,2,3,…,n} The best case analysis of a linear search of a list of size n. Questions (cont’d) The wors-case analysis of a linear search of a list of size n The number of print statements in the following while n>1 { print “hello” n=n/2 } Questions (cont’d) The number of print statements in the following for (i=1, i≤n; i++) for (j=1, j ≤n; j++) print “hello” The number of print statements in the following for (i=1, i≤n; i++) for (j=1, j ≤i; j++) print “hello” Introduction to Number Theory Introduction to Number Theory Number theory is about integers and their properties. We will start with the basic principles of divisibility, greatest common divisors, least common multiples, and modular arithmetic and look at some relevant algorithms. Division If a and b are integers with a 0, we say that a divides b if there is an integer c so that b = ac. When a divides b we say that a is a factor of b and that b is a multiple of a. The notation a | b means that a divides b. We write a χ b when a does not divide b (see book for correct symbol). Divisibility Theorems For integers a, b, and c it is true that if a | b and a | c, then a | (b + c) Example: 3 | 6 and 3 | 9, so 3 | 15. if a | b, then a | bc for all integers c Example: 5 | 10, so 5 | 20, 5 | 30, 5 | 40, … if a | b and b | c, then a | c Example: 4 | 8 and 8 | 24, so 4 | 24. Primes A positive integer p greater than 1 is called prime if the only positive factors of p are 1 and p. Note: 1 is not a prime A positive integer that is greater than 1 and is not prime is called composite. The fundamental theorem of arithmetic: Every positive integer can be written uniquely as the product of primes, where the prime factors are written in order of increasing size. Primes Examples: 15 = 3·5 48 = 2·2·2·2·3 = 24·3 17 = 17 100 = 2·2·5·5 = 22·52 512 = 2·2·2·2·2·2·2·2·2 = 29 515 = 5·103 28 = 2·2·7 Primes If n is a composite integer, then n has a prime divisor less than or equal n . This is easy to see: if n is a composite integer, it must have at least two prime divisors. Let the largest two be p1 and p2. Then p1p2 <= n. p and p2 cannot both be greater than 1 n , because then p1p2 > n. The Division Algorithm Let a be an integer and d a positive integer. Then there are unique integers q and r, with 0 r < d, such that a = dq + r. In the above equation, d is called the divisor, a is called the dividend, q is called the quotient, and r is called the remainder. The Division Algorithm Example: When we divide 17 by 5, we have 17 = 53 + 2. 17 is the dividend, 5 is the divisor, 3 is called the quotient, and 2 is called the remainder. The Division Algorithm Another example: What happens when we divide -11 by 3 ? Note that the remainder cannot be negative. -11 = 3(-4) + 1. -11 is the dividend, 3 is the divisor, -4 is called the quotient, and 1 is called the remainder. Greatest Common Divisors Let a and b be integers, not both zero. The largest integer d such that d | a and d | b is called the greatest common divisor of a and b. The greatest common divisor of a and b is denoted by gcd(a, b). Example 1: What is gcd(48, 72) ? The positive common divisors of 48 and 72 are 1, 2, 3, 4, 6, 8, 12, 16, and 24, so gcd(48, 72) = 24. Example 2: What is gcd(19, 72) ? The only positive common divisor of 19 and 72 is 1, so gcd(19, 72) = 1. Greatest Common Divisors Using prime factorizations: a = p1a1 p2a2 … pnan , b = p1b1 p2b2 … pnbn , where p1 < p2 < … < pn and ai, bi N for 1 i n gcd(a, b) = p1min(a1, b1 ) p2min(a2, b2 ) … pnmin(an, bn ) Example: a = 60 = 22 31 51 b = 54 = 21 33 50 gcd(a, b) = 21 31 50 = 6 Relatively Prime Integers Definition: Two integers a and b are relatively prime if gcd(a, b) = 1. Examples: Are 15 and 28 relatively prime? Yes, gcd(15, 28) = 1. Are 55 and 28 relatively prime? Yes, gcd(55, 28) = 1. Are 35 and 28 relatively prime? No, gcd(35, 28) = 7. Relatively Prime Integers Definition: The integers a1, a2, …, an are pairwise relatively prime if gcd(ai, aj) = 1 whenever 1 i < j n. Examples: Are 15, 17, and 27 pairwise relatively prime? No, because gcd(15, 27) = 3. Are 15, 17, and 28 pairwise relatively prime? Yes, because gcd(15, 17) = 1, gcd(15, 28) = 1 and gcd(17, 28) = 1. Least Common Multiples Definition: The least common multiple of the positive integers a and b is the smallest positive integer that is divisible by both a and b. We denote the least common multiple of a and b by lcm(a, b). Examples: lcm(3, 7) = 21 lcm(4, 6) = 12 lcm(5, 10) = 10 Least Common Multiples Using prime factorizations: a = p1a1 p2a2 … pnan , b = p1b1 p2b2 … pnbn , where p1 < p2 < … < pn and ai, bi N for 1 i n lcm(a, b) = p1max(a1, b1 ) p2max(a2, b2 ) … pnmax(an, bn ) Example: a = 60 = 22 31 51 b = 54 = 21 33 50 lcm(a, b) = 22 33 51 = 4275 = 540 GCD and LCM a = 60 = 22 31 51 b = 54 = 21 33 50 gcd(a, b) = 21 31 50 =6 lcm(a, b) = 22 33 51 = 540 Theorem: ab = gcd(a,b)lcm(a,b) Modular Arithmetic Let a be an integer and m be a positive integer. We denote by a mod m the remainder when a is divided by m. Examples: 9 mod 4 = 1 9 mod 3 = 0 9 mod 10 = 9 -13 mod 4 = 3 Congruences Let a and b be integers and m be a positive integer. We say that a is congruent to b modulo m if m divides a – b. We use the notation a b (mod m) to indicate that a is congruent to b modulo m. In other words: a b (mod m) if and only if a mod m = b mod m. Congruences Examples: Is it true that 46 68 (mod 11) ? Yes, because 11 | (46 – 68). Is it true that 46 68 (mod 22)? Yes, because 22 | (46 – 68). For which integers z is it true that z 12 (mod 10)? It is true for any z{…,-28, -18, -8, 2, 12, 22, 32, …} Theorem: Let m be a positive integer. The integers a and b are congruent modulo m if and only if there is an integer k such that a = b + km. Congruences Theorem: Let m be a positive integer. If a b (mod m) and c d (mod m), then a + c b + d (mod m) and ac bd (mod m). Proof: We know that a b (mod m) and c d (mod m) implies that there are integers s and t with b = a + sm and d = c + tm. Therefore, b + d = (a + sm) + (c + tm) = (a + c) + m(s + t) and bd = (a + sm)(c + tm) = ac + m(at + cs + stm). Hence, a + c b + d (mod m) and ac bd (mod m). Congruences Theorem: Let m be a positive integer. a b (mod m) iff a mod m = b mod m. Proof: Let a = mq1 + r1, and b = mq2 + r2. Only if part: a mod m = b mod m r1 = r2, therefore a – b = m(q1 – q2), and a b (mod m). If part: a b (mod m) implies a – b = mq mq1 + r1 – (mq2 + r2) = mq r1 – r2 = m(q – q1 + q2). Since 0 r1, r2 m, 0 |r1 - r2| m. The only multiple in that range is 0. Therefore r1 = r2, and a mod m = b mod m. The Euclidean Algorithm The Euclidean Algorithm finds the greatest common divisor of two integers a and b. For example, if we want to find gcd(287, 91), we divide 287 by 91: 287 = 913 + 14 We know that for integers a, b and c, if a | b and a | c, then a | (b + c). Therefore, any divisor (including their gcd) of 287 and 91 must also be a divisor of 287 - 913 = 14. Consequently, gcd(287, 91) = gcd(14, 91). The Euclidean Algorithm In the next step, we divide 91 by 14: 91 = 146 + 7 This means that gcd(14, 91) = gcd(14, 7). So we divide 14 by 7: 14 = 72 + 0 We find that 7 | 14, and thus gcd(14, 7) = 7. Therefore, gcd(287, 91) = 7. The Euclidean Algorithm In pseudocode, the algorithm can be implemented as follows: procedure gcd(a, b: positive integers) x := a y := b while y 0 begin r := x mod y x := y y := r end {x is gcd(a, b)} Representations of Integers Let b be a positive integer greater than 1. Then if n is a positive integer, it can be expressed uniquely in the form: n = akbk + ak-1bk-1 + … + a1b + a0, where k is a nonnegative integer, a0, a1, …, ak are nonnegative integers less than b, and ak 0. Example for b=10: 859 = 8102 + 5101 + 9100 Representations of Integers Example for b=2 (binary expansion): (10110)2 = 124 + 122 + 121 = (22)10 Example for b=16 (hexadecimal expansion): (we use letters A to F to indicate numbers 10 to 15) (3A0F)16 = 3163 + 10162 + 15160 = (14863)10 Representations of Integers How can we construct the base b expansion of an integer n? First, divide n by b to obtain a quotient q0 and remainder a0, that is, n = bq0 + a0, where 0 a0 < b. The remainder a0 is the rightmost digit in the base b expansion of n. Next, divide q0 by b to obtain: q0 = bq1 + a1, where 0 a1 < b. a1 is the second digit from the right in the base b expansion of n. Continue this process until you obtain a quotient equal to zero. Representations of Integers Example: What is the base 8 expansion of (12345)10 ? First, divide 12345 by 8: 12345 = 81543 + 1 1543 = 8192 + 7 192 = 824 + 0 24 = 83 + 0 3 = 80 + 3 The result is: (12345)10 = (30071)8. Representations of Integers procedure base_b_expansion(n, b: positive integers) q := n k := 0 while q 0 begin ak := q mod b q := q/b k := k + 1 end {the base b expansion of n is (ak-1 … a1a0)b } Addition of Integers How do we (humans) add two integers? 1 11 carry Example: 7583 + 4932 12515 1 1 carry Binary expansions: (1011)2 + (1010)2 ( 1 0 1 0 1 )2 Addition of Integers Let a = (an-1an-2…a1a0)2, b = (bn-1bn-2…b1b0)2. How can we algorithmically add these two binary numbers? First, add their rightmost bits: a0 + b0 = c02 + s0, where s0 is the rightmost bit in the binary expansion of a + b, and c0 is the carry. Then, add the next pair of bits and the carry: a1 + b1 + c0 = c12 + s1, where s1 is the next bit in the binary expansion of a + b, and c1 is the carry. Addition of Integers Continue this process until you obtain cn-1. The leading bit of the sum is sn = cn-1. The result is: a + b = (snsn-1…s1s0)2 Addition of Integers Example: Add a = (1110)2 and b = (1011)2. 0 a + b0 = 0 + 1 = 02 + 1, so that c0 = 0 and s0 = 1. 1 a + b1 + c0 = 1 + 1 + 0 = 12 + 0, so c1 = 1 and s1 = 0. 2 a + b2 + c1 = 1 + 0 + 1 = 12 + 0, so c2 = 1 and s2 = 0. 3 a + b3 + c2 = 1 + 1 + 1 = 12 + 1, so c3 = 1 and s3 = 1. 4 s = c3 = 1. Therefore, s = a + b = (11001)2. Addition of Integers procedure add(a, b: positive integers) c := 0 for j := 0 to n-1 begin d := (aj + bj + c)/2 sj := aj + bj + c – 2d c := d end sn := c {the binary expansion of the sum is (snsn-1…s1s0)2} Applications of Number Theory GCD as linear combination If a and b are positive integers, then there exist integers s and t such that gcd(a,b) = sa + tb Example Express gcd(252, 198) = 18 as a linear combination of 252 and 198 18 = 54 – 1*36 252 = 1*198 + 54 36 = 198 – 3*54 198 = 3*54 + 36 18 = 54 – 198 + 3*54 54 = 1*36 + 18 = 4*54 – 1*198 36 = 2*18 Start from next-to-last division 54 = 252 – 1*198 18 = 4*252 – 5*198 Hashing Function How to assign memory location that the data can be retrieved quickly h(k) = k mod m, where k is key and m is the number of available memory locations Example m = 111, h(064212848) = ? h(064212848) = 14 So this data will be placed at memory location = 14 Caesar‟s Encryption Invented by Julius Caesar to make a secret message Algorithm 1. Convert each alphabet in a message to its corresponding number p: A = 0, B = 1,…, Z = 25, and p 25 only 2. Select key k as a positive integer, Caesar himself often use k = 3 3. To encrypt, applies this formula: f(p) = (p+k) mod 26 4. Then convert each number back to alphabet and pack the ciphertext with its key 5. To decrypt, do step 1 and 2 then applies this formula: f-1(p) = (p-k) mod 26 6. If p+k or p-k is greater than or lesser than 25, shift it with 25 Example: Caesar‟s Encryption Encrypt “DISCRETE MATH” DISCRETE MATH 3 8 18 2 17 4 19 4 12 0 19 7 f(p+k) mod 26 6 11 21 5 20 7 22 7 15 3 22 10 GLVFUHWH PDWK GLVFUHWH PDWK Linear Congruences Congruence of a form ax b(mod m), where m is positive integer, a and b are integers, x is a variable is called Linear Congruence To solve this congruence is to find all possible x that satisfies the formula Inverse modulo More details in Section 3.7 Chinese Reminder Theorem from San-Tsu‟s puzzle, ancient Chinese mathematician can be solved using Modular arithmetic More details in Section 3.7 RSA Public key crypto system Based on modular exponential To encrypt C = Me mod n To decrypt Cd M(mod pq) More details see RSA section and Example 11 and 12 on page 241 - 244 Matrices Matrices A matrix is a rectangular array of numbers. An mn (“m by n”) matrix has exactly m horizontal rows, and n vertical columns. An nn matrix is called a square matrix, whose order is n. The plural of matrix is matrices. 2 3 5 1 a 32 7 0 matrix Example Row: 3 rows 1 1 Column: 2 columns 0 2 Size: 3 x 2 matrix 1 3 Square: NO Example Row: 2 rows 1 2 Column: 2 columns 3 4 Size: 2 x 2 matrix Square: YES Example 0 1 1 1 2 1 2 2 1 A B C 2 0 1 2 0 1 0 Rows and columns? 0 1 Size? D 2 0 Square? Equal? Matrix Equality Two matrices A and B are equal iff they have the same number of rows, the same number of columns, and all corresponding elements are equal. 3 2 3 2 0 1 6 1 6 0 Row and Column Order The rows in a matrix are usually indexed 1 to m from top to bottom. The columns are usually indexed 1 to n from left to right. Elements are indexed by row, then column. a1,1 a1, 2 a1,n a a2 , 2 a2 , n A [ai , j ] 2 ,1 am,1 am, 2 am , n Let The ith row of A is the 1 x n matrix jth column a11 a12 ... a1n a i1 a i2 ... a in a a 22 ... a 2n The jth column of A is the n x 1 matrix 21 . . . a1j A a ith row . . . 2j . . . . a n1 a n2 ... a nn . . a nj a11 a12 ... a1n a a 22 ... a 2n 21 . . . A . . . . . . a n1 a n2 ... a nn The (i,j)th element or entry of A is the element aij, that is, the number in the ith row and jth column of A. A convenient shorthand notation for expressing the matrix A is to write A = [aij], which indicates that A is the matrix with its (i,j)th element equal to aij. Matrix Arithmetic Addition/Sum Multiplication/Product Matrix Sums The sum A+B of two matrices A, B (which must have the same number of rows, and the same number of columns) is the matrix (also with the same shape) given by adding corresponding elements. A+B = [ai,j+bi,j] 2 6 9 3 11 9 0 8 11 3 11 5 Example: Addition Find A + B, where: 1 2 4 3 A B 3 4 2 1 Example: Addition Find A + B, where: 1 2 0 1 3 4 A 2 3 B 5 6 4 5 Example: Addition Find A + B, where: 1 2 3 4 0 1 2 A B 5 6 3 4 5 Matrix Products For an mk matrix A and a kn matrix B, the product AB is the mn matrix: k AB C [ci , j ] ai ,b, j 1 i.e., element (i,j) of AB is given by the vector dot product of the ith row of A and the jth column of B (considered as vectors). In other words, if AB = [cij], then cij = ai1b1j + ai2b2j + … + aikbkj. Note: Matrix multiplication is not commutative! Matrix Product Example An example matrix multiplication to practice in class: 0 1 1 0 0 1 1 1 0 5 1 2 0 3 2 0 2 0 3 2 11 3 1 0 3 1 Example: Multiplication Find AB, where: 1 2 4 3 A B 3 4 2 1 Find BA Example: Multiplication Find AB, where: 1 2 3 4 0 1 2 A B 5 6 3 4 5 Example: Multiplication Find AB, where: 0 1 2 A 2 1 B 3 4 5 Example: Multiplication Suppose A is a 6 × 8 matrix, B is an 8 × 5 matrix, and C is a 5 × 9 matrix. Find the number of rows, the number of columns, and the number of entries in A(BC). Example: Multiplication Find A2 where, 1 2 A 3 4 Identity Matrices The identity matrix of order n, In, is the order-n matrix with 1’s along the upper-left to lower-right diagonal and 0’s everywhere else. 1 0 0 1 if i j 0 1 0 I n 0 if i j 0 0 1 Identity matrix Multiplying a matrix by an appropriate sized identity matrix does not change this matrix. In other words, when A is an m x n matrix, we have AIn = ImA = A Example: Identity matrix Find identity matrix of each of the following matrices: 1 2 3 1 2 4 3 2 A B C 1 2 3 4 2 1 2 3 3 0 1 2 D 3 4 5 Matrix Inverses For some (but not all) square matrices A, there exists a unique multiplicative inverse A-1 of A, a matrix such that A-1A = In. If the inverse exists, it is unique, and A-1A = AA-1. We won’t go into the algorithms for matrix inversion... Matrix Multiplication Algorithm procedure matmul(matrices A: mk, B: kn) for i := 1 to m (m)· What’s the of its time complexity? for j := 1 to n begin (n)·( (1)+ Answer: cij := 0 (mnk) for q := 1 to k (k) · cij := cij + aiqbqj (1)) end {C=[cij] is the product of A and B} Powers of Matrices If A is an nn square matrix and p0, then: Ap AAA···A (A0 In) 3 p times 2 1 2 1 2 1 2 1 1 0 1 1 0 1 0 0 2 1 3 2 Example: 2 1 1 0 4 3 3 2 Matrix Transposition If A=[aij] is an mn matrix, the transpose of A (often written At or AT) is the nm matrix given by At = B = [bij] = [aji] (1in,1jm) t 2 0 2 1 3 0 1 2 1 1 3 2 Flip across diagonal Example: Transpose Find transpose of the following matrices: 1 2 1 2 3 0 1 2 A B 4 D 3 4 3 4 5 5 6 Symmetric Matrices A square matrix A is symmetric iff A=At. I.e., i,jn: aij = aji . Which is symmetric? 1 1 2 1 3 3 0 1 1 1 1 0 1 0 2 1 1 1 3 1 2 1 1 2 Definition 7: Symmetric A square matrix A is called symmetric if A = At. Thus A = [aij] is symmetric if aij = aji for all i and j with 1 ≤ i ≤ n and 1 ≤ j ≤ n. Example: Symmetric 2 4 2 1 4 1 3 4 1 1 3 2 3 1 4 2 0 2 1 Zero-One Matrices All elements of a zero-one matrix are 0 or 1 Representing False & True respectively. Useful for representing other structures. E.g., relations, directed graphs The meet of A, B (both mn zero-one matrices): AB : [aijbij] = [aij bij] The join of A, B: AB : [aijbij] Boolean Products Let A=[aij] be an mk zero-one matrix, & let B=[bij] be a kn zero-one matrix, The boolean product of A and B is like normal matrix , but using instead + in the row-column “vector dot product.” k A⊙B C [cij ] ai bj 1 Arithmetic/Boolean Products k AB C [ci , j ] ai ,b, j 1 k A⊙B C [cij ] ai bj 1 Boolean Powers For a square zero-one matrix A, and any k0, the kth Boolean power of A is simply the Boolean product of k copies of A. A[k] A⊙A⊙…⊙A k times Homework 3 Chapter 3 Section 3.1: 1, 5, 22, 39, 42 Section 3.2: 1, 7, 9, 19, 21 Section 3.3: 1, 3, 9, 11, 25, 27 Section 3.4: 1, 9, 17, 31, 32 Section 3.5: 1, 21, 25 Section 3.6: 1, 5, 19, 23 Section 3.7: 7, 21 Section 3.8: 1, 3, 5, 31, 33 Supplementary: none

DOCUMENT INFO

Shared By:

Categories:

Tags:

Stats:

views: | 52 |

posted: | 8/3/2011 |

language: | English |

pages: | 192 |

OTHER DOCS BY MikeJenny

How are you planning on using Docstoc?
BUSINESS
PERSONAL

By registering with docstoc.com you agree to our
privacy policy and
terms of service, and to receive content and offer notifications.

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.