# Discrete Mathematics Lecture Algorithms The Integers

Document Sample

```					         461191 Discrete Mathematics

Lecture 3: Algorithms, The
Integers, and Matrices

San Ratanasanya
CS, KMUTNB

Michael P. Frank, University of Florida, University of Nevada, and University of Maryland at
Baltimore
Today‟s Topics
   Review
   Algorithms
   Algorithmic and Problem Complexity
   Introduction to Number Theory
   Applications of Number Theory
   Matrices
…from last week…
   Sets and Set operations
   Set is an unordered collection of objects
   Operations on Set: union, intersection, difference, complement,
symmetric difference
   Venn Diagram and equivalence
   Functions
   Domain, Codomain, Range
   One-to-one, onto, bijection, inverse, and composite functions
   Sequences
   Represents ordered list of elements
 a1,a,2a3,…,an or {an}

   It is convenient to describe Sequence using formula
 an = 2an-1

   Summations
Example: Set
   Prove that A  A = U is true

A = {x | x  A}
A = {x | x  A}
A  A = {x | x  A  x  A}
A  A = {x | x  U}
AA=U
Example: Function
   Let f(x) = x2/3. Find f(S) if S = {x | xZ  |x|  3}
f(3) = 32/3 = 3, f(2) = 22/3 = 1, f(1) = 12/3 = 0, f(0) = 0
 f(S) = {0, 1, 3}

   Suppose that g is a function from A to B and f is a function form B
to C. Show that if both f and g are one-to-one functions, then fg
is also one-to-one.
We need to show that x A  y A (x  y  f(g(x))  f(g(y))).
Or we need to show that x A  y A (x  y  g(x)  g(y))
 x B  y B (x  y  f(x)  f(y)).
Given that g is one-to-one, therefore g(x)  g(y) in B.
Given that f is one-to-one, therefore f(x)  f(y) in C.
We can now conclude that f(g(x))  f(g(y)).
Example: Sequence
   Find a simple formula for an if the first 10
terms of the sequence {an} are 1, 7, 25, 79,
241, 727, 2185, 6559, 19681, 59047

a1 = 1
an = 3n - 2
Example: Summations
n
3    3                          a a          i 1   ?
i  ?
i
i 1
 a1  a0  a2  a1 
i 1 j  0
3        3                             a3  a2    an 1  an  2
  i 1
i 1     j 0
 an  an 1
3                                       a n  a0
  4i
i 1
n

 4 i
3
 (2k  1)  ?
k 1
i 1                                   n           n       n

3 * (3  1)                       2 k   1  2 k  n
 4*                                      k 1        k 1    k 1
2                                      (n)(n  1)
 24                                  2*                    n  n2  n  n
2
 n2
   Homeworks due today
   Homework 1.1 – 1.4
   Homework 2
from class webpage
   they are TOO BIG…
class webpage
   Prolog, ISETLW
   Python, Scheme
webpage
Algorithms
   The foundation of computer programming.
   Most generally, an algorithm just means a definite
procedure for performing some sort of task.
   A computer program is simply a description of an
algorithm, in a language precise enough for a computer to
understand, requiring only operations that the computer
   We say that a program implements (or “is an
implementation of”) its algorithm.
   How to add any two natural numbers written
in decimal on paper, using carries.
   Similar: Subtraction using borrowing.
   Multiplication & long division.
   How to register for classes at KMUTNB.
Programming Languages
   Some common programming languages:
   Newer: Java, C, C++, C#, Visual Basic,
JavaScript, Perl, Tcl, Pascal, many others…
   Older: Fortran, Cobol, Lisp, Basic
   Assembly languages, for low-level coding.
   In this class we will use an informal, Pascal-
like “pseudo-code” language.
   You should know at least 1 real language!
Algorithm Example (English)
   Task: Given a sequence {ai}=a1,…,an, aiN,
say what its largest element is.
   One algorithm for doing this, in English:
   Set the value of a temporary variable v (largest
element seen so far) to a1’s value.
   Look at the next element ai in the sequence.
   If ai>v, then re-assign v to the number ai.
   Repeat then previous 2 steps until there are no
more elements in the sequence, & return v.
Executing an Algorithm
   When you start up a piece of software, we
say the program or its algorithm are being
run or executed by the computer.
   Given a description of an algorithm, you
can also execute it by hand, by working
through all of its steps with pencil & paper.
   Before ~1940, “computer” meant a person
whose job was to execute algorithms!
Executing the Max algorithm
   Let {ai}=7,12,3,15,8. Find its maximum…
   Set v = a1 = 7.
   Look at next element: a2 = 12.
   Is a2>v? Yes, so change v to 12.
   Look at next element: a2 = 3.
   Is 3>12? No, leave v alone….
   Is 15>12? Yes, v=15…
Algorithm Characteristics
Some important general features of algorithms:
 Input. Information or data that comes in.

 Output. Information or data that goes out.

 Definiteness. Algorithm is precisely defined.

 Correctness. Outputs correctly relate to inputs.

 Finiteness. Won’t take forever to describe or run.

 Effectiveness. Individual steps are all do-able.

 Generality. Works for many possible inputs.

 Efficiency. Takes little time & memory to run.
Our Pseudocode Language
procedure                 for variable := initial
name(argument: type)     value to final value
variable := expression         statement
informal statement        while condition
begin statements end        statement
{comment}                 procname(arguments)
if condition then           Not defined in book:
statement [else        return expression
statement]
What is Pseudocode
   Pseudocode is an intermediate step between
language description, normally English, of the
steps of a procedure and a specification of this
procedure using an actual programming language.
                                  ่   ั
เป็ นการอธิบายโปรแกรมด้วยภาษาทีใช้กนทั่วไป ด้วยลักษณะที่
่
คล้ายภาษาทีใช้ในการเขียนโปรแกรม

Human                         Programming
Pseudocode
Languages                        Languages
What is Program
   Program = Algorithm + Data Structure
   Each step of a program may make changes to variables
   Trace = a sequence of changes in variables as each step of
program is executed

   Trace is a simple model of program executions
   State = variable
procedure procname(arg: type)
   Declares that the following text defines a
procedure named procname that takes
inputs (arguments) named arg which are
data objects of the type type.
   Example:
procedure maximum(L: list of integers)
[statements defining maximum…]
variable := expression
   An assignment statement evaluates the
expression expression, then reassigns the
variable variable to the value that results.
   Example assignment statement:
v := 3x+7    (If x is 2, changes v to 13.)
   In pseudocode (but not real code), the
expression might be informally stated:
   x := the largest integer in the list L
Informal statement
   Sometimes we may write a statement as an
informal English imperative, if the meaning
is still clear and precise: e.g.,“swap x and y”
   Keep in mind that real programming
languages never allow this.
   When we ask for an algorithm to do so-
and-so, writing “Do so-and-so” isn’t
enough!
   Break down algorithm into detailed steps.
begin statements end
   Groups a sequence of           Allows the sequence
statements together:            to be used just like a
begin                           single statement.
statement 1                  Might be used:
statement 2                      After a procedure
…                                 declaration.
statement n                      In an if statement after
end                                 then or else.
   In the body of a for or
Curly braces {} are used instead
while loop.
in many languages.
{comment}
   Not executed (does nothing).
   Natural-language text explaining some
aspect of the procedure to human readers.
   Also called a remark in some real
programming languages, e.g. BASIC.
   Example, might appear in a max program:
   {Note that v is the largest integer seen so far.}
Conditional Constructions
if condition then statement         if condition then statement 1
else statement 2
if condition then
begin
Block of                    if condition 1 then statement 1
statements                         else if condition 2 then statement 2
end     if condition then          else if condition 2 then statement 2
begin                     ………………………………………………………
statement 1             else if condition n then statement n
end                       else statement n+1
else
begin
statement 2
end
if condition then statement
   Evaluate the propositional expression
condition.
   If the resulting truth value is True, then
execute the statement statement;
   otherwise, just skip on ahead to the next
statement after the if statement.
   Variant: if cond then stmt1 else stmt2
   Like before, but iff truth value is False,
executes stmt2.
Loop Constructions
for variable := initial value to final value
statement
for all element with certain property
for variable := initial value to
final value
while condition      while condition
begin
statement          begin
Block of statements
end                                                 Block of statements 1
end
sum := 0
sum := 0                                while n > 0
for i := 1 to n                         begin
sum := sum + i                         sum := sum + n
n := n - 1
end
while condition statement
   Evaluate the propositional (Boolean)
expression condition.
   If the resulting value is True, then execute
statement.
   Continue repeating the above two actions
over and over until finally the condition
evaluates to False; then proceed to the
next statement.
while condition statement
   Also equivalent to infinite nested ifs, like so:
if condition
begin
statement
if condition
begin
statement
…(continue infinite nested if’s)
end
end
for var := initial to final stmt
   Initial is an integer expression.
   Final is another integer expression.
   Semantics: Repeatedly execute stmt, first with
variable var := initial, then with var := initial+1,
then with var := initial+2, etc., then finally with
var := final.
   Question: What happens if stmt changes the
value of var, or the value that initial or final
evaluates to?
for var := initial to final stmt
   For can be exactly defined in terms of
while, like so:
begin
var := initial
while var  final
begin
stmt
var := var + 1
end
end
procedure(argument)
   A procedure call statement invokes the
named procedure, giving it as its input the
value of the argument expression.
   Various real programming languages refer
to procedures as functions (since the
procedure call notation works similarly to
function application f(x)), or as subroutines,
subprograms, or methods.
Nested
   Block of statements
   Conditional
   Loop
   Procedure – a call to procedure within procedure
Procedure statistic (L:list of
integers)
begin
max(L)
min(L)
average(L)
………..
end
Max procedure in pseudocode
procedure max(a1, a2, …, an: integers)
v := a1 {largest element so far}
for i := 2 to n {go thru rest of elems}
if ai > v then v := ai {found bigger?}
{at this point v’s value is the same as
the largest integer in the list}
return v
Inventing an Algorithm
   Requires a lot of creativity and intuition
   Like writing proofs.
   Unfortunately, we can’t give you an
algorithm for inventing algorithms.
   Just look at lots of examples…
   And practice (preferably, on a computer)
   And look at more examples…
   And practice some more… etc., etc.
Algorithm-Inventing Example
   Suppose we ask you to write an algorithm
to compute the predicate:
IsPrime:N→{T,F}
   Computes whether a given natural number is a
prime number.
definition of the desired function:
n: IsPrime(n)  ¬1<d<n: d|n
Means d divides n
evenly (without remainder)
IsPrime example, cont.
   Notice that the negated exponential can be
rewritten as a universal:            Means d does not
divide n evenly
¬1<d<n: d|n      1<d<n: d | n             (the remainder is ≠0)
 2≤ d ≤ n−1: d | n
   This universal can then be translated directly into
a corresponding for loop:
for d := 2 to n−1 { Try all potential divisors >1 & <n }
if d|n then return F { n has divisor d; not prime }
return T { no divisors were found; n must be prime}
Optimizing IsPrime
   The IsPrime algorithm can be further optimized:
for d := 2 to n1/2       Note smaller range of search.
if d|n then return F
return T
   This works because of this theorem: If n has any
(integer) divisors, it must have one less than n1/2.
Proof: Suppose n’s smallest divisor >1 is a, and let
b :≡ n/a. Then n = ab, but if a > n1/2 then b > n1/2 (since a
is n’s smallest divisor) and so n = ab > (n1/2)2 = n, an
absurdity. Further optimizations are possible:
- E.g., only try divisors that
are primes less than n1/2.
   Problem of searching an ordered list.
   Given a list L of n elements that are sorted into
a definite order (e.g., numeric, alphabetical),
   And given a particular element x,
   Determine whether x appears in the list,
   and if so, return its index (position) in the list.
   Problem occurs often in many contexts.
   Let’s find an efficient algorithm!
Search alg. #1: Linear Search
procedure linear search
(x: integer, a1, a2, …, an: distinct integers)
i := 1 {start at beginning of list}
while (i  n  x  ai) {not done, not found}
i := i + 1 {go to the next position}
if i  n then location := i {it was found}
else location := 0 {it wasn’t found}
Search alg. #2: Binary Search
   Basic idea: On each step, look at the middle
element of the remaining list to eliminate
half of it, and quickly zero in on the desired
element.

<x   <x <x >x
Search alg. #2: Binary Search
procedure binary search
(x:integer, a1, a2, …, an: distinct integers)
i := 1 {left endpoint of search interval}
j := n {right endpoint of search interval}
while i<j begin {while interval has >1 item}
m := (i+j)/2 {midpoint}
if x>am then i := m+1 else j := m
end
if x = ai then location := i else location := 0
return location
binary search for the letter „j‟
search interval
a c d f g h j l m o p r s u v x z

center element

search interval

a c d f g h j l m o p r s u v x z

center element
binary search for the letter „j‟
search interval

a c d f g h j l m o p r s u v x z

center element

search interval
a c d f g h j l m o p r s u v x z
center element
binary search for the letter „j‟

search interval

a c d f g h j l m o p r s u v x z

center element
found !
Practice exercises
   Devise an algorithm that finds the sum of
all the integers in a list. [2 min]
   procedure sum(a1, a2, …, an: integers)
s := 0 {sum of elems so far}
for i := 1 to n {go thru all elems}
s := s + ai {add current item}
{at this point s is the sum of all items}
return s
Sorting Algorithms
   Sorting is a common operation in many
applications.
   It is also widely used as a subroutine in
other data-processing algorithms.
   Two sorting algorithms shown in textbook:
   Bubble sort       However, these are not
very efficient, and you should
   Insertion sort    not use them on large data sets!
We’ll see some more efficient algorithms later in the course.
Bubble Sort
   Smallest elements “float” up to the top of
the list, like bubbles in a container of liquid.
30     30       1      1      1      1
31      1      30     30      2      2
1     31      31      2     30      3
32     32       2     31      3     30
33      2      32      3     31     31
2     33       3     32     32     32
34      3      33     33     33     33
3     34      34     34     34     34
Insertion Sort Algorithm
   English description of algorithm:
   For each item in the input list,
   “Insert” it into the correct place in the sorted
output list generated so far. Like so:
   Use linear or binary search to find the location where the
new item should be inserted.
   Then, shift the items from that position onwards down by
one position.
   Put the new item in the hole remaining.
Pseudocodes
Procedure insertionsort(a1,…,an: real numbers with n  2)
for j := 2 to n
begin
Procedure bubblesort(a1,…,an: real numbers with n  2)
i := 1
for i := 1 to n-1
while aj > ai
for j := 1 to n-1
i := i +1
if aj > aj+1 then interchange aj and aj+1
m : = aj
{a1,…,an is in increasing order}
for k := 0 to j – i – 1
aj-k := aj-k-1
ai := am
end {a1, a2,…,an are sorted}
Greedy Algorithms
    Any algorithms selecting the „best‟
choice at each step are called Greedy
    Can solve optimization problem
Procedure change(c1, c2,…,cr: values of denomination of coins,
where c1 > c2 >… > cr; n: a positive integer)
for i := 1 to r
while n  ci
begin
add a coin with value ci to the change
n := n - ci
end
Analysis of Algorithms
   What is the goal of analysis of algorithms?
   To compare algorithms mainly in terms of
running time but also in terms of other factors
(e.g., memory requirements, programmer's
effort etc.)
   What do we mean by running time
analysis?
   Determine how running time increases as the
size of the problem increases.
Is Binary Search more efficient?
   Number of iterations:
   For a list of n elements, Binary Search can
execute at most log2 n times!!
   Linear Search, on the other hand, can execute up
to n times !!

Average Number of Iterations
Length     Linear Search    Binary Search
10               5.5              2.9
100              50.5              5.8
1,000             500.5              9.0
10,000            5000.5             12.0
Is Binary Search more efficient?
   Number of computations per iteration:
   Binary search does more computations than
Linear Search per iteration.
   Overall:
   If the number of components is small (say, less
than 20), then Linear Search is faster.
   If the number of components is large, then
Binary Search is faster.
How do we analyze algorithms?

   We need to define a number of objective measures.

(1) Compare execution times?
Not good: times are specific to a particular computer !!

(2) Count the number of statements
executed?
Not good: number of statements vary with the
programming language as well as the style of the
individual programmer.
Example (# of statements)
Algorithm 1       Algorithm 2

arr[0] = 0;     for(i=0; i<N; i++)
arr[1] = 0;      arr[i] = 0;
arr[2] = 0;
...
arr[N-1] = 0;
How do we analyze algorithms?

(3) Express running time as a function of
the input size n (i.e., f(n)).
   To compare two algorithms with running times
f(n) and g(n), we need a rough measure of
how fast a function grows.
   Such an analysis is independent of machine
time, programming style, etc.
Computing running time

   Associate a "cost" with each statement and find
the "total cost“ by finding the total number of
times each statement is executed.
   Express running time in terms of the size of the
problem.
Computing running time (contd.)

Algorithm 1                Algorithm 2
Cost                          Cost
arr[0] = 0;     c1        for(i=0; i<N; i++)    c2
arr[1] = 0;     c1          arr[i] = 0;        c1
arr[2] = 0;     c1
...
arr[N-1] = 0; c1
-----------                  -------------
c1+c1+...+c1 = c1 x N        (N+1) x c2 + N x c1 =
(c2 + c1) x N + c2
Computing running time (cont.)
Cost
sum = 0;                    c1
for(i=0; i<N; i++)           c2
for(j=0; j<N; j++)         c2
sum += arr[i][j];        c3
------------
c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N x N
Comparing Functions Using
Rate of Growth
  Consider the example of buying elephants and
goldfish:
Cost: cost_of_elephants + cost_of_goldfish
Cost ~ cost_of_elephants (approximation)
 The low order terms in a function are relatively
insignificant for large n
n4 + 100n2 + 10n + 50 ~ n4
i.e., n4 + 100n2 + 10n + 50 and n4 have the same
rate of growth
Rate of Growth ≡Asymptotic Analysis

   Using rate of growth as a measure to
compare different functions implies
comparing them asymptotically.
   If f(x) is faster growing than g(x), then f(x)
always eventually becomes larger than g(x)
in the limit (for large enough values of x).
Example
   Suppose you are designing a web site to process
user data (e.g., financial records).
   Suppose program A takes fA(n)=30n+8
microseconds to process any n records, while
program B takes fB(n)=n2+1 microseconds to
process the n records.
   Which program would you choose, knowing you’ll
want to support millions of users?
Visualizing Orders of Growth
   On a graph, as
you go to the

Value of function 
fA(n)=30n+8
right, a faster
growing
function
fB(n)=n2+1
eventually
becomes
larger...                           Increasing n 
Big-O Notation
   We say fA(n)=30n+8 is order n, or O(n).
It is, at most, roughly proportional to n.
   fB(n)=n2+1 is order n2, or O(n2). It is, at most,
roughly proportional to n2.
   In general, an O(n2) algorithm will be
slower than O(n) algorithm.
   Warning: an O(n2) function will grow faster
than an O(n) function.
More Examples …

   We say that n4 + 100n2 + 10n + 50 is of the
order of n4 or O(n4)
   We say that 10n3 + 2n2 is O(n3)
   We say that n3 - n2 is O(n3)
   We say that 10 is O(1),
   We say that 1273 is O(1)
Big-O Visualization
Computing running time
Algorithm 1                 Algorithm 2
Cost                          Cost
arr[0] = 0;     c1        for(i=0; i<N; i++)    c2
arr[1] = 0;     c1          arr[i] = 0;        c1
arr[2] = 0;     c1
...
arr[N-1] = 0; c1
-----------                  -------------
c1+c1+...+c1 = c1 x N        (N+1) x c2 + N x c1 =
(c2 + c1) x N + c2

O(n)
Computing running time (cont.)
Cost
sum = 0;                        c1
for(i=0; i<N; i++)               c2
for(j=0; j<N; j++)             c2
sum += arr[i][j];             c3
------------
c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N x N
O(n2)
Running time of various statements
while-loop   for-loop
Examples

i = 0;
while (i<N) {
X=X+Y;             // O(1)
result = mystery(X); // O(N), just an example...
i++;          // O(1)
}
   The body of the while loop: O(N)
   Loop is executed:           N times
N x O(N) = O(N2)
Examples (cont.’d)
if (i<j)
for ( i=0; i<N; i++ )     O(N)

X = X+i;
else               O(1)
X=0;

Max ( O(N), O(1) ) = O (N)
The Growth of Functions

   Example:
   Show that f(x) = x2 + 2x + 1 is O(x2).

For x > 1 we have:
x + 2x + 1  x + 2x + x
2            2     2   2

 x + 2x + 1  4x
2             2

Therefore, for C = 4 and k = 1:
f(x)  Cx2 whenever x > k.

    f(x) is O(x2).
The Growth of Functions

   Question: If f(x) is O(x2), is it also O(x3)?

   Yes. x3 grows faster than x2, so x3 grows also faster than f(x).

Therefore, we always have to find the smallest simple function
g(x) for which f(x) is O(g(x)).
The Growth of Functions

“Popular” functions g(n) are
n log n, 1, 2n, n2, n!, n, n3, log n

Listed from slowest to fastest growth:
 1
 log n
 n
 n log n
 n
2

 n
3

 2n
 n!
Complexity Examples

What does the following algorithm compute?
procedure who_knows(a1, a2, …, an: integers)
m := 0
for i := 1 to n-1
for j := i + 1 to n
if |ai – aj| > m then m := |ai – aj|
{m is the maximum difference between any two numbers in the input
sequence}
Comparisons: n-1 + n-2 + n-3 + … + 1

= (n – 1)n/2 = 0.5n2 – 0.5n

   Time complexity is O(n2).
Complexity Examples

   Another algorithm solving the same problem:
procedure max_diff(a1, a2, …, an: integers)
min := a1
max := a1
for i := 2 to n
if ai < min then min := ai
else if ai > max then max := ai
m := max - min
Comparisons: 2n - 2

   Time complexity is O(n).
Asymptotic Notation
   O notation: asymptotic “less than”:

   f(n)=O(g(n)) implies: f(n) “≤” g(n)

    notation: asymptotic “greater than”:

   f(n)=  (g(n)) implies: f(n) “≥” g(n)

    notation: asymptotic “equality”:

   f(n)=  (g(n)) implies: f(n) “=” g(n)
Definition: O(g), at most order g
Let f,g are functions RR.
 We say that “f is at most order g”, if:

c,k: f(x)  cg(x), x>k
   “Beyond some point k, function f is at most a
constant c times g (i.e., proportional to g).”
   “f is at most order g”, or “f is O(g)”, or
“f=O(g)” all just mean that fO(g).
   Sometimes the phrase “at most” is omitted.
Big-O Visualization

k
   Note that f is O(g) as long as any values of c
and k exist that satisfy the definition.
   But: The particular c, k, values that make
the statement true are not unique: Any
larger value of c and/or k will also work.
   You are not required to find the smallest c
and k values that work. (Indeed, in some
cases, there may be no smallest values!)
However, you should prove that the values you choose do work.
“Big-O” Proof Examples
   Show that 30n+8 is O(n).
   Show c,k: 30n+8  cn, n>k .
   Let c=31, k=8. Assume n>k=8. Then
cn = 31n = 30n + n > 30n+8, so 30n+8 < cn.
   Show that n2+1 is O(n2).
   Show c,k: n2+1  cn2, n>k: .
 Let c=2, k=1. Assume n>1. Then
cn2 = 2n2 = n2+n2 > n2+1, or n2+1< cn2.
Big-O example, graphically
   Note 30n+8 isn’t                          cn =
less than n                               31n    30n+8

Value of function 
anywhere (n>0).
   It isn’t even
less than 31n                                            30n+8
everywhere.
n
O(n)
   But it is less than                      n>k=8 
31n everywhere to
Increasing n 
the right of n=8.
Common orders of magnitude
Order-of-Growth in Expressions
 “O(f)” can be used as a term in an arithmetic
expression .
E.g.: we can write “x2+x+1” as “x2+O(x)” meaning
“x2 plus some function that is O(x)”.
 Formally, you can think of any such expression as
denoting a set of functions:
“x2+O(x)” : {g | fO(x): g(x)= x2+f(x)}
   Constants ...
c>0, O(cf)=O(f+c)=O(fc)=O(f)
   Sums:
- If gO(f) and hO(f), then g+hO(f).
- If gO(f1) and hO(f2), then
g+hO(f1+f2) =O(max(f1,f2))
(Very useful!)
More Big-O facts
   Products:
If gO(f1) and hO(f2), then ghO(f1f2)

   Big O, as a relation, is transitive:
fO(g)  gO(h)  fO(h)
More Big O facts
    f,g & constants a,bR, with b0,
   af = O(f)          (e.g. 3x2 = O(x2))
   f+O(f) = O(f)     (e.g. x2+x = O(x2))
   |f|1-b = O(f)        (e.g. x1 = O(x))
   (logb |f|)a = O(f) (e.g. log x = O(x))
   g=O(fg)             (e.g. x = O(x log x))
   fg  O(g)           (e.g. x log x  O(x))
   a=O(f)             (e.g. 3 = O(x))
Definition: (g), at least order g
Let f,g be any function RR.
 We say that “f is at least order g”, written (g), if

c,k: f(x)  cg(x), x>k
   “Beyond some point k, function f is at least a constant c
times g (i.e., proportional to g).”
   Often, one deals only with positive functions and can
ignore absolute value symbols.
   “f is at least order g”, or “f is (g)”, or “f= (g)” all
just mean that f (g).
Big-  Visualization
Definition: (g), exactly order g
  If fO(g) and gO(f) then we say “g and f are
of the same order” or “f is (exactly) order g”
and write f(g).
 Another equivalent definition:
c1c2,k: c1g(x)f(x)c2g(x), x>k
 “Everywhere beyond some point k, f(x) lies in
between two multiples of g(x).”
 (g)  O(g)  (g)

(i.e., fO(g) and f(g) )
Big-  Visualization
Rules for 
   Mostly like rules for O( ), except:
    f,g>0 & constants a,bR, with b>0,
af  (f)          Same as with O.
f  (fg) unless g=(1)  Unlike O.
|f| 1-b  (f), and       Unlike with O.
(logb |f|)c  (f).       Unlike with O.
   The functions in the latter two cases we say
are strictly of lower order than (f).
 example
 n ?
   Determine whether:   i (n 2 )
   Quick solution:     i 1 
 n 
  i   n(n  1) / 2
 i 1 
 n  ( n ) / 2
 n  ( n )
 ( n 2 )
Other Order-of-Growth Relations
   o(g) = {f | c k: f(x) < cg(x), x>k}
“The functions that are strictly lower order
than g.” o(g)  O(g)  (g).

   (g) = {f | c k: cg(x) < f(x), x>k }
“The functions that are strictly higher order
than g.” (g)  (g)  (g).
Relations Between the Relations
   Subset relations between order-of-growth
sets.
RR
O( f )       ( f )
•f
o( f )    ( f )   ( f )
Strict Ordering of Functions
   Temporarily let’s write fg to mean fo(g),
f~g to mean f(g)
   Note that               f ( x)
f  g  lim              0.
x  g ( x )
   Let k>1. Then the following are true:
1  log log n  log n ~ logk n  logk n
 n1/k  n  n log n  nk  kn  n!  nn …
Common orders of magnitude
Review: Orders of Growth
Definitions of order-of-growth sets,
g:RR
   O(g)  {f |  c,k: f(x)  cg(x), x>k }
   o(g)  {f | c k: f(x) < cg(x),x>k }
   (g)  {f| | c,k : f(x)  cg(x),x>k }
   (g)  {f |  c k: f(x) >cg(x), x>k}
   (g)  {f | c1c2,k: c1g(x)f(x)|c2g(x), x>k}
Examples
Algorithmic Complexity
   The algorithmic complexity of a
computation is some measure of how
difficult it is to perform the computation.
   Measures some aspect of cost of
computation (in a general sense of cost).
Time complexity
   Best-case time = minimum time needed to
execute the algorithm for inputs of size n
   Worst-case time = maximum time needed
to execute the algorithm for inputs of size n
   Average-case time = average time needed

   focus on best and worst time
Problem Complexity
   The complexity of a computational problem
or task is the complexity of the algorithm
with the lowest order of growth of
complexity for solving that problem or
   E.g. the problem of searching an ordered
list has at most logarithmic time complexity.
(Complexity is O(log n).)
Tractable vs. Intractable Problems
   A problem or algorithm with at most polynomial
time complexity is considered tractable (or
feasible). P is the set of all tractable problems.
   A problem or algorithm that has more than
polynomial complexity is considered intractable
(or infeasible).
   Note
   n1,000,000 is technically tractable, but really impossible.
nlog log log n is technically intractable, but easy.
   Such cases are rare though.
Dealing with Intractable Problems
   Many times, a problem is intractable for a small
number of input cases that do not arise in
practice very often.
   Average running time is a better measure of problem
complexity in this case.
   Find approximate solutions instead of exact solutions.
Unsolvable problems
   It can be shown that there exist problems that no
algorithm exists for solving them.
   Turing discovered in the 1930’s that there are
problems unsolvable by any algorithm.
   Example: the halting problem (see page 176)
   Given an arbitrary algorithm and its input, will that
algorithm eventually halt, or will it continue forever in
an “infinite loop?”
NP and NP-complete
   NP is the set of problems for which there exists a
tractable algorithm for checking solutions to see if
they are correct.
   NP-complete is a class of problems with the
property that if any one of them can be solved by
a polynomial worst-case algorithm, then all of
them can be solved by polynomial worst-case
algorithms.
   Satisfiability problem: find an assignment of truth
values that makes a compound proposition true.
P vs. NP
   We know PNP, but the most famous
unproven conjecture in computer science is
that this inclusion is proper (i.e., that PNP
rather than P=NP).
   It is generally accepted that no NP-
complete problem can be solved in
polynomial time.
   Whoever first proves it will be famous!
Questions
   Find the best big-O notation to describe the
complexity of following algorithms:
   A binary search of n elements
   A linear search to find the smallest number in a
list of n numbers
   An algorithm that lists all ways to put the
numbers 1,2,3,…,n in a row.
Questions (cont’d)
   An algorithm that prints all bit strings of length n
   An iterative algorithm to compute n!
   An algorithm that finds the average of n
numbers by adding them and dividing by n
   An algorithm that prints all subsets of size three
of the set {1,2,3,…,n}
   The best case analysis of a linear search of a list
of size n.
Questions (cont’d)
   The wors-case analysis of a linear search of a list
of size n
   The number of print statements in the following
while n>1 {
print “hello”
n=n/2
}
Questions (cont’d)
   The number of print statements in the following
for (i=1, i≤n; i++)
for (j=1, j ≤n; j++)
print “hello”
   The number of print statements in the following
for (i=1, i≤n; i++)
for (j=1, j ≤i; j++)
print “hello”
Introduction to Number
Theory
Introduction to Number Theory

   Number theory is about integers and their properties.

    divisibility,
    greatest common divisors,
    least common multiples, and
    modular arithmetic

   and look at some relevant algorithms.
Division

If a and b are integers with a  0, we say that
a divides b if there is an integer c so that b = ac.

When a divides b we say that a is a factor of b and that b is
a multiple of a.

   The notation a | b means that a divides b.

We write a χ b when a does not divide b
(see book for correct symbol).
Divisibility Theorems

   For integers a, b, and c it is true that

    if a | b and a | c, then a | (b + c)
    Example: 3 | 6 and 3 | 9, so 3 | 15.

    if a | b, then a | bc for all integers c
    Example: 5 | 10, so 5 | 20, 5 | 30, 5 | 40, …

    if a | b and b | c, then a | c
    Example: 4 | 8 and 8 | 24, so 4 | 24.
Primes

A positive integer p greater than 1 is called prime if the only
positive factors of p are 1 and p.
Note: 1 is not a prime

A positive integer that is greater than 1 and is not prime is called
composite.
   The fundamental theorem of arithmetic:
Every positive integer can be written uniquely as the product of
primes, where the prime factors are written in order of increasing
size.
Primes

   Examples:

15 =       3·5
48 =       2·2·2·2·3 = 24·3
17 =       17
100 =      2·2·5·5 = 22·52
512 =      2·2·2·2·2·2·2·2·2 = 29
515 =      5·103
28 =       2·2·7
Primes

If n is a composite integer, then n has a prime divisor less
than or equal n .

This is easy to see: if n is a composite integer, it must
have at least two prime divisors. Let the largest two be p1
and p2. Then p1p2 <= n.

p and p2 cannot both be greater than
 1

 n , because then p1p2 > n.
The Division Algorithm

Let a be an integer and d a positive integer.
Then there are unique integers q and r, with

0  r < d, such that a = dq + r.

In the above equation,
 d is called the divisor,
 a is called the dividend,

 q is called the quotient, and
 r is called the remainder.
The Division Algorithm

   Example:

   When we divide 17 by 5, we have

   17 = 53 + 2.

    17 is the dividend,
    5 is the divisor,
    3 is called the quotient, and
    2 is called the remainder.
The Division Algorithm

   Another example:
   What happens when we divide -11 by 3 ?
   Note that the remainder cannot be negative.
   -11 = 3(-4) + 1.

    -11 is the dividend,
    3 is the divisor,
    -4 is called the quotient, and
    1 is called the remainder.
Greatest Common Divisors

Let a and b be integers, not both zero.
The largest integer d such that d | a and d | b is called the greatest
common divisor of a and b.
The greatest common divisor of a and b is denoted by gcd(a, b).

Example 1: What is gcd(48, 72) ?
The positive common divisors of 48 and 72 are
1, 2, 3, 4, 6, 8, 12, 16, and 24, so gcd(48, 72) = 24.
Example 2: What is gcd(19, 72) ?
The only positive common divisor of 19 and 72 is
1, so gcd(19, 72) = 1.
Greatest Common Divisors

   Using prime factorizations:
a = p1a1 p2a2 … pnan , b = p1b1 p2b2 … pnbn ,
where p1 < p2 < … < pn and ai, bi  N for 1  i  n

   gcd(a, b) = p1min(a1, b1 ) p2min(a2, b2 ) … pnmin(an, bn )

   Example:

a = 60 = 22 31 51
b = 54 = 21 33 50
gcd(a, b) =            21 31 50 = 6
Relatively Prime Integers

   Definition:
Two integers a and b are relatively prime if
gcd(a, b) = 1.

   Examples:
Are 15 and 28 relatively prime?
Yes, gcd(15, 28) = 1.

Are 55 and 28 relatively prime?

Yes, gcd(55, 28) = 1.

Are 35 and 28 relatively prime?

No, gcd(35, 28) = 7.
Relatively Prime Integers

   Definition:
The integers a1, a2, …, an are pairwise relatively prime if gcd(ai, aj)
= 1 whenever 1  i < j  n.

   Examples:
Are 15, 17, and 27 pairwise relatively prime?
No, because gcd(15, 27) = 3.

Are 15, 17, and 28 pairwise relatively prime?
Yes, because gcd(15, 17) = 1, gcd(15, 28) = 1 and gcd(17, 28) = 1.
Least Common Multiples

   Definition:
The least common multiple of the positive integers a and b is the
smallest positive integer that is divisible by both a and b.
   We denote the least common multiple of a and b by lcm(a, b).
   Examples:
lcm(3, 7) = 21
lcm(4, 6) = 12
lcm(5, 10) = 10
Least Common Multiples

   Using prime factorizations:
a = p1a1 p2a2 … pnan , b = p1b1 p2b2 … pnbn ,
where p1 < p2 < … < pn and ai, bi  N for 1  i  n

   lcm(a, b) = p1max(a1, b1 ) p2max(a2, b2 ) … pnmax(an, bn )

   Example:
a = 60 = 22 31 51
b = 54 = 21 33 50
lcm(a, b) =         22 33 51 = 4275 = 540
GCD and LCM

a = 60 =   22 31 51

b = 54 =   21 33 50

gcd(a, b) =      21 31 50   =6

lcm(a, b) =      22 33 51   = 540

Theorem: ab =       gcd(a,b)lcm(a,b)
Modular Arithmetic

Let a be an integer and m be a positive integer.
We denote by a mod m the remainder when a is divided by m.

   Examples:

9 mod 4 =       1
9 mod 3 =       0
9 mod 10 =      9
-13 mod 4 =         3
Congruences

Let a and b be integers and m be a positive integer. We say that a is
congruent to b modulo m if
m divides a – b.

We use the notation a  b (mod m) to indicate that a is congruent
to b modulo m.

In other words:
a  b (mod m) if and only if a mod m = b mod m.
Congruences

Examples:
Is it true that 46  68 (mod 11) ?
Yes, because 11 | (46 – 68).

Is it true that 46  68 (mod 22)?
Yes, because 22 | (46 – 68).

For which integers z is it true that z  12 (mod 10)?
It is true for any z{…,-28, -18, -8, 2, 12, 22, 32, …}

Theorem: Let m be a positive integer. The integers a and b are
congruent modulo m if and only if there is an integer k such that a = b
+ km.
Congruences

 Theorem: Let m be a positive integer.
If a  b (mod m) and c  d (mod m), then
a + c  b + d (mod m) and ac  bd (mod m).
Proof:
We know that a  b (mod m) and c  d (mod m) implies that there
are integers s and t with
b = a + sm and d = c + tm.
Therefore,
b + d = (a + sm) + (c + tm) = (a + c) + m(s + t) and
bd = (a + sm)(c + tm) = ac + m(at + cs + stm).

Hence, a + c  b + d (mod m) and ac  bd (mod m).
Congruences

Theorem: Let m be a positive integer. a  b (mod m) iff a mod m = b mod m.
Proof:
Let a = mq1 + r1, and b = mq2 + r2.
Only if part: a mod m = b mod m  r1 = r2, therefore
        a – b = m(q1 – q2), and a  b (mod m).
If part: a  b (mod m) implies
                   a – b = mq
   mq1 + r1 – (mq2 + r2) = mq
                 r1 – r2 = m(q – q1 + q2).
Since 0  r1, r2  m, 0  |r1 - r2|  m. The only multiple in that range is 0.
Therefore r1 = r2, and a mod m = b mod m.
The Euclidean Algorithm

 The Euclidean Algorithm finds the greatest common divisor of two
integers a and b.
For example, if we want to find gcd(287, 91), we divide 287 by 91:

287 = 913 + 14

We know that for integers a, b and c,
if a | b and a | c, then a | (b + c).
Therefore, any divisor (including their gcd) of 287 and 91 must also
be a divisor of 287 - 913 = 14.
Consequently, gcd(287, 91) = gcd(14, 91).
The Euclidean Algorithm

   In the next step, we divide 91 by 14:
   91 = 146 + 7
   This means that gcd(14, 91) = gcd(14, 7).

   So we divide 14 by 7:
   14 = 72 + 0
   We find that 7 | 14, and thus gcd(14, 7) = 7.

   Therefore, gcd(287, 91) = 7.
The Euclidean Algorithm

   In pseudocode, the algorithm can be implemented as follows:

 procedure gcd(a, b: positive integers)
x := a
y := b
while y  0
begin
       r := x mod y
       x := y
       y := r
end {x is gcd(a, b)}
Representations of Integers

Let b be a positive integer greater than 1.
Then if n is a positive integer, it can be expressed uniquely in the
form:

   n = akbk + ak-1bk-1 + … + a1b + a0,

where k is a nonnegative integer,
a0, a1, …, ak are nonnegative integers less than b,
and ak  0.

Example for b=10:
859 = 8102 + 5101 + 9100
Representations of Integers

   Example for b=2 (binary expansion):
   (10110)2 = 124 + 122 + 121 = (22)10

   Example for b=16 (hexadecimal expansion):
   (we use letters A to F to indicate numbers 10 to 15)
   (3A0F)16 = 3163 + 10162 + 15160 = (14863)10
Representations of Integers

How can we construct the base b expansion of an integer n?
First, divide n by b to obtain a quotient q0 and remainder a0, that is,

n = bq0 + a0, where 0  a0 < b.

The remainder a0 is the rightmost digit in the base b expansion of n.

Next, divide q0 by b to obtain:

q0 = bq1 + a1, where 0  a1 < b.

a1 is the second digit from the right in the base b expansion of n.
Continue this process until you obtain a quotient equal to zero.
Representations of Integers

Example:
What is the base 8 expansion of (12345)10 ?

First, divide 12345 by 8:
12345 = 81543 + 1

1543 = 8192 + 7
192 = 824 + 0

24 = 83 + 0

3 = 80 + 3

   The result is: (12345)10 = (30071)8.
Representations of Integers

 procedure base_b_expansion(n, b: positive integers)
q := n

k := 0

while q  0
begin

       ak := q mod b
       q := q/b
       k := k + 1
end

{the base b expansion of n is (ak-1 … a1a0)b }

   How do we (humans) add two integers?

1 11    carry
   Example:            7583
+    4932

12515
1 1              carry
Binary expansions:                 (1011)2
+ (1010)2

( 1 0 1 0 1 )2

Let a = (an-1an-2…a1a0)2, b = (bn-1bn-2…b1b0)2.
How can we algorithmically add these two binary numbers?

a0 + b0 = c02 + s0,

where s0 is the rightmost bit in the binary expansion of a + b, and
c0 is the carry.
Then, add the next pair of bits and the carry:
a1 + b1 + c0 = c12 + s1,

where s1 is the next bit in the binary expansion of a + b, and c1 is
the carry.

   Continue this process until you obtain cn-1.

   The leading bit of the sum is sn = cn-1.

   The result is:
   a + b = (snsn-1…s1s0)2

Example:
Add a = (1110)2 and b = (1011)2.

 0 a + b0 = 0 + 1 = 02 + 1, so that c0 = 0 and s0 = 1.
 1 a + b1 + c0 = 1 + 1 + 0 = 12 + 0, so c1 = 1 and s1 = 0.
 2 a + b2 + c1 = 1 + 0 + 1 = 12 + 0, so c2 = 1 and s2 = 0.
 3 a + b3 + c2 = 1 + 1 + 1 = 12 + 1, so c3 = 1 and s3 = 1.
 4 s = c3 = 1.

   Therefore, s = a + b = (11001)2.

 procedure add(a, b: positive integers)
c := 0

for j := 0 to n-1

begin

        d := (aj + bj + c)/2
        sj := aj + bj + c – 2d
        c := d
end

sn := c
{the binary expansion of the sum is (snsn-1…s1s0)2}
Applications of Number
Theory
GCD as linear combination
   If a and b are positive integers, then
there exist integers s and t such that
gcd(a,b) = sa + tb
Example
Express gcd(252, 198) = 18 as a linear combination of 252 and 198
18 = 54 – 1*36
252 = 1*198 + 54
36 = 198 – 3*54
198 = 3*54 + 36
18 = 54 – 198 + 3*54
54 = 1*36 + 18
= 4*54 – 1*198
36 = 2*18        Start from next-to-last division   54 = 252 – 1*198

18 = 4*252 – 5*198
Hashing Function
   How to assign memory location that the data
can be retrieved quickly
   h(k) = k mod m, where k is key and m is the
number of available memory locations

   Example
   m = 111, h(064212848) = ?
   h(064212848) = 14
   So this data will be placed at memory location =
14
Caesar‟s Encryption
   Invented by Julius Caesar to make a
secret message
Algorithm
1. Convert each alphabet in a message to its corresponding
number p: A = 0, B = 1,…, Z = 25, and p  25 only
2. Select key k as a positive integer, Caesar himself often use k = 3
3. To encrypt, applies this formula:
f(p) = (p+k) mod 26
4. Then convert each number back to alphabet and pack the
ciphertext with its key
5. To decrypt, do step 1 and 2 then applies this formula:
f-1(p) = (p-k) mod 26
6. If p+k or p-k is greater than or lesser than 25, shift it with 25
Example: Caesar‟s Encryption
   Encrypt “DISCRETE MATH”

DISCRETE             MATH

3 8 18 2 17 4 19 4   12 0 19 7

f(p+k) mod 26

6 11 21 5 20 7 22 7 15 3 22 10

GLVFUHWH             PDWK

GLVFUHWH PDWK
Linear Congruences
   Congruence of a form ax  b(mod m), where
m is positive integer, a and b are integers, x
is a variable is called Linear Congruence
   To solve this congruence is to find all
possible x that satisfies the formula
   Inverse modulo
   More details in Section 3.7
Chinese Reminder Theorem
   from San-Tsu‟s puzzle, ancient Chinese
mathematician
   can be solved using Modular arithmetic
   More details in Section 3.7
RSA
   Public key crypto system
   Based on modular exponential
   To encrypt
   C = Me mod n
   To decrypt
   Cd  M(mod pq)
   More details see RSA section and
Example 11 and 12 on page 241 - 244
Matrices
Matrices
   A matrix is a rectangular array of numbers.
   An mn (“m by n”) matrix has exactly m
horizontal rows, and n vertical columns.
   An nn matrix is called a square matrix,
whose order is n.
   The plural of matrix is matrices.

2 3 
5  1      a 32
     
7 0        matrix
     
Example

   Row: 3 rows
1 1        Column: 2 columns
0 2         Size: 3 x 2 matrix
    


1 3        Square: NO
    
Example

   Row: 2 rows
1 2        Column: 2 columns
3 4        Size: 2 x 2 matrix
   
   Square: YES
Example
0 1 
1 1      2 1         2 2 1
A       B          C     
2 0      1 2         0 1 0
    
   Rows and columns?
0 1           Size?
D   


2 0          Square?
   Equal?
Matrix Equality
   Two matrices A and B are equal iff they
have the same number of rows, the same
number of columns, and all corresponding
elements are equal.

 3 2  3 2 0
  1 6     1 6 0
                  
Row and Column Order
   The rows in a matrix are usually indexed 1
to m from top to bottom. The columns are
usually indexed 1 to n from left to right.
Elements are indexed by row, then column.
 a1,1 a1, 2      a1,n 
a      a2 , 2            
 a2 , n 
A  [ai , j ]  
2 ,1

                
                         
am,1 am, 2       am , n 
   Let                           The ith row of A is the 1 x n matrix
jth column

 a11 a12 ... a1n                   a i1   a i2 ... a in 
a    a 22 ... a 2n        The jth column of A is the n x 1 matrix
 21                   

 .     .        .                            a1j 
A                                              a 
ith row
.    .        . 
                                              2j 
 .     .        .                            . 
                                             
a n1 a n2 ... a nn 
                                             . 
 . 
 
a nj 
 
 a11 a12 ... a1n 
a    a 22 ... a 2n 
 21                
 .     .        . 
A                   
  .    .        . 
 .     .        . 
                   
a n1 a n2 ... a nn 
                   
   The (i,j)th element or entry of A is the element aij, that is, the number in
the ith row and jth column of A.
   A convenient shorthand notation for expressing the matrix A is to write
A = [aij], which indicates that A is the matrix with its (i,j)th element
equal to aij.
Matrix Arithmetic
   Multiplication/Product
Matrix Sums
   The sum A+B of two matrices A, B (which
must have the same number of rows, and
the same number of columns) is the matrix
(also with the same shape) given by adding
corresponding elements.
   A+B = [ai,j+bi,j]
2 6   9 3  11          9
0  8   11 3   11  5
                         
   Find A + B, where:
1 2         4 3
A          B   
3 4         2 1
   Find A + B, where:

1 2          0 1
3 4
A             2 3
B
              
5 6
            4 5
    
   Find A + B, where:

1 2
3 4        0 1 2
A         B      
5 6        3 4 5 
   
Matrix Products
   For an mk matrix A and a kn matrix B, the product AB is
the mn matrix:
k            
AB  C  [ci , j ]   ai ,b, j 
 1         
   i.e., element (i,j) of AB is given by the vector dot product
of the ith row of A and the jth column of B (considered as
vectors).
   In other words, if AB = [cij], then

cij = ai1b1j + ai2b2j + … + aikbkj.
   Note: Matrix multiplication is not commutative!
Matrix Product Example
   An example matrix multiplication to
practice in class:

0  1 1 0
0 1  1             1 0  5  1
2 0 3   2 0  2 0 3  2 11 3 
        1 0    3 1              
         
Example: Multiplication
   Find AB, where:

1 2            4 3
A             B   
3 4            2 1

   Find BA
Example: Multiplication
   Find AB, where:

1 2
3 4        0 1 2
A         B      
5 6        3 4 5 
   
Example: Multiplication
   Find AB, where:

0 1 2
A  2 1       B      
3 4 5 
Example: Multiplication
       Suppose
    A is a 6 × 8 matrix,
    B is an 8 × 5 matrix, and
    C is a 5 × 9 matrix.
Find the number of rows, the number of
columns, and the number of entries in
A(BC).
Example: Multiplication
   Find A2 where,

1 2
A   
3 4
Identity Matrices
   The identity matrix of order n, In, is the
order-n matrix with 1’s along the upper-left
to lower-right diagonal and 0’s everywhere
else.                   1 0  0
1 if i  j    0 1  0 
I n                
           
 0 if i  j         
        
0 0  1 
Identity matrix
   Multiplying a matrix by an appropriate
sized identity matrix does not change
this matrix. In other words, when A is
an m x n matrix, we have
AIn = ImA = A
Example: Identity matrix
Find identity matrix of each of   the
following matrices:
1     2 3
1 2           4 3         2        
A         B            C       1 2
3 4           2 1
2
      3 3

0 1 2
D      
3 4 5 
Matrix Inverses
   For some (but not all) square matrices A,
there exists a unique multiplicative inverse
A-1 of A, a matrix such that A-1A = In.
   If the inverse exists, it is unique, and
A-1A = AA-1.
   We won’t go into the algorithms for matrix
inversion...
Matrix Multiplication Algorithm
procedure matmul(matrices A: mk, B: kn)
for i := 1 to m               (m)· What’s the  of its
time complexity?
for j := 1 to n begin (n)·(
cij := 0
(mnk)
for q := 1 to k               (k) ·
cij := cij + aiqbqj      (1))

end {C=[cij] is the product of A and B}
Powers of Matrices
If A is an nn square matrix and p0, then:
 Ap  AAA···A    (A0  In)
3
p times    2 1  2         1  2 1  2 1
  1 0    1        1 0    1 0
                0                  
2      1  3      2
   Example:                            2  1
 1    0            
4     3

  3  2

Matrix Transposition
   If A=[aij] is an mn matrix, the transpose of
A (often written At or AT) is the nm matrix
given by At = B = [bij] = [aji] (1in,1jm)

t  2 0 
2 1     3           
 0  1  2   1  1 
             3  2
Flip
      
across
diagonal
Example: Transpose
Find transpose of   the following
matrices:
1    2
1 2           3           0 1 2
A          B       4    D      
3 4                         3 4 5 
5
     6

Symmetric Matrices
   A square matrix A is symmetric iff A=At. I.e.,
i,jn: aij = aji .
   Which is symmetric?

1 1  2 1 3  3 0 1 
1 1  1        
0  1 0 2  1 
                      
1 1  3  1 2  1 1  2
                     
Definition 7: Symmetric
   A square matrix A is called symmetric if A = At.
Thus A = [aij] is symmetric if aij = aji for all i
and j with 1 ≤ i ≤ n and 1 ≤ j ≤ n.
Example: Symmetric

 2 4    2 1 4   1 3
4 1    1 3 2     3 1
                    
4 2 0
          2 1
    
Zero-One Matrices
   All elements of a zero-one matrix are 0 or 1
   Representing False & True respectively.
   Useful for representing other structures.
   E.g., relations, directed graphs
   The meet of A, B (both mn zero-one matrices):
 AB : [aijbij] = [aij bij]

   The join of A, B:
 AB : [aijbij]
Boolean Products
   Let A=[aij] be an mk zero-one matrix,
& let B=[bij] be a kn zero-one matrix,
   The boolean product of A and B is like
normal matrix , but using  instead + in
the row-column “vector dot product.”

     
k
A⊙B  C  [cij ]   ai  bj 
  1      
Arithmetic/Boolean Products

    k

AB  C  [ci , j ]   ai ,b, j 
 1         


       
k
A⊙B  C  [cij ]   ai  bj 
  1      
Boolean Powers
   For a square zero-one matrix A, and any
k0, the kth Boolean power of A is simply
the Boolean product of k copies of A.
   A[k]  A⊙A⊙…⊙A
k times
Homework 3
   Chapter 3
   Section 3.1:
 1, 5, 22, 39, 42

   Section 3.2:
 1, 7, 9, 19, 21

   Section 3.3:
 1, 3, 9, 11, 25, 27

   Section 3.4:
 1, 9, 17, 31, 32

   Section 3.5:
 1, 21, 25

   Section 3.6:
 1, 5, 19, 23

   Section 3.7:
 7, 21

   Section 3.8:
 1, 3, 5, 31, 33

   Supplementary:
   none

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 52 posted: 8/3/2011 language: English pages: 192