Discrete Mathematics Lecture Algorithms The Integers

Document Sample
Discrete Mathematics Lecture Algorithms The Integers Powered By Docstoc
					         461191 Discrete Mathematics

         Lecture 3: Algorithms, The
         Integers, and Matrices

                              San Ratanasanya
                                CS, KMUTNB

Adapted from several sources:
Michael P. Frank, University of Florida, University of Nevada, and University of Maryland at
Baltimore
Today‟s Topics
   Review
   Administrivia
   Algorithms
   Algorithmic and Problem Complexity
   Introduction to Number Theory
   Applications of Number Theory
   Matrices
…from last week…
   Sets and Set operations
       Set is an unordered collection of objects
       Operations on Set: union, intersection, difference, complement,
        symmetric difference
       Venn Diagram and equivalence
   Functions
       Domain, Codomain, Range
       One-to-one, onto, bijection, inverse, and composite functions
   Sequences
       Represents ordered list of elements
            a1,a,2a3,…,an or {an}

       It is convenient to describe Sequence using formula
            an = 2an-1


   Summations
Example: Set
   Prove that A  A = U is true

       A = {x | x  A}
       A = {x | x  A}
       A  A = {x | x  A  x  A}
       A  A = {x | x  U}
       AA=U
         Example: Function
   Let f(x) = x2/3. Find f(S) if S = {x | xZ  |x|  3}
         f(3) = 32/3 = 3, f(2) = 22/3 = 1, f(1) = 12/3 = 0, f(0) = 0
          f(S) = {0, 1, 3}

   Suppose that g is a function from A to B and f is a function form B
    to C. Show that if both f and g are one-to-one functions, then fg
    is also one-to-one.
         We need to show that x A  y A (x  y  f(g(x))  f(g(y))).
         Or we need to show that x A  y A (x  y  g(x)  g(y))
          x B  y B (x  y  f(x)  f(y)).
         Given that g is one-to-one, therefore g(x)  g(y) in B.
         Given that f is one-to-one, therefore f(x)  f(y) in C.
         We can now conclude that f(g(x))  f(g(y)).
       Example: Sequence
   Find a simple formula for an if the first 10
    terms of the sequence {an} are 1, 7, 25, 79,
    241, 727, 2185, 6559, 19681, 59047

    a1 = 1
    an = 3n - 2
          Example: Summations
                                      n
 3    3                          a a          i 1   ?
i  ?
                                            i
                                     i 1
                                                        a1  a0  a2  a1 
i 1 j  0
                3        3                             a3  a2    an 1  an  2
               i 1
               i 1     j 0
                                                        an  an 1
                3                                       a n  a0
               4i
               i 1
                                 n

              4 i
                    3
                                 (2k  1)  ?
                                k 1
                 i 1                                   n           n       n

                  3 * (3  1)                       2 k   1  2 k  n
              4*                                      k 1        k 1    k 1
                       2                                      (n)(n  1)
              24                                  2*                    n  n2  n  n
                                                                  2
                                                   n2
Administrivia
   Homeworks due today
       Homework 1.1 – 1.4
       Homework 2
   Programs used in this class cannot be downloaded
    from class webpage
       they are TOO BIG…
   There are only links to download those programs on
    class webpage
       Prolog, ISETLW
       Python, Scheme
   Information on what to download is also on the
    webpage
    Algorithms
   The foundation of computer programming.
   Most generally, an algorithm just means a definite
    procedure for performing some sort of task.
   A computer program is simply a description of an
    algorithm, in a language precise enough for a computer to
    understand, requiring only operations that the computer
    already knows how to do.
   We say that a program implements (or “is an
    implementation of”) its algorithm.
Algorithms You Already Know
   Grade-school arithmetic algorithms:
       How to add any two natural numbers written
        in decimal on paper, using carries.
       Similar: Subtraction using borrowing.
       Multiplication & long division.
   Your favorite cooking recipe.
   How to register for classes at KMUTNB.
    Programming Languages
   Some common programming languages:
       Newer: Java, C, C++, C#, Visual Basic,
        JavaScript, Perl, Tcl, Pascal, many others…
       Older: Fortran, Cobol, Lisp, Basic
       Assembly languages, for low-level coding.
   In this class we will use an informal, Pascal-
    like “pseudo-code” language.
   You should know at least 1 real language!
    Algorithm Example (English)
   Task: Given a sequence {ai}=a1,…,an, aiN,
    say what its largest element is.
   One algorithm for doing this, in English:
       Set the value of a temporary variable v (largest
        element seen so far) to a1’s value.
       Look at the next element ai in the sequence.
       If ai>v, then re-assign v to the number ai.
       Repeat then previous 2 steps until there are no
        more elements in the sequence, & return v.
    Executing an Algorithm
   When you start up a piece of software, we
    say the program or its algorithm are being
    run or executed by the computer.
   Given a description of an algorithm, you
    can also execute it by hand, by working
    through all of its steps with pencil & paper.
   Before ~1940, “computer” meant a person
    whose job was to execute algorithms!
    Executing the Max algorithm
   Let {ai}=7,12,3,15,8. Find its maximum…
   Set v = a1 = 7.
   Look at next element: a2 = 12.
   Is a2>v? Yes, so change v to 12.
   Look at next element: a2 = 3.
   Is 3>12? No, leave v alone….
   Is 15>12? Yes, v=15…
   Algorithm Characteristics
Some important general features of algorithms:
 Input. Information or data that comes in.

 Output. Information or data that goes out.

 Definiteness. Algorithm is precisely defined.

 Correctness. Outputs correctly relate to inputs.

 Finiteness. Won’t take forever to describe or run.

 Effectiveness. Individual steps are all do-able.

 Generality. Works for many possible inputs.

 Efficiency. Takes little time & memory to run.
  Our Pseudocode Language
procedure                 for variable := initial
   name(argument: type)     value to final value
variable := expression         statement
informal statement        while condition
begin statements end        statement
{comment}                 procname(arguments)
if condition then           Not defined in book:
   statement [else        return expression
   statement]
What is Pseudocode
   Pseudocode is an intermediate step between
    language description, normally English, of the
    steps of a procedure and a specification of this
    procedure using an actual programming language.
                                  ่   ั
    เป็ นการอธิบายโปรแกรมด้วยภาษาทีใช้กนทั่วไป ด้วยลักษณะที่
                ่
    คล้ายภาษาทีใช้ในการเขียนโปรแกรม



          Human                         Programming
                        Pseudocode
        Languages                        Languages
What is Program
   Program = Algorithm + Data Structure
   Each step of a program may make changes to variables
   Trace = a sequence of changes in variables as each step of
    program is executed




   Trace is a simple model of program executions
       State = variable
procedure procname(arg: type)
   Declares that the following text defines a
    procedure named procname that takes
    inputs (arguments) named arg which are
    data objects of the type type.
       Example:
        procedure maximum(L: list of integers)
         [statements defining maximum…]
variable := expression
   An assignment statement evaluates the
    expression expression, then reassigns the
    variable variable to the value that results.
       Example assignment statement:
        v := 3x+7    (If x is 2, changes v to 13.)
   In pseudocode (but not real code), the
    expression might be informally stated:
       x := the largest integer in the list L
    Informal statement
   Sometimes we may write a statement as an
    informal English imperative, if the meaning
    is still clear and precise: e.g.,“swap x and y”
   Keep in mind that real programming
    languages never allow this.
   When we ask for an algorithm to do so-
    and-so, writing “Do so-and-so” isn’t
    enough!
       Break down algorithm into detailed steps.
       begin statements end
      Groups a sequence of           Allows the sequence
       statements together:            to be used just like a
       begin                           single statement.
         statement 1                  Might be used:
         statement 2                      After a procedure
         …                                 declaration.
         statement n                      In an if statement after
       end                                 then or else.
                                          In the body of a for or
Curly braces {} are used instead
                                           while loop.
in many languages.
{comment}
   Not executed (does nothing).
   Natural-language text explaining some
    aspect of the procedure to human readers.
   Also called a remark in some real
    programming languages, e.g. BASIC.
   Example, might appear in a max program:
       {Note that v is the largest integer seen so far.}
       Conditional Constructions
     if condition then statement         if condition then statement 1
                                             else statement 2
if condition then
    begin
      Block of                    if condition 1 then statement 1
   statements                         else if condition 2 then statement 2
   end     if condition then          else if condition 2 then statement 2
           begin                     ………………………………………………………
             statement 1             else if condition n then statement n
           end                       else statement n+1
               else
               begin
                    statement 2
               end
    if condition then statement
   Evaluate the propositional expression
    condition.
       If the resulting truth value is True, then
        execute the statement statement;
       otherwise, just skip on ahead to the next
        statement after the if statement.
   Variant: if cond then stmt1 else stmt2
       Like before, but iff truth value is False,
        executes stmt2.
          Loop Constructions
for variable := initial value to final value
   statement
                                    for all element with certain property
for variable := initial value to
final value
                                 while condition      while condition
begin
                                   statement          begin
      Block of statements
    end                                                 Block of statements 1
                                                       end
                                               sum := 0
       sum := 0                                while n > 0
       for i := 1 to n                         begin
          sum := sum + i                         sum := sum + n
                                                n := n - 1
                                               end
    while condition statement
   Evaluate the propositional (Boolean)
    expression condition.
   If the resulting value is True, then execute
    statement.
   Continue repeating the above two actions
    over and over until finally the condition
    evaluates to False; then proceed to the
    next statement.
    while condition statement
   Also equivalent to infinite nested ifs, like so:
    if condition
        begin
          statement
          if condition
             begin
               statement
               …(continue infinite nested if’s)
             end
        end
    for var := initial to final stmt
   Initial is an integer expression.
   Final is another integer expression.
   Semantics: Repeatedly execute stmt, first with
    variable var := initial, then with var := initial+1,
    then with var := initial+2, etc., then finally with
    var := final.
   Question: What happens if stmt changes the
    value of var, or the value that initial or final
    evaluates to?
    for var := initial to final stmt
   For can be exactly defined in terms of
    while, like so:
                    begin
                       var := initial
                       while var  final
                         begin
                             stmt
                             var := var + 1
                         end
                    end
procedure(argument)
   A procedure call statement invokes the
    named procedure, giving it as its input the
    value of the argument expression.
   Various real programming languages refer
    to procedures as functions (since the
    procedure call notation works similarly to
    function application f(x)), or as subroutines,
    subprograms, or methods.
Nested
   Block of statements
   Conditional
   Loop
   Procedure – a call to procedure within procedure
           Procedure statistic (L:list of
           integers)
           begin
             max(L)
             min(L)
             average(L)
             ………..
           end
Max procedure in pseudocode
procedure max(a1, a2, …, an: integers)
  v := a1 {largest element so far}
  for i := 2 to n {go thru rest of elems}
     if ai > v then v := ai {found bigger?}
  {at this point v’s value is the same as
    the largest integer in the list}
  return v
Inventing an Algorithm
   Requires a lot of creativity and intuition
       Like writing proofs.
   Unfortunately, we can’t give you an
    algorithm for inventing algorithms.
       Just look at lots of examples…
       And practice (preferably, on a computer)
       And look at more examples…
       And practice some more… etc., etc.
Algorithm-Inventing Example
   Suppose we ask you to write an algorithm
    to compute the predicate:
               IsPrime:N→{T,F}
       Computes whether a given natural number is a
        prime number.
   First, start with a correct predicate-logic
    definition of the desired function:
    n: IsPrime(n)  ¬1<d<n: d|n
                                 Means d divides n
                                 evenly (without remainder)
IsPrime example, cont.
   Notice that the negated exponential can be
    rewritten as a universal:            Means d does not
                                                 divide n evenly
    ¬1<d<n: d|n      1<d<n: d | n             (the remainder is ≠0)
                      2≤ d ≤ n−1: d | n
   This universal can then be translated directly into
    a corresponding for loop:
    for d := 2 to n−1 { Try all potential divisors >1 & <n }
      if d|n then return F { n has divisor d; not prime }
    return T { no divisors were found; n must be prime}
Optimizing IsPrime
   The IsPrime algorithm can be further optimized:
    for d := 2 to n1/2       Note smaller range of search.
      if d|n then return F
    return T
   This works because of this theorem: If n has any
    (integer) divisors, it must have one less than n1/2.
    Proof: Suppose n’s smallest divisor >1 is a, and let
    b :≡ n/a. Then n = ab, but if a > n1/2 then b > n1/2 (since a
    is n’s smallest divisor) and so n = ab > (n1/2)2 = n, an
    absurdity. Further optimizations are possible:
                  - E.g., only try divisors that
                    are primes less than n1/2.
    Another example task
   Problem of searching an ordered list.
       Given a list L of n elements that are sorted into
        a definite order (e.g., numeric, alphabetical),
       And given a particular element x,
       Determine whether x appears in the list,
       and if so, return its index (position) in the list.
   Problem occurs often in many contexts.
   Let’s find an efficient algorithm!
Search alg. #1: Linear Search
procedure linear search
  (x: integer, a1, a2, …, an: distinct integers)
  i := 1 {start at beginning of list}
  while (i  n  x  ai) {not done, not found}
       i := i + 1 {go to the next position}
  if i  n then location := i {it was found}
  else location := 0 {it wasn’t found}
  return location {index or 0 if not found}
Search alg. #2: Binary Search
   Basic idea: On each step, look at the middle
    element of the remaining list to eliminate
    half of it, and quickly zero in on the desired
    element.



                      <x   <x <x >x
   Search alg. #2: Binary Search
procedure binary search
  (x:integer, a1, a2, …, an: distinct integers)
  i := 1 {left endpoint of search interval}
  j := n {right endpoint of search interval}
  while i<j begin {while interval has >1 item}
       m := (i+j)/2 {midpoint}
       if x>am then i := m+1 else j := m
  end
  if x = ai then location := i else location := 0
  return location
binary search for the letter „j‟
               search interval
     a c d f g h j l m o p r s u v x z

               center element

   search interval

   a c d f g h j l m o p r s u v x z

     center element
binary search for the letter „j‟
         search interval

    a c d f g h j l m o p r s u v x z

           center element

       search interval
    a c d f g h j l m o p r s u v x z
         center element
binary search for the letter „j‟

       search interval


    a c d f g h j l m o p r s u v x z


         center element
               found !
Practice exercises
   Devise an algorithm that finds the sum of
    all the integers in a list. [2 min]
   procedure sum(a1, a2, …, an: integers)
         s := 0 {sum of elems so far}
         for i := 1 to n {go thru all elems}
                s := s + ai {add current item}
         {at this point s is the sum of all items}
         return s
   Sorting Algorithms
      Sorting is a common operation in many
       applications.
          E.g. spreadsheets and databases
      It is also widely used as a subroutine in
       other data-processing algorithms.
      Two sorting algorithms shown in textbook:
          Bubble sort       However, these are not
                             very efficient, and you should
          Insertion sort    not use them on large data sets!
We’ll see some more efficient algorithms later in the course.
    Bubble Sort
   Smallest elements “float” up to the top of
    the list, like bubbles in a container of liquid.
     30     30       1      1      1      1
     31      1      30     30      2      2
      1     31      31      2     30      3
     32     32       2     31      3     30
     33      2      32      3     31     31
      2     33       3     32     32     32
     34      3      33     33     33     33
      3     34      34     34     34     34
Insertion Sort Algorithm
   English description of algorithm:
       For each item in the input list,
            “Insert” it into the correct place in the sorted
             output list generated so far. Like so:
                 Use linear or binary search to find the location where the
                  new item should be inserted.
                 Then, shift the items from that position onwards down by
                  one position.
                 Put the new item in the hole remaining.
         Pseudocodes
Procedure insertionsort(a1,…,an: real numbers with n  2)
for j := 2 to n
begin
                             Procedure bubblesort(a1,…,an: real numbers with n  2)
    i := 1
                             for i := 1 to n-1
  while aj > ai
                               for j := 1 to n-1
           i := i +1
                                 if aj > aj+1 then interchange aj and aj+1
   m : = aj
                             {a1,…,an is in increasing order}
    for k := 0 to j – i – 1
           aj-k := aj-k-1
    ai := am
end {a1, a2,…,an are sorted}
Greedy Algorithms
    Any algorithms selecting the „best‟
     choice at each step are called Greedy
    Can solve optimization problem
    Procedure change(c1, c2,…,cr: values of denomination of coins,
                        where c1 > c2 >… > cr; n: a positive integer)
    for i := 1 to r
      while n  ci
      begin
        add a coin with value ci to the change
        n := n - ci
      end
    Analysis of Algorithms
   What is the goal of analysis of algorithms?
       To compare algorithms mainly in terms of
        running time but also in terms of other factors
        (e.g., memory requirements, programmer's
        effort etc.)
   What do we mean by running time
    analysis?
       Determine how running time increases as the
        size of the problem increases.
Is Binary Search more efficient?
    Number of iterations:
        For a list of n elements, Binary Search can
         execute at most log2 n times!!
        Linear Search, on the other hand, can execute up
         to n times !!

               Average Number of Iterations
          Length     Linear Search    Binary Search
              10               5.5              2.9
             100              50.5              5.8
           1,000             500.5              9.0
          10,000            5000.5             12.0
Is Binary Search more efficient?
   Number of computations per iteration:
       Binary search does more computations than
        Linear Search per iteration.
   Overall:
       If the number of components is small (say, less
        than 20), then Linear Search is faster.
       If the number of components is large, then
        Binary Search is faster.
     How do we analyze algorithms?

   We need to define a number of objective measures.

    (1) Compare execution times?
       Not good: times are specific to a particular computer !!

    (2) Count the number of statements
    executed?
       Not good: number of statements vary with the
       programming language as well as the style of the
       individual programmer.
   Example (# of statements)
Algorithm 1       Algorithm 2

arr[0] = 0;     for(i=0; i<N; i++)
arr[1] = 0;      arr[i] = 0;
arr[2] = 0;
...
arr[N-1] = 0;
How do we analyze algorithms?

(3) Express running time as a function of
    the input size n (i.e., f(n)).
   To compare two algorithms with running times
    f(n) and g(n), we need a rough measure of
    how fast a function grows.
   Such an analysis is independent of machine
    time, programming style, etc.
    Computing running time

   Associate a "cost" with each statement and find
    the "total cost“ by finding the total number of
    times each statement is executed.
   Express running time in terms of the size of the
    problem.
   Computing running time (contd.)

 Algorithm 1                Algorithm 2
               Cost                          Cost
arr[0] = 0;     c1        for(i=0; i<N; i++)    c2
arr[1] = 0;     c1          arr[i] = 0;        c1
arr[2] = 0;     c1
...
arr[N-1] = 0; c1
            -----------                  -------------
 c1+c1+...+c1 = c1 x N        (N+1) x c2 + N x c1 =
                               (c2 + c1) x N + c2
  Computing running time (cont.)
                              Cost
  sum = 0;                    c1
  for(i=0; i<N; i++)           c2
    for(j=0; j<N; j++)         c2
        sum += arr[i][j];        c3
                                 ------------
c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N x N
Comparing Functions Using
Rate of Growth
  Consider the example of buying elephants and
   goldfish:
        Cost: cost_of_elephants + cost_of_goldfish
        Cost ~ cost_of_elephants (approximation)
 The low order terms in a function are relatively
   insignificant for large n
               n4 + 100n2 + 10n + 50 ~ n4
i.e., n4 + 100n2 + 10n + 50 and n4 have the same
   rate of growth
    Rate of Growth ≡Asymptotic Analysis

   Using rate of growth as a measure to
    compare different functions implies
    comparing them asymptotically.
   If f(x) is faster growing than g(x), then f(x)
    always eventually becomes larger than g(x)
    in the limit (for large enough values of x).
    Example
   Suppose you are designing a web site to process
    user data (e.g., financial records).
   Suppose program A takes fA(n)=30n+8
    microseconds to process any n records, while
    program B takes fB(n)=n2+1 microseconds to
    process the n records.
   Which program would you choose, knowing you’ll
    want to support millions of users?
Visualizing Orders of Growth
   On a graph, as
    you go to the

                  Value of function 
                                                     fA(n)=30n+8
    right, a faster
    growing
    function
                                            fB(n)=n2+1
    eventually
    becomes
    larger...                           Increasing n 
    Big-O Notation
   We say fA(n)=30n+8 is order n, or O(n).
    It is, at most, roughly proportional to n.
   fB(n)=n2+1 is order n2, or O(n2). It is, at most,
    roughly proportional to n2.
   In general, an O(n2) algorithm will be
    slower than O(n) algorithm.
   Warning: an O(n2) function will grow faster
    than an O(n) function.
     More Examples …

   We say that n4 + 100n2 + 10n + 50 is of the
    order of n4 or O(n4)
   We say that 10n3 + 2n2 is O(n3)
   We say that n3 - n2 is O(n3)
   We say that 10 is O(1),
   We say that 1273 is O(1)
Big-O Visualization
Computing running time
 Algorithm 1                 Algorithm 2
               Cost                          Cost
arr[0] = 0;     c1        for(i=0; i<N; i++)    c2
arr[1] = 0;     c1          arr[i] = 0;        c1
arr[2] = 0;     c1
...
arr[N-1] = 0; c1
            -----------                  -------------
 c1+c1+...+c1 = c1 x N        (N+1) x c2 + N x c1 =
                               (c2 + c1) x N + c2


                      O(n)
 Computing running time (cont.)
                                    Cost
sum = 0;                        c1
for(i=0; i<N; i++)               c2
  for(j=0; j<N; j++)             c2
 sum += arr[i][j];             c3
                               ------------
c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N x N
                       O(n2)
Running time of various statements
  while-loop   for-loop
    Examples

i = 0;
while (i<N) {
   X=X+Y;             // O(1)
   result = mystery(X); // O(N), just an example...
   i++;          // O(1)
}
   The body of the while loop: O(N)
   Loop is executed:           N times
                             N x O(N) = O(N2)
Examples (cont.’d)
   if (i<j)
      for ( i=0; i<N; i++ )     O(N)

           X = X+i;
   else               O(1)
      X=0;

   Max ( O(N), O(1) ) = O (N)
       The Growth of Functions

   Example:
   Show that f(x) = x2 + 2x + 1 is O(x2).

For x > 1 we have:
x + 2x + 1  x + 2x + x
  2            2     2   2

 x + 2x + 1  4x
     2             2


Therefore, for C = 4 and k = 1:
f(x)  Cx2 whenever x > k.


    f(x) is O(x2).
         The Growth of Functions


   Question: If f(x) is O(x2), is it also O(x3)?

   Yes. x3 grows faster than x2, so x3 grows also faster than f(x).

Therefore, we always have to find the smallest simple function
g(x) for which f(x) is O(g(x)).
The Growth of Functions


 “Popular” functions g(n) are
 n log n, 1, 2n, n2, n!, n, n3, log n


 Listed from slowest to fastest growth:
  1
  log n
  n
  n log n
  n
     2

  n
     3

  2n
  n!
         Complexity Examples

What does the following algorithm compute?
procedure who_knows(a1, a2, …, an: integers)
m := 0
for i := 1 to n-1
         for j := i + 1 to n
                   if |ai – aj| > m then m := |ai – aj|
{m is the maximum difference between any two numbers in the input
sequence}
Comparisons: n-1 + n-2 + n-3 + … + 1

                   = (n – 1)n/2 = 0.5n2 – 0.5n

   Time complexity is O(n2).
     Complexity Examples

   Another algorithm solving the same problem:
procedure max_diff(a1, a2, …, an: integers)
min := a1
max := a1
for i := 2 to n
         if ai < min then min := ai
         else if ai > max then max := ai
m := max - min
Comparisons: 2n - 2


   Time complexity is O(n).
Asymptotic Notation
   O notation: asymptotic “less than”:

       f(n)=O(g(n)) implies: f(n) “≤” g(n)

    notation: asymptotic “greater than”:

       f(n)=  (g(n)) implies: f(n) “≥” g(n)

    notation: asymptotic “equality”:

       f(n)=  (g(n)) implies: f(n) “=” g(n)
    Definition: O(g), at most order g
Let f,g are functions RR.
 We say that “f is at most order g”, if:

          c,k: f(x)  cg(x), x>k
       “Beyond some point k, function f is at most a
        constant c times g (i.e., proportional to g).”
   “f is at most order g”, or “f is O(g)”, or
    “f=O(g)” all just mean that fO(g).
   Sometimes the phrase “at most” is omitted.
Big-O Visualization




      k
     Points about the definition
   Note that f is O(g) as long as any values of c
    and k exist that satisfy the definition.
   But: The particular c, k, values that make
    the statement true are not unique: Any
    larger value of c and/or k will also work.
   You are not required to find the smallest c
    and k values that work. (Indeed, in some
    cases, there may be no smallest values!)
    However, you should prove that the values you choose do work.
“Big-O” Proof Examples
   Show that 30n+8 is O(n).
       Show c,k: 30n+8  cn, n>k .
            Let c=31, k=8. Assume n>k=8. Then
             cn = 31n = 30n + n > 30n+8, so 30n+8 < cn.
   Show that n2+1 is O(n2).
       Show c,k: n2+1  cn2, n>k: .
          Let c=2, k=1. Assume n>1. Then
         cn2 = 2n2 = n2+n2 > n2+1, or n2+1< cn2.
Big-O example, graphically
   Note 30n+8 isn’t                          cn =
    less than n                               31n    30n+8




                      Value of function 
    anywhere (n>0).
   It isn’t even
    less than 31n                                            30n+8
    everywhere.
                                                         n
                                                             O(n)
   But it is less than                      n>k=8 
    31n everywhere to
                                            Increasing n 
    the right of n=8.
Common orders of magnitude
    Order-of-Growth in Expressions
 “O(f)” can be used as a term in an arithmetic
  expression .
E.g.: we can write “x2+x+1” as “x2+O(x)” meaning
  “x2 plus some function that is O(x)”.
 Formally, you can think of any such expression as
  denoting a set of functions:
     “x2+O(x)” : {g | fO(x): g(x)= x2+f(x)}
Useful Facts about Big O
   Constants ...
      c>0, O(cf)=O(f+c)=O(fc)=O(f)
   Sums:
      - If gO(f) and hO(f), then g+hO(f).
      - If gO(f1) and hO(f2), then
               g+hO(f1+f2) =O(max(f1,f2))
                    (Very useful!)
More Big-O facts
   Products:
     If gO(f1) and hO(f2), then ghO(f1f2)

   Big O, as a relation, is transitive:
         fO(g)  gO(h)  fO(h)
    More Big O facts
    f,g & constants a,bR, with b0,
       af = O(f)          (e.g. 3x2 = O(x2))
       f+O(f) = O(f)     (e.g. x2+x = O(x2))
       |f|1-b = O(f)        (e.g. x1 = O(x))
       (logb |f|)a = O(f) (e.g. log x = O(x))
       g=O(fg)             (e.g. x = O(x log x))
       fg  O(g)           (e.g. x log x  O(x))
       a=O(f)             (e.g. 3 = O(x))
    Definition: (g), at least order g
Let f,g be any function RR.
 We say that “f is at least order g”, written (g), if

             c,k: f(x)  cg(x), x>k
       “Beyond some point k, function f is at least a constant c
        times g (i.e., proportional to g).”
       Often, one deals only with positive functions and can
        ignore absolute value symbols.
   “f is at least order g”, or “f is (g)”, or “f= (g)” all
    just mean that f (g).
Big-  Visualization
    Definition: (g), exactly order g
  If fO(g) and gO(f) then we say “g and f are
   of the same order” or “f is (exactly) order g”
   and write f(g).
 Another equivalent definition:
      c1c2,k: c1g(x)f(x)c2g(x), x>k
 “Everywhere beyond some point k, f(x) lies in
   between two multiples of g(x).”
 (g)  O(g)  (g)

  (i.e., fO(g) and f(g) )
Big-  Visualization
    Rules for 
   Mostly like rules for O( ), except:
    f,g>0 & constants a,bR, with b>0,
     af  (f)          Same as with O.
     f  (fg) unless g=(1)  Unlike O.
    |f| 1-b  (f), and       Unlike with O.
    (logb |f|)c  (f).       Unlike with O.
   The functions in the latter two cases we say
    are strictly of lower order than (f).
 example
                        n ?
   Determine whether:   i (n 2 )
   Quick solution:     i 1 
     n 
      i   n(n  1) / 2
     i 1 
              n  ( n ) / 2
              n  ( n )
             ( n 2 )
    Other Order-of-Growth Relations
   o(g) = {f | c k: f(x) < cg(x), x>k}
    “The functions that are strictly lower order
    than g.” o(g)  O(g)  (g).

   (g) = {f | c k: cg(x) < f(x), x>k }
    “The functions that are strictly higher order
    than g.” (g)  (g)  (g).
Relations Between the Relations
   Subset relations between order-of-growth
    sets.
                   RR
            O( f )       ( f )
                    •f
        o( f )    ( f )   ( f )
Strict Ordering of Functions
   Temporarily let’s write fg to mean fo(g),
                            f~g to mean f(g)
   Note that               f ( x)
            f  g  lim              0.
                       x  g ( x )
   Let k>1. Then the following are true:
    1  log log n  log n ~ logk n  logk n
      n1/k  n  n log n  nk  kn  n!  nn …
Common orders of magnitude
     Review: Orders of Growth
Definitions of order-of-growth sets,
 g:RR
   O(g)  {f |  c,k: f(x)  cg(x), x>k }
   o(g)  {f | c k: f(x) < cg(x),x>k }
   (g)  {f| | c,k : f(x)  cg(x),x>k }
   (g)  {f |  c k: f(x) >cg(x), x>k}
   (g)  {f | c1c2,k: c1g(x)f(x)|c2g(x), x>k}
Examples
    Algorithmic Complexity
   The algorithmic complexity of a
    computation is some measure of how
    difficult it is to perform the computation.
   Measures some aspect of cost of
    computation (in a general sense of cost).
    Time complexity
   Best-case time = minimum time needed to
    execute the algorithm for inputs of size n
   Worst-case time = maximum time needed
    to execute the algorithm for inputs of size n
   Average-case time = average time needed

   focus on best and worst time
Problem Complexity
   The complexity of a computational problem
    or task is the complexity of the algorithm
    with the lowest order of growth of
    complexity for solving that problem or
    performing that task.
   E.g. the problem of searching an ordered
    list has at most logarithmic time complexity.
    (Complexity is O(log n).)
Tractable vs. Intractable Problems
   A problem or algorithm with at most polynomial
    time complexity is considered tractable (or
    feasible). P is the set of all tractable problems.
   A problem or algorithm that has more than
    polynomial complexity is considered intractable
    (or infeasible).
   Note
       n1,000,000 is technically tractable, but really impossible.
        nlog log log n is technically intractable, but easy.
       Such cases are rare though.
Dealing with Intractable Problems
   Many times, a problem is intractable for a small
    number of input cases that do not arise in
    practice very often.
       Average running time is a better measure of problem
        complexity in this case.
       Find approximate solutions instead of exact solutions.
Unsolvable problems
   It can be shown that there exist problems that no
    algorithm exists for solving them.
   Turing discovered in the 1930’s that there are
    problems unsolvable by any algorithm.
   Example: the halting problem (see page 176)
       Given an arbitrary algorithm and its input, will that
        algorithm eventually halt, or will it continue forever in
        an “infinite loop?”
     NP and NP-complete
   NP is the set of problems for which there exists a
    tractable algorithm for checking solutions to see if
    they are correct.
   NP-complete is a class of problems with the
    property that if any one of them can be solved by
    a polynomial worst-case algorithm, then all of
    them can be solved by polynomial worst-case
    algorithms.
       Satisfiability problem: find an assignment of truth
        values that makes a compound proposition true.
    P vs. NP
   We know PNP, but the most famous
    unproven conjecture in computer science is
    that this inclusion is proper (i.e., that PNP
    rather than P=NP).
   It is generally accepted that no NP-
    complete problem can be solved in
    polynomial time.
   Whoever first proves it will be famous!
Questions
   Find the best big-O notation to describe the
    complexity of following algorithms:
       A binary search of n elements
       A linear search to find the smallest number in a
        list of n numbers
       An algorithm that lists all ways to put the
        numbers 1,2,3,…,n in a row.
Questions (cont’d)
   An algorithm that prints all bit strings of length n
   An iterative algorithm to compute n!
   An algorithm that finds the average of n
    numbers by adding them and dividing by n
   An algorithm that prints all subsets of size three
    of the set {1,2,3,…,n}
   The best case analysis of a linear search of a list
    of size n.
Questions (cont’d)
   The wors-case analysis of a linear search of a list
    of size n
   The number of print statements in the following
     while n>1 {
       print “hello”
       n=n/2
     }
Questions (cont’d)
   The number of print statements in the following
    for (i=1, i≤n; i++)
       for (j=1, j ≤n; j++)
         print “hello”
   The number of print statements in the following
    for (i=1, i≤n; i++)
       for (j=1, j ≤i; j++)
         print “hello”
Introduction to Number
Theory
    Introduction to Number Theory

   Number theory is about integers and their properties.

   We will start with the basic principles of
    divisibility,
    greatest common divisors,
    least common multiples, and
    modular arithmetic

   and look at some relevant algorithms.
       Division

If a and b are integers with a  0, we say that
a divides b if there is an integer c so that b = ac.

When a divides b we say that a is a factor of b and that b is
a multiple of a.

   The notation a | b means that a divides b.

We write a χ b when a does not divide b
(see book for correct symbol).
          Divisibility Theorems


   For integers a, b, and c it is true that

    if a | b and a | c, then a | (b + c)
    Example: 3 | 6 and 3 | 9, so 3 | 15.

    if a | b, then a | bc for all integers c
    Example: 5 | 10, so 5 | 20, 5 | 30, 5 | 40, …

    if a | b and b | c, then a | c
    Example: 4 | 8 and 8 | 24, so 4 | 24.
         Primes


A positive integer p greater than 1 is called prime if the only
positive factors of p are 1 and p.
Note: 1 is not a prime


A positive integer that is greater than 1 and is not prime is called
composite.
   The fundamental theorem of arithmetic:
Every positive integer can be written uniquely as the product of
primes, where the prime factors are written in order of increasing
size.
       Primes

   Examples:

     15 =       3·5
     48 =       2·2·2·2·3 = 24·3
     17 =       17
     100 =      2·2·5·5 = 22·52
     512 =      2·2·2·2·2·2·2·2·2 = 29
     515 =      5·103
     28 =       2·2·7
    Primes

If n is a composite integer, then n has a prime divisor less
than or equal n .

This is easy to see: if n is a composite integer, it must
have at least two prime divisors. Let the largest two be p1
and p2. Then p1p2 <= n.

 p and p2 cannot both be greater than
 1

 n , because then p1p2 > n.
        The Division Algorithm

Let a be an integer and d a positive integer.
Then there are unique integers q and r, with

0  r < d, such that a = dq + r.

In the above equation,
 d is called the divisor,
 a is called the dividend,

 q is called the quotient, and
 r is called the remainder.
        The Division Algorithm


   Example:

   When we divide 17 by 5, we have

   17 = 53 + 2.

    17 is the dividend,
    5 is the divisor,
    3 is called the quotient, and
    2 is called the remainder.
         The Division Algorithm

   Another example:
   What happens when we divide -11 by 3 ?
   Note that the remainder cannot be negative.
   -11 = 3(-4) + 1.

    -11 is the dividend,
    3 is the divisor,
    -4 is called the quotient, and
    1 is called the remainder.
      Greatest Common Divisors

Let a and b be integers, not both zero.
The largest integer d such that d | a and d | b is called the greatest
common divisor of a and b.
The greatest common divisor of a and b is denoted by gcd(a, b).

Example 1: What is gcd(48, 72) ?
The positive common divisors of 48 and 72 are
1, 2, 3, 4, 6, 8, 12, 16, and 24, so gcd(48, 72) = 24.
Example 2: What is gcd(19, 72) ?
The only positive common divisor of 19 and 72 is
1, so gcd(19, 72) = 1.
      Greatest Common Divisors

   Using prime factorizations:
a = p1a1 p2a2 … pnan , b = p1b1 p2b2 … pnbn ,
where p1 < p2 < … < pn and ai, bi  N for 1  i  n


   gcd(a, b) = p1min(a1, b1 ) p2min(a2, b2 ) … pnmin(an, bn )

   Example:

a = 60 = 22 31 51
b = 54 = 21 33 50
gcd(a, b) =            21 31 50 = 6
        Relatively Prime Integers

   Definition:
Two integers a and b are relatively prime if
gcd(a, b) = 1.

   Examples:
Are 15 and 28 relatively prime?
Yes, gcd(15, 28) = 1.

Are 55 and 28 relatively prime?

Yes, gcd(55, 28) = 1.

Are 35 and 28 relatively prime?

No, gcd(35, 28) = 7.
        Relatively Prime Integers


   Definition:
The integers a1, a2, …, an are pairwise relatively prime if gcd(ai, aj)
= 1 whenever 1  i < j  n.

   Examples:
Are 15, 17, and 27 pairwise relatively prime?
No, because gcd(15, 27) = 3.

Are 15, 17, and 28 pairwise relatively prime?
Yes, because gcd(15, 17) = 1, gcd(15, 28) = 1 and gcd(17, 28) = 1.
         Least Common Multiples

   Definition:
The least common multiple of the positive integers a and b is the
smallest positive integer that is divisible by both a and b.
   We denote the least common multiple of a and b by lcm(a, b).
   Examples:
    lcm(3, 7) = 21
    lcm(4, 6) = 12
    lcm(5, 10) = 10
        Least Common Multiples

   Using prime factorizations:
a = p1a1 p2a2 … pnan , b = p1b1 p2b2 … pnbn ,
where p1 < p2 < … < pn and ai, bi  N for 1  i  n


   lcm(a, b) = p1max(a1, b1 ) p2max(a2, b2 ) … pnmax(an, bn )

   Example:
    a = 60 = 22 31 51
    b = 54 = 21 33 50
    lcm(a, b) =         22 33 51 = 4275 = 540
GCD and LCM

         a = 60 =   22 31 51

         b = 54 =   21 33 50


   gcd(a, b) =      21 31 50   =6

   lcm(a, b) =      22 33 51   = 540


 Theorem: ab =       gcd(a,b)lcm(a,b)
     Modular Arithmetic

Let a be an integer and m be a positive integer.
We denote by a mod m the remainder when a is divided by m.

   Examples:


9 mod 4 =       1
9 mod 3 =       0
9 mod 10 =      9
-13 mod 4 =         3
      Congruences

Let a and b be integers and m be a positive integer. We say that a is
congruent to b modulo m if
m divides a – b.

We use the notation a  b (mod m) to indicate that a is congruent
to b modulo m.

In other words:
a  b (mod m) if and only if a mod m = b mod m.
       Congruences

Examples:
Is it true that 46  68 (mod 11) ?
Yes, because 11 | (46 – 68).

Is it true that 46  68 (mod 22)?
Yes, because 22 | (46 – 68).

For which integers z is it true that z  12 (mod 10)?
It is true for any z{…,-28, -18, -8, 2, 12, 22, 32, …}


Theorem: Let m be a positive integer. The integers a and b are
congruent modulo m if and only if there is an integer k such that a = b
+ km.
      Congruences

 Theorem: Let m be a positive integer.
If a  b (mod m) and c  d (mod m), then
a + c  b + d (mod m) and ac  bd (mod m).
Proof:
We know that a  b (mod m) and c  d (mod m) implies that there
are integers s and t with
b = a + sm and d = c + tm.
Therefore,
b + d = (a + sm) + (c + tm) = (a + c) + m(s + t) and
bd = (a + sm)(c + tm) = ac + m(at + cs + stm).

Hence, a + c  b + d (mod m) and ac  bd (mod m).
        Congruences

Theorem: Let m be a positive integer. a  b (mod m) iff a mod m = b mod m.
Proof:
Let a = mq1 + r1, and b = mq2 + r2.
Only if part: a mod m = b mod m  r1 = r2, therefore
        a – b = m(q1 – q2), and a  b (mod m).
If part: a  b (mod m) implies
                   a – b = mq
   mq1 + r1 – (mq2 + r2) = mq
                 r1 – r2 = m(q – q1 + q2).
Since 0  r1, r2  m, 0  |r1 - r2|  m. The only multiple in that range is 0.
Therefore r1 = r2, and a mod m = b mod m.
      The Euclidean Algorithm

 The Euclidean Algorithm finds the greatest common divisor of two
integers a and b.
For example, if we want to find gcd(287, 91), we divide 287 by 91:

287 = 913 + 14

We know that for integers a, b and c,
if a | b and a | c, then a | (b + c).
Therefore, any divisor (including their gcd) of 287 and 91 must also
be a divisor of 287 - 913 = 14.
Consequently, gcd(287, 91) = gcd(14, 91).
       The Euclidean Algorithm

   In the next step, we divide 91 by 14:
   91 = 146 + 7
   This means that gcd(14, 91) = gcd(14, 7).

   So we divide 14 by 7:
   14 = 72 + 0
   We find that 7 | 14, and thus gcd(14, 7) = 7.

   Therefore, gcd(287, 91) = 7.
       The Euclidean Algorithm

   In pseudocode, the algorithm can be implemented as follows:

 procedure gcd(a, b: positive integers)
x := a
y := b
while y  0
begin
       r := x mod y
       x := y
       y := r
end {x is gcd(a, b)}
           Representations of Integers

Let b be a positive integer greater than 1.
Then if n is a positive integer, it can be expressed uniquely in the
form:

   n = akbk + ak-1bk-1 + … + a1b + a0,

where k is a nonnegative integer,
a0, a1, …, ak are nonnegative integers less than b,
and ak  0.


Example for b=10:
859 = 8102 + 5101 + 9100
        Representations of Integers

   Example for b=2 (binary expansion):
   (10110)2 = 124 + 122 + 121 = (22)10

   Example for b=16 (hexadecimal expansion):
   (we use letters A to F to indicate numbers 10 to 15)
   (3A0F)16 = 3163 + 10162 + 15160 = (14863)10
       Representations of Integers


How can we construct the base b expansion of an integer n?
First, divide n by b to obtain a quotient q0 and remainder a0, that is,

n = bq0 + a0, where 0  a0 < b.

The remainder a0 is the rightmost digit in the base b expansion of n.

Next, divide q0 by b to obtain:

q0 = bq1 + a1, where 0  a1 < b.

a1 is the second digit from the right in the base b expansion of n.
Continue this process until you obtain a quotient equal to zero.
       Representations of Integers

Example:
What is the base 8 expansion of (12345)10 ?

First, divide 12345 by 8:
12345 = 81543 + 1


1543 = 8192 + 7
192 = 824 + 0

24 = 83 + 0

3 = 80 + 3


   The result is: (12345)10 = (30071)8.
    Representations of Integers

 procedure base_b_expansion(n, b: positive integers)
q := n

k := 0

while q  0
begin

       ak := q mod b
       q := q/b
       k := k + 1
end

{the base b expansion of n is (ak-1 … a1a0)b }
       Addition of Integers

   How do we (humans) add two integers?

                       1 11    carry
   Example:            7583
                   +    4932

                       12515
                                  1 1              carry
Binary expansions:                 (1011)2
                                 + (1010)2

                                  ( 1 0 1 0 1 )2
       Addition of Integers

Let a = (an-1an-2…a1a0)2, b = (bn-1bn-2…b1b0)2.
How can we algorithmically add these two binary numbers?

First, add their rightmost bits:

a0 + b0 = c02 + s0,

where s0 is the rightmost bit in the binary expansion of a + b, and
c0 is the carry.
Then, add the next pair of bits and the carry:
a1 + b1 + c0 = c12 + s1,

where s1 is the next bit in the binary expansion of a + b, and c1 is
the carry.
           Addition of Integers


   Continue this process until you obtain cn-1.

   The leading bit of the sum is sn = cn-1.

   The result is:
   a + b = (snsn-1…s1s0)2
        Addition of Integers

Example:
Add a = (1110)2 and b = (1011)2.



 0 a + b0 = 0 + 1 = 02 + 1, so that c0 = 0 and s0 = 1.
 1 a + b1 + c0 = 1 + 1 + 0 = 12 + 0, so c1 = 1 and s1 = 0.
 2 a + b2 + c1 = 1 + 0 + 1 = 12 + 0, so c2 = 1 and s2 = 0.
 3 a + b3 + c2 = 1 + 1 + 1 = 12 + 1, so c3 = 1 and s3 = 1.
 4 s = c3 = 1.

   Therefore, s = a + b = (11001)2.
     Addition of Integers


 procedure add(a, b: positive integers)
c := 0

for j := 0 to n-1

begin

        d := (aj + bj + c)/2
        sj := aj + bj + c – 2d
        c := d
end

sn := c
{the binary expansion of the sum is (snsn-1…s1s0)2}
Applications of Number
Theory
       GCD as linear combination
           If a and b are positive integers, then
            there exist integers s and t such that
            gcd(a,b) = sa + tb
Example
Express gcd(252, 198) = 18 as a linear combination of 252 and 198
                                                      18 = 54 – 1*36
  252 = 1*198 + 54
                                                      36 = 198 – 3*54
  198 = 3*54 + 36
                                                      18 = 54 – 198 + 3*54
  54 = 1*36 + 18
                                                           = 4*54 – 1*198
  36 = 2*18        Start from next-to-last division   54 = 252 – 1*198

                                                      18 = 4*252 – 5*198
Hashing Function
   How to assign memory location that the data
    can be retrieved quickly
       h(k) = k mod m, where k is key and m is the
        number of available memory locations


   Example
       m = 111, h(064212848) = ?
       h(064212848) = 14
       So this data will be placed at memory location =
        14
        Caesar‟s Encryption
           Invented by Julius Caesar to make a
            secret message
Algorithm
1. Convert each alphabet in a message to its corresponding
    number p: A = 0, B = 1,…, Z = 25, and p  25 only
2. Select key k as a positive integer, Caesar himself often use k = 3
3. To encrypt, applies this formula:
               f(p) = (p+k) mod 26
4. Then convert each number back to alphabet and pack the
    ciphertext with its key
5. To decrypt, do step 1 and 2 then applies this formula:
               f-1(p) = (p-k) mod 26
6. If p+k or p-k is greater than or lesser than 25, shift it with 25
Example: Caesar‟s Encryption
   Encrypt “DISCRETE MATH”

DISCRETE             MATH

3 8 18 2 17 4 19 4   12 0 19 7

      f(p+k) mod 26

6 11 21 5 20 7 22 7 15 3 22 10

GLVFUHWH             PDWK

GLVFUHWH PDWK
      Linear Congruences
   Congruence of a form ax  b(mod m), where
    m is positive integer, a and b are integers, x
    is a variable is called Linear Congruence
   To solve this congruence is to find all
    possible x that satisfies the formula
   Inverse modulo
   More details in Section 3.7
      Chinese Reminder Theorem
   from San-Tsu‟s puzzle, ancient Chinese
    mathematician
   can be solved using Modular arithmetic
   More details in Section 3.7
RSA
   Public key crypto system
   Based on modular exponential
   To encrypt
       C = Me mod n
   To decrypt
       Cd  M(mod pq)
   More details see RSA section and
    Example 11 and 12 on page 241 - 244
Matrices
    Matrices
   A matrix is a rectangular array of numbers.
   An mn (“m by n”) matrix has exactly m
    horizontal rows, and n vertical columns.
   An nn matrix is called a square matrix,
    whose order is n.
   The plural of matrix is matrices.

                                     2 3 
                                     5  1      a 32
                                          
                                     7 0        matrix
                                          
Example

             Row: 3 rows
1 1        Column: 2 columns
0 2         Size: 3 x 2 matrix
    
          


1 3        Square: NO
    
Example

             Row: 2 rows
1 2        Column: 2 columns
3 4        Size: 2 x 2 matrix
   
             Square: YES
 Example
             0 1 
  1 1      2 1         2 2 1
A       B          C     
  2 0      1 2         0 1 0
                 
                 Rows and columns?
  0 1           Size?
D   
              

  2 0          Square?
                 Equal?
Matrix Equality
   Two matrices A and B are equal iff they
    have the same number of rows, the same
    number of columns, and all corresponding
    elements are equal.

       3 2  3 2 0
        1 6     1 6 0
                        
Row and Column Order
   The rows in a matrix are usually indexed 1
    to m from top to bottom. The columns are
    usually indexed 1 to n from left to right.
    Elements are indexed by row, then column.
                     a1,1 a1, 2      a1,n 
                    a      a2 , 2            
                                      a2 , n 
    A  [ai , j ]  
                       2 ,1

                                    
                                             
                    am,1 am, 2       am , n 
    Let                           The ith row of A is the 1 x n matrix
                 jth column

         a11 a12 ... a1n                   a i1   a i2 ... a in 
        a    a 22 ... a 2n        The jth column of A is the n x 1 matrix
         21                   

         .     .        .                            a1j 
    A                                              a 
ith row
           .    .        . 
                                                      2j 
         .     .        .                            . 
                                                     
        a n1 a n2 ... a nn 
                                                     . 
                                                       . 
                                                       
                                                      a nj 
                                                       
                           a11 a12 ... a1n 
                          a    a 22 ... a 2n 
                           21                
                           .     .        . 
                        A                   
                            .    .        . 
                           .     .        . 
                                             
                          a n1 a n2 ... a nn 
                                             
   The (i,j)th element or entry of A is the element aij, that is, the number in
    the ith row and jth column of A.
   A convenient shorthand notation for expressing the matrix A is to write
    A = [aij], which indicates that A is the matrix with its (i,j)th element
    equal to aij.
Matrix Arithmetic
   Addition/Sum
   Multiplication/Product
Matrix Sums
   The sum A+B of two matrices A, B (which
    must have the same number of rows, and
    the same number of columns) is the matrix
    (also with the same shape) given by adding
    corresponding elements.
   A+B = [ai,j+bi,j]
                 2 6   9 3  11          9
                 0  8   11 3   11  5
                                          
Example: Addition
   Find A + B, where:
      1 2         4 3
    A          B   
      3 4         2 1
Example: Addition
   Find A + B, where:

      1 2          0 1
      3 4
    A             2 3
                  B
                        
      5 6
                  4 5
                        
Example: Addition
   Find A + B, where:

      1 2
      3 4        0 1 2
    A         B      
      5 6        3 4 5 
         
     Matrix Products
   For an mk matrix A and a kn matrix B, the product AB is
    the mn matrix:
                                k            
           AB  C  [ci , j ]   ai ,b, j 
                                 1         
   i.e., element (i,j) of AB is given by the vector dot product
    of the ith row of A and the jth column of B (considered as
    vectors).
   In other words, if AB = [cij], then

        cij = ai1b1j + ai2b2j + … + aikbkj.
   Note: Matrix multiplication is not commutative!
 Matrix Product Example
    An example matrix multiplication to
     practice in class:

           0  1 1 0
0 1  1             1 0  5  1
2 0 3   2 0  2 0 3  2 11 3 
        1 0    3 1              
                    
Example: Multiplication
   Find AB, where:

       1 2            4 3
     A             B   
       3 4            2 1

   Find BA
Example: Multiplication
   Find AB, where:

      1 2
      3 4        0 1 2
    A         B      
      5 6        3 4 5 
         
Example: Multiplication
   Find AB, where:

                        0 1 2
      A  2 1       B      
                        3 4 5 
    Example: Multiplication
       Suppose
        A is a 6 × 8 matrix,
        B is an 8 × 5 matrix, and
        C is a 5 × 9 matrix.
Find the number of rows, the number of
   columns, and the number of entries in
   A(BC).
    Example: Multiplication
   Find A2 where,


                 1 2
               A   
                 3 4
    Identity Matrices
   The identity matrix of order n, In, is the
    order-n matrix with 1’s along the upper-left
    to lower-right diagonal and 0’s everywhere
    else.                   1 0  0
           1 if i  j    0 1  0 
     I n                
                                    
            0 if i  j         
                                     
                             0 0  1 
Identity matrix
   Multiplying a matrix by an appropriate
    sized identity matrix does not change
    this matrix. In other words, when A is
    an m x n matrix, we have
               AIn = ImA = A
   Example: Identity matrix
   Find identity matrix of each of   the
    following matrices:
                                1     2 3
  1 2           4 3         2        
A         B            C       1 2
  3 4           2 1
                                2
                                      3 3
                                          
             0 1 2
           D      
             3 4 5 
Matrix Inverses
   For some (but not all) square matrices A,
    there exists a unique multiplicative inverse
    A-1 of A, a matrix such that A-1A = In.
   If the inverse exists, it is unique, and
    A-1A = AA-1.
   We won’t go into the algorithms for matrix
    inversion...
Matrix Multiplication Algorithm
procedure matmul(matrices A: mk, B: kn)
for i := 1 to m               (m)· What’s the  of its
                                       time complexity?
  for j := 1 to n begin (n)·(
                                   (1)+      Answer:
       cij := 0
                                              (mnk)
       for q := 1 to k               (k) ·
               cij := cij + aiqbqj      (1))

  end {C=[cij] is the product of A and B}
Powers of Matrices
If A is an nn square matrix and p0, then:
 Ap  AAA···A    (A0  In)
                           3
         p times    2 1  2         1  2 1  2 1
                     1 0    1        1 0    1 0
                                   0                  
                              2      1  3      2
   Example:                            2  1
                               1    0            
                                4     3
                               
                                  3  2
                                         
Matrix Transposition
   If A=[aij] is an mn matrix, the transpose of
    A (often written At or AT) is the nm matrix
    given by At = B = [bij] = [aji] (1in,1jm)

                      t  2 0 
         2 1     3           
          0  1  2   1  1 
                      3  2
      Flip
                               
     across
    diagonal
   Example: Transpose
   Find transpose of   the following
    matrices:
                  1    2
  1 2           3           0 1 2
A          B       4    D      
  3 4                         3 4 5 
                  5
                       6
                         
Symmetric Matrices
   A square matrix A is symmetric iff A=At. I.e.,
    i,jn: aij = aji .
   Which is symmetric?

      1 1  2 1 3  3 0 1 
      1 1  1        
                 0  1 0 2  1 
                            
      1 1  3  1 2  1 1  2
                           
    Definition 7: Symmetric
   A square matrix A is called symmetric if A = At.
    Thus A = [aij] is symmetric if aij = aji for all i
    and j with 1 ≤ i ≤ n and 1 ≤ j ≤ n.
Example: Symmetric

  2 4    2 1 4   1 3
 4 1    1 3 2     3 1
                     
          4 2 0
                    2 1
                         
Zero-One Matrices
   All elements of a zero-one matrix are 0 or 1
       Representing False & True respectively.
   Useful for representing other structures.
       E.g., relations, directed graphs
   The meet of A, B (both mn zero-one matrices):
      AB : [aijbij] = [aij bij]

   The join of A, B:
      AB : [aijbij]
    Boolean Products
   Let A=[aij] be an mk zero-one matrix,
    & let B=[bij] be a kn zero-one matrix,
   The boolean product of A and B is like
    normal matrix , but using  instead + in
    the row-column “vector dot product.”
                       
                                  
                              k
    A⊙B  C  [cij ]   ai  bj 
                         1      
Arithmetic/Boolean Products


                            k
                                      
   AB  C  [ci , j ]   ai ,b, j 
                         1         

                   
                              
                         k
A⊙B  C  [cij ]   ai  bj 
                     1      
Boolean Powers
   For a square zero-one matrix A, and any
    k0, the kth Boolean power of A is simply
    the Boolean product of k copies of A.
   A[k]  A⊙A⊙…⊙A
            k times
Homework 3
   Chapter 3
       Section 3.1:
           1, 5, 22, 39, 42

       Section 3.2:
           1, 7, 9, 19, 21

       Section 3.3:
           1, 3, 9, 11, 25, 27

       Section 3.4:
           1, 9, 17, 31, 32

       Section 3.5:
           1, 21, 25

       Section 3.6:
           1, 5, 19, 23

       Section 3.7:
           7, 21

       Section 3.8:
           1, 3, 5, 31, 33

       Supplementary:
            none

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:52
posted:8/3/2011
language:English
pages:192