Coding Theory

Document Sample
Coding Theory Powered By Docstoc
					Coding Theory

Nikki
Mark
April 28th, 2006
What is Coding Theory?

 Algebraic codes are now used in CD
  players, fax machines, modems, and
  bar code scanners.
 Algebraic coding has nothing to do
  with secret codes.
 The goal of coding theory is to devise
  message encoding and decoding
  methods that are reliable, efficient,
  and reasonably easy to implement.
Why do we need Coding
Theory?
 To motivate this theory, we imagine
  wanting to send one of two signals to
  a spaceship on Mars.
 0 to orbit

 1 to land

 But it is possible that some sort of
  interference (noise) can cause an
  incorrect message to be received.
Adding Redundancy to our
Messages
 To decrease the effects of noise, we
  add redundancy to our messages.
 First method: repeat the digits multiple
  times.
 Thus, the computer is programmed to
  take any five-digit message received
  and decode the result by majority rule.
Majority Rule

   So, if we sent 00000, and the computer
    receives any of the following, it will still be
    decoded as 0.

00000          11000         Notice that for the
10000          10100         computer to decode
01000          10010         incorrectly, at least
00010          10001         three errors must be
00001          etc.          made.
Independent Errors

 Using the five-time repeats, and
  assuming the errors happen
  independently, it is less likely that
  three errors will occur than two or
  fewer will occur.
 This is called the maximum likelihood
  decoding.
More complicated numbers

   We are going to send a sequence of 0s
    and 1s of length 500. Assume the
    probability of an error occurring is .01 for
    any particular digit (and they happen
    independently).

   No redundancy:
    P(error free) = .99 500  .0066
“Three fold Repetition
Scheme”
   Sending each digit three times, and
    decoding each block of three digits by
    majority rule, the probability increases:
   Suppose 1 is sent, it will be decoded as 0 iff
    the block received is 001, 010, 100, 000.
    The probability that this will occur is:
   P(error) =(. 01) 2 [3(. 99 )  .01]  .000298
   => P(error free for one digit) >= .9997
   =>P(error free message) >=.9997500 = .86
          A Purdy Picture

          Original message 0       Encoded message 00000

earth                       encoder              transmitter 
     spacecraft                        decoder        
 Noise                 Received message 10001      Decoded message 0
Why don’t we use this?

 Repetition codes have the advantage of
  simplicity, both for encoding and
  decoding
 But, they are too inefficient!
 In a five-fold repetition code, 80% of all
  transmitted information is redundant.
 Can we do better?
 Yes!
What is a linear code?

   Def: An (n,k) linear code over a finite
    field F is a k-dimensional subspace V of
    the vector space

      F  F  F  ... F
        n


                 n copies
    over F. The members of V are called the
      code words. When F is 2 , the code is
      called binary.
A Hamming (7,4) code
   A Hamming code of (n,k) means the
    message of k digits long is encoded into the
    code word of n digits.

 The 16 possible messages:
0000       1010        0011            1111
0001       1100        1110
0010       1001        1101
0100       0110        1011
1000       0101        0111
Encoding with a Generating
Matrix
   A generating matrix is one of k x n matrix,
    with a k x k Identity matrix adjoined with a
    k x (n – k) matrix.
   Our generating matrix:

        1     0 0 0 0 1 1
        0     1 0 0 1 0 1
     G=                  
        0     0 1 0 1 1 0
                         
        0     0 0 1 1 1 1
Encoding a message

   For each message v, find the left coset
    vG:
   Let v = [ 0 1 1 0 ]
                          1 0   0   0   0   1 1
   vG = [ 0 1 1 0 ]      0 1
                                0   0   1   0 1
                                                
                          0 0   1   0   1   1 0
                                               
                          0 0   0 1     1   1 1

    =[0110011]
Encoding those messages

          Message  codeword
0000  0000000      0110  0110011
0001  0001111      0101  0101010
0010  0010110      0011  0011001
0100  0100101      1110  1110000
1000  1000011      1101  1101001
1010  1010101      1011  1011010
1100  1100110      0111  0111100
1001  1001100      1111  1111111
Hamming Distance

   Def: The Hamming distance between
    two vectors of a vector space is the
    number of components in which they
    differ, denoted d(u,v).




                           Mr. Hamming…
Hamming Distance

 Ex. 1: The Hamming distance
  between
   v=[1011010]
   u=[0111100]
d(u, v) = 4

   Notice: d(u,v) = d(v,u)
Hamming weight of a Vector

   Def: The Hamming weight of a vector
    is the number of nonzero components
    of the vector, denoted wt(u).
Hamming weight of a code

   Def: The Hamming weight of a linear
    code is the minimum weight of any
    nonzero vector in the code.
Hamming Weight

 Ex. 2: The Hamming weight of
   v=[1011010]
   u=[0111100]
   w=[0100101]
are:
  wt(v) = 4
  wt(u) = 4
  wt(w) = 3
Theorem 31.1 ~ Properties of
Hamming Distance and Weight
 For any vectors u, v, and w,
   1) d(u,v) = wt(u – v).
   2) d(u,v) < d(u,w) + d(w,v).
Proof:
1) Observe that both d(u, v) and
    wt(u – v) equal the number of
    positions in which u and v differ.
Proving Part 2

2) d(u,v) < d(u,w) + d(w,v).

Proof:
2) Note that, if u and v differ in the ith
  position and u and w agree in the ith
  position, then w and v differ in the ith
  position. Refer to Example 2.
Example of Proof

   v=[1011010]
   u=[0111100]
   w=[0100101]

d(u,v) = 4        d(u,w) = 3
d(w,v) = 7

And: d(u,v) < d(u,w) + d(w,v)
   4 < 3 + 7.
Theorem 31.2 ~ Correcting
Capability of a Linear Code
   If the Hamming weight of a linear
    code is at least 2t + 1, then the code
    can correct any t or fewer errors.
    Alternatively, the same code can
    detect any 2t or fewer errors.

 Example: Using our original code, the
  Hamming weight is 3 = 2(1) + 1.
 Thus, it can: correct 1 error,
   OR, it can: detect 2 errors.
Why OR?

 We’ll do this by counterexample.
 We have our (7,4) Hamming Code,
  and the weight is 3 = 2(1) + 1.
 Say we receive the word 0001010.

 It could have had one error, in which
  0101010 was intended, and we can
  detect that single error.
Why OR, cont.

   It could have had two errors, in which
    0000000, 0001111, 1011010 could be
    candidates.
   What do we do?
   If we chose detection, we would note that
    too many possibilities exist, and we would
    ask for retransmission.
   If we chose correction, we could potentially
    make a mistake and assume that the
    message was intended to be 0101010.
Moral of the story

 We must be careful about detecting
  and correcting errors.
 It is safest to ask for retransmission if
  possible, to eliminate any guess-work.
A better check

 If we write the Hamming weight of a
  linear code in the form:
      2t + s + 1
 We can correct any t errors AND
  detect any t + s errors.
An example for ya
 Let’s assume some linear code has a
  Hamming weight of 5.
 Well, 5 = 2(1) + 2 + 1
       • So we can correct any 1 error and detect
         up to 3 errors.
   Or, 5 = 2(0) + 4 + 1
       • So we can detect up to any 4 errors.
   Or, 5 = 2(2) + 0 + 1
       • So we can correct any 2 or fewer errors.
Let’s Play a Game!

   Yay for Math Horizons!




 This uses the fact that each error is
  uniquely determined.
 Two different code words differ in at
  least three different positions.
Creating the
Parity-Check Matrix
   How to make H, the Parity-Check
    Matrix from G, the Generating Matrix.
       G [ Ik | A ]
       Compute –A
       H=
              A
               In  k 
                      
Homework #1

 From your textbook:
 Page 540, numbers 4, 5, and 12.



   Not too bad, we promise!
Decoding the Code Words

   Four types of decoding methods for
    linear codes:

     Nearest-neighbor
     Parity-Check Matrix
     Coset Decoding
     Same Coset-Same Syndrome
The Steps of
Nearest Neighbor Decoding
   This is a comparison between the
    received code word, v, and all
    possible code words. Look for the
    smallest distance between v and v’.
     If v’ is unique, assume v = v’.
     If v’ is not unique, ask for
      retransmission.
Nearest Neighbor Example
   Ex. 3:
    We receive 0001110. This is not in our
     list of possible code words from our
     original list.
    We compare 0001110 to each code
     word, and see that it differs from
     0001111 by only 1, and all others by
     more than one.
    We assume 0001110 was intended to
     be 0001111.
    We decode 0001111 to its original
     message of 0001.
Creating the
Parity-Check Matrix
   How to make H, the Parity-Check
    Matrix from G, the Generating Matrix.
       G [ Ik | A ]
       Compute –A
       H=
              A
               In  k 
                      
How to Use the Parity-
Check Matrix to Decode
   For any received word w, compute wH.
   If wH is the zero vector, assume that no error
    was made.
   If there is exactly one nonzero element,
      s  F and a row i of H such that wH is s
    times row i, assume that the sent word was
    w – (0 … s … 0), where s occurs in the ith
    component. If there is more than one such
    instance, do not decode.
   If wH does not fit into either category, we
    know that at least two errors occurred in
    transmission, and we do not decode.
Parity-Check Decoding
Example
Using G = 1 0 0 0 0 1 1
             0 1 0 0 1 0 1 
                           
             0 0 1 0 1 1 0 
                           
              0 0 0 1 1 1 1
                0   1   1
Create H =      1
                    0   1
                          
                1   1   0
                         
                1   1   1
                1   0   0
                         
                0   1   0
                0       1
                    0    
Example cont.
 We receive 0001110.
 Compute wH:
     We get 001.
     001 is the 7th (last) row of our Parity-
      Check Matrix H.
     Thus we assume there is a problem in
      position 7.
     Compute 0001110 – 0000001
      = 0001111, which is the intended
      code word, which is decoded to be the
      message 0001.
Theorem 31.3 ~ Parity-
Check Matrix Decoding
   Parity-check matrix decoding will
    correct any single error iff the rows of
    the parity-check matrix are nonzero
    and no one row is a scalar multiple of
    any other row.
Example of Thm. 31.3
         1   1 0
         0   1 1
   A=             Good! Rows are
         1   0 1   linearly independent
                
         1   0 0
         0   1 0
                
         0
             0 1
                 

         2   1
   B=   2   2     Bad! Row 1 and row
              
         1   2     3 are NOT linearly
                   independent.
         1   0
         0
             1
               
How to Use Coset Decoding

 To use coset decoding, one must
  create a “standard array” for the linear
  code.
 Once the standard array is
  constructed, find the received code
  word in the array and identify the
  column in which it lies.
 Decode the code word as the number
  at the top of the column minus the
  “coset leader.”
How to create the dreaded
“Standard Array”
 Use C, the group of code words, as
  the heading for each column.
 Next, look at all the possible vectors,
  and choose the next available one
  with minimum weight.
 This becomes the first “coset leader,”
  place this vector as the beginning of
  the second row.
 Constructing the Standard
 Array
0000000       0001111      0010110       0011001
1000000

    C = {0000000, 0001111, 0010110,
     0011001}
    The vector 1000000 is the next number
     with least weight, so it becomes the first
     coset leader.
Continuing the Standard
Array
 Next, add the coset leader to each
  member in C, the column headings.
 Of all the code words left, find the one
  with least weight, and this becomes
  the coset leader for the third row.
 Continue this until all possible code
  words are placed in the standard
  array.
  Standard Array
0000000 0001111 0010110 0011001
1000000 1001111 1010110 1011001
0100000

     The vector 0100000 is the next code
      word with least weight that is still left, so
      it becomes the next coset leader.
The Complete Standard Array
   We can’t complete the whole array, it’d be 32 rows,
    but here’s as much as we need to decode 0001110

    0000000 0001111 0010110 0011001
    1000000 1001111 1010110 1011001
    0100000 0101111 0110110 0111001
    0010000 0011111 0000110 0001001
    0001000 0000111 0011110 0010001
    0000100 0001011 0010010 0011101
    0000010 0001101 0010100 0011010
    0000001 0001110 0010111 0011000
    1100000 1101111 1110110 1111001 …
        Decoding with the Standard
        Array
           0000000   0001111     0010110 0011001
           1000000   1001111     1010110 1011001
           0100000   0101111     0110110 0111001
           0010000   0011111     0000110 0001001
           0001000   0000111     0011110 0010001
           0000100   0001011     0010010 0011101
           0000010   0001101     0010100 0011010
           0000001   0001110   0010111 0011000
           1100000   1101111     1110110 1111001
   We received 0001110. Notice 0001111 is the column
    heading, and 0000001 is the coset leader.
Final Decoding

 So, we found 0001110 in the second
  column, under 0001111, with a coset
  leader of 0000001.
 So, we assume the column heading is
  the intended message: 0001111.
 We could also do the received word
  0001110, plus the coset leader,
  0000001, to get 0001111.
Theorem 31.4 ~ Coset Decoding
is Nearest-Neighbor Decoding
 In coset decoding, a received word w is
  decoded as a code word c such that d(w,
  c) is minimal.
Proof:
 Let C be a linear code, and let w be any
  received word.
 Suppose that v is the coset leader for the
  coset w+C.
Thus, w+C = v+C.
 Proof Continued

 Thus, w = v + c, for some c in C.
  w is decoded as c.
 Now, if c’ is any code word, then w-c’ exists in w
  + C = v + C, so the
 wt(w-c’) > wt(v), since v was chosen because
  wt(v) is minimal among the members of v+C.
Therefore, d(w,c’) = wt(w-c’) > wt(v) = wt(w-c) =
  d(w,c)
Thus, w is decoded as a code word c such that
  d(w,c) is minimal.
The Syndrome

   Def: If an (n, k) linear code over F has
    parity-check matrix H, then, for any vector u
    in F n , the vector uH is called the
    syndrome of u.
How to use
Syndrome Decoding
 Calculate wH, the syndrome of w.
 Find the coset leader v such that

  wH = vH.
 Assume that the vector sent was

  w – v.
Syndrome Decoding
Example
                                      0   1 1
 Let w = 0001110                     1   0 1
 Let H be our parity-check matrix:          
                                      1   1 0
 Thus, wH = 001                             
 [1000000]H = 011
                                      1   1 1
                                      1   0 0
  [0100000]H = 101                           
                                      0   1 0
  [0010000]H = 110                    0
                                          0 1
                                              
  [0001000]H = 111
  [0000100]H = 100
  [0000010]H = 010
  [0000001]H = 001…
Thus, v = 0000001
Syndrome Decoding
Continued
 So, we have w = 0001110
 And we have v = 0000001

 Finally, all we have to do is compute

  w – v = 0001111. Which was the
  intended message.
Theorem 31.5 ~ Same
Coset - Same Syndrome
 Let C be an (n, k) linear code over F with a
  parity-check matrix H. Then, two vectors of
  are in the same coset of C iff they have the
  same syndrome.
Proof:
  Two vectors u and v are in the same coset of
  C iff u-v is in C, since C is closed by
  definition. Note, vH = 0 iff v is in C.
 Thus, u and v are in the same coset iff 0 = (u-
  v)H = uH – vH.
 0 = uH – vH  uH = vH.
Other Codes
   So, we only touched base on linear codes, and
    more specifically, the linear Hamming codes.
   There are many others!
       Reed-Solomon Codes
       Self-Dual Codes
       Golay Codes
       Cyclic Codes
       Quadric Residue Codes
       Bose-Chaudhuri-Hocquenghem (B.C.H.) Codes
       Etc.
Any Questions?

   Homework:

Problems from the chapter, page 540:
17 (and complete the standard array given to
  you) and 29

Use hint in back of book for 29. Sorry, we had
  to have some Abstract in there for ya!

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:26
posted:9/29/2012
language:Unknown
pages:59