Distributed Compression
For Binary Symetric Channels
Kivanc Ozonat
Distributed Compression For Binary 1
Symetric Channels
Introduction
• Description of the Problem
• Slepian-Wolf Theorem
• Prior Work
• Basic Encoder-Decoder Scheme
• Methodology
• Results
Distributed Compression For Binary 2
Symetric Channels
Problem Description
• Given two correlated data sets, a noisy version, X , at the
decoder and the original, Y, at the encoder, how to transmit
Y with the best coding efficiency?
• No communication of X and Y at the encoder
Y Encoder Decoder
X
Distributed Compression For Binary 3
Symetric Channels
Slepian-Wolf Theorem
• Slepian-Wolf : Given the following scheme,
R1
Encode X
(X,Y) (X,Y)
Encode Y
R2
Distributed Compression For Binary 4
Symetric Channels
Slepian-Wolf Theorem
• Can transmit X and Y, if:
- R1 > H(X|Y) , R2 > H(Y|X), and
- R1+ R2 > H(X,Y).
R2
H(Y)
H(Y|X)
R1
H(X|Y) H(X)
Distributed Compression For Binary 5
Symetric Channels
Slepian-Wolf Theorem
• Our problem is a special case of this:
R2
H(Y)
H(Y|X)
H(Y|X)
R1
H(X|Y) H(X)
H(X)
Distributed Compression For Binary 6
Symetric Channels
Prior Work
Bin 1
[0 0 0]
[1 1 1] [0 10]
[0 0 1] Bin 2 [1 0 1]
[1 1 0] [0 1 0]
[0 10] Bin 3
[1 0 1]
Bin 4 Y = [0 1 1]
[0 1 1] Channel
[1 0 0]
Encoder Decoder
Distributed Compression For Binary 7
Symetric Channels
Prior Work
How to maximally separate “very long”
input sequences?
Use error-correcting codes.
Distributed Compression For Binary 8
Symetric Channels
Prior Work
1-p
0 0
p
with EQUAL input
p probabilities of 0 and 1.
1 1
1-p
by Ramchandran, Pradhan.
Distributed Compression For Binary 9
Symetric Channels
Prior Work
What if
the input probabilities are NOT EQUAL?
Distributed Compression For Binary 10
Symetric Channels
Methodology
Plane 1 Bit Plane 1
Form the
Huffman Bit Plane 2
Plane 2 Bins using
Code
Error Decoder
The Input
Correcting
Sequence Plane N Bit Plane N
Codes
X
Y sequence
Distributed Compression For Binary 11
Symetric Channels
Encoder
Inputs: 0 (with probability .7) and 1 (with probability .3)
Huffman code 2-length sequences:
00 0 (with probability .49)
01 10 (with probability .21)
10 110 (with probability .21)
11 111 (with probability .09)
Bit-Plane 1: 0, 1 , 1 ,1
Bit-Plane 2: -, 0 , 1 ,1
Bit-Plane 3: - , - , 0 ,1
Distributed Compression For Binary 12
Symetric Channels
Encoder
[001001]
[00], [10], [01]
011 Error
Control
[0], [110], [10] -10
Coding
-0- To Form
Bins
Distributed Compression For Binary 13
Symetric Channels
Decoder
• Decoder receives a BIN NUMBER, which corresponds to
MULTIPLE CODEWORDS.
• How to select the “correct codeword” out of these multiple
codewords?
• Use MAXIMUM LIKELIHOOD detection.
Distributed Compression For Binary 14
Symetric Channels
Decoder
[011]
Bin 4 Decoder [011]
[110]
This is
what the Huffman
decoder codes for
receives 2 length
Assume Y= [01, 11, 10]
sequences
Compute the probability of
[z1 z2 z3]
[z1 z2 z3] given 01,11,10,
using the channel error
probability.
Distributed Compression For Binary 15
Symetric Channels
Parameters
Plane 1 Bit Plane 1
Form the
Huffman Bit Plane 2
Plane 2 Bins using
Code
Error Decoder
The Input
Correcting
Sequence Plane N Bit Plane N
Codes
X
Length 4 Use BCH
(15,k) Y sequence
Distributed Compression For Binary 16
Symetric Channels
Bit Rate vs. Probability of Occurrence of 0’s
(at the fixed error rate p of 0.06)
0.8
0.75
0.7
0.65
0.6
0.55
0.5
0.45
0.4
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9
Distributed Compression For Binary 17
Symetric Channels
Difference between the Actual Bit Rate and the Slepian-Wolf Bound
vs
Error Probability (p)
0.36
0.34
0.32
0.3
0.28
0.26
0.24
0.22
0.05 0.055 0.06 0.065 0.07 0.075 0.08 0.085 0.09
Distributed Compression For Binary 18
Symetric Channels
Conclusions
• Huffman Code is not a very good choice
• Better error correcting codes can be selected.
• Gives good results for low error (p) cases
and for cases in which the Huffman code gives nearly
equal distribution of 0s and 1s.
Distributed Compression For Binary 19
Symetric Channels