# Wavelet Transform

Document Sample

```					 Orthogonal
Transforms

Fourier
Review
• Introduce the concepts of base functions:
– For Reed-Muller, FPRM
– For Walsh
•   Linearly independent matrix
•   Non-Singular matrix
•   Examples
•   Butterflies, Kronecker Products, Matrices
•   Using matrices to calculate the vector of spectral
coefficients from the data vector

Our goal is to discuss the best
approximation of a function using
orthogonal functions
Orthogonal
Functions
Orthogonal Functions
Note that these are arbitrary functions,
we do not assume sinusoids
Illustrate it for Walsh and RM
Mean
Square
Error
Mean Square Error
Important result
• We want to minimize this kinds of
errors.

• Other error measures are also used.
Unitary
Transforms
Unitary Transforms
• Unitary Transformation for 1-Dim. Sequence
– Series representation of                   {u(n),0  n  N  1}
N 1
v  Au  v(k )   a(k , n)u (n), 0  k  N  1
n 0
N 1
u  A* v  u (n)   a* (k , n)v(k ), 0  n  N  1
n 0

where A1  A*T (unitary matrix )

– Basis vectors :       a*  {a* (k , n), 0  n  N  1}T
k

– Energy conservation :
v  Au  || v || 2 || u || 2                  Here is the
proof

N 1                               N 1
( || v ||   | v(k ) |  u A Au  u u   | u (n) |2 || u || 2 )
2            2    *T   *T        *T

k 0                               n 0
• Unitary Transformation for 2-Dim. Sequence
– Definition :             N 1 N 1
v(k , l )   u (m, n)ak ,l (m, n), 0  k , l  N  1
m 0 n 0
N 1 N 1
u (m, n)   v(k , l )ak ,l (m, n), 0  m, n  N  1
*

k 0 l 0
*
– Basis images :  {ak ,l (m, n)}
– Orthonormality and completeness properties
• Orthonormality :   N 1 N 1

 ak ,l (m, n)ak ' ,l ' (m, n)   (k  k ' , l  l ' )
m 0 n 0
*

• Completeness :
N 1 N 1

 ak ,l (m, n)ak ,l (m ' , n ' )   (m  m ' , n  n ' )
k 0 l 0
*
• Unitary Transformation for 2-Dim. Sequence
– Separable Unitary Transforms
ak ,l (m, n)  ak (m)bl (n)
N 1 N 1
v(k , l )   ak (m)u (m, n)al (n)  V  AUA T
m 0 n 0
N 1 N 1
u (m, n)   ak (m)v(k , l )al* (n)  U  A*T VA*
*

k 0 l 0
• separable transform reduces the number of multiplications and
additions from O ( N 4 ) to O ( N 3 )
– Energy conservation
N 1 N 1                N 1 N 1

 | u (m, n) |2   | v(k , l ) |2
m 0 n 0                 k 0 l 0
Properties of Unitary Transform
transform
Consider v  Au, then R v  AR u (A* )t
Covariance
A 2 1 zero mean vecto u is trasforme
r             d                       matrix
1 3            1                        1 ρ 
v                 u                 Ru  
ρ 1
where
2 1            3                           
1  3 ρ         ρ     
then R v               2        2   
 ρ          1 3 ρ 
      2             2
 σ 2 (0)  1  σ 2 (1) ; total energy is equally distributed
u             u

but σ 2 (0)  1  3 ρ        σ 2 (1)  1  3 ρ
v              2          v       2
if ρ  0.95 , then 91% energy in v(0)
and the correlation between v(0) and v(1) is
Ε[v(0)v(1)]
ρ v (0,1)                     0.83
σ v (0) σ v (1)
 energy compactionand decorrelation
Example of arbitrary
basis functions being
rectangular waves
This determining first
function determines next
functions
0   1
Small error with
just 3 coefficients
This slide shows four
base functions
multiplied by their
respective coefficients
This slide shows that
using only four base
functions the
approximation is quite
good

End of example
Orthogonality
and
separability
Orthogonal and separable Image
Transforms
Extending general transforms to 2-
dimensions
Forward
transform

inverse
transform

separable
Fourier
Transforms in
new notations
1. We emphasize generality
2. Matrices
Fourier Transform

separable
Extension of Fourier Transform to
two dimensions
Discrete Fourier Transform (DFT)

The DFT of u(n) ,            n  0,1,2, ,N-1        is
N 1
v(k)  u (n) wN                    k  0,1, , N  1
kn

n 0

 j 2N
New
where wN  e                                       notation
 not scaled properly to be unitary transform
N 1
1
v(k )          u (n)wN ,            k  0,1, ,N  1
kn

N     n 0
N 1
1
 v(k )wN
 kn
u ( n)                           ,    n  0,1, ,N  1
N     k 0
For N  4
v(0)    1 1     1    1  u (0)
 v(1)   1 w1     2
w4   w4   u (1) 
3
      1   4                      v  Fu
v(2) 2 1 w42     0
w4   w4  u (2)
2

           3     2    1         
 v(3)   1 w4    w4   w4  u (3) 
 1 kn   1  j 2N kn 
F       wN       e     
 N         N         
 easy to handle mathematically
Fast Algorithms for Fourier
Transform
1. F 1  F *
(F* )t  F *  F 1 , since F(F* )t  I
2. Fast algorithm                                               Task for students:

Ο( N 2 )  O( N log 2 N ), where N  2 n                     Draw the butterfly for
these matrices,
2
v(0)  1 w 0 0  1 0 w4 0  u (0) 
0
4
0                   similarly as we have

 v ( 2)                                  0 
done it for Walsh and


0
  1 w 0 0  0 1 0 w4  
4
u (1) 

Reed-Muller
Transforms
 v(1)  0 0 1 w1  1 0 w4 0  u (2) 2
Pay attention to
                       3                 2        
4
regularity of kernels
 v(3)  0 0 1 w4  0 1 0 w4  u (3) 
                                            and order of columns
corresponding to
direct :16 multiplica tions , 12 additions                   factorized matrices

factorization : 4 multiplica tions , 8 additions
Fast Factorization Algorithms are
general and there is many of them

In general, if A  A1 A 2  A p : factorization
where A i , i  1,2,, p( N ) are matrices with
just a few nonzero elements, say r  N ,
then v  A x requires rpN operations.

 Fourier, Cosine, Sine transform
• 1-dim. DFT (cont.)
– Calculation of DFT : Fast Fourier Transform Algorithm
(FFT)
• Decimation-in-time algorithm
Derivation of
N 1          j 2
kn
G ( k )   g ( n )e          N                                                        decimation in time
n 0
N
2 1                 k 2 n         N
2 1              k  2 n 1
 j 2                                  j 2
    g ( 2n )e
n 0
N
  g (2n  1)e
n 0
N

           j 2   j 2 N
kn                                                  kn
2 1        j 2                   2 1
N                                   N                    k
    g ( 2n )e                         g (2n  1)e
N
2                           N
e        2

n 0                               n 0                   
 N 2 1       j 2
kn
 j 2
k N 2 1              j 2
kn

  g ( 2n )e            e              g (2n  1)e              , 0  k  N 1
N                                          N
2           N                              2

 n 0                                                                         2
  N 1               kn
n 0
k  N 2  N 2 1                  kn
 j 2                                            j 2
 g ( 2n )e                  j 2
                                            g (2n  1)e
2

e                                               ,N kN
N                                                N
2              N                                 2
 n 0
                                              n 0
2
Decimation in
Time versus
Decismation in
Frequency
• 1-dim. DFT (cont.)
– FFT (cont.)                               Butterfly for
• Decimation-in-time algorithm (cont.)   Derivation of
decimation in time

note
recursion
• 1-dim. DFT (cont.)
– FFT (cont.)
• Decimation-in-frequency algorithm (cont.)
N 1          j 2
kn
Derivation of
G ( k )   g ( n )e            N
Decimation-in-
n 0
frequency
2 1                            N 1
N                          kn                                 kn
 j 2                                 j 2
    g ( 2 n )e
n 0
N
    g ( n )e
n N 2
N       algorithm

2 1                                                  j 2 kn
N                             k N
 j 2  
   g ( n)  g ( n  2
N )e        N 2
e       N

n 0                                                     

 g (n)  g (n                                 
2 1
N                                                             kn
 j 2
                                           ) 1 e
N            k                N
2
n 0
k n
 N 2 1                       j 2
  g ( n)  g ( n  N 2 )e             , k  2k 
N
2

 n 0
  N 1                               k n
 j 2                n
                                             j 2
 g (n)  g (n  N 2 )e
2

e          , k  2k   1
N
2           N
 n 0

Decimation in frequency
• 1-dim. DFT (cont.)                butterfly shows recursion
– FFT (cont.)
• Decimation-in-frequency algorithm (cont.)
Conjugate Symmetry of
DFT
– For a real sequence, the DFT is conjugate symmetry

v* ( N  k )  v(k )
                    N                          N 
 v(0) Rev(k ) , k  1,,  1 , Imv(k ) , k  1,,  1
                    2                          2 
and v( N ) completely defines the DFT
2

 do not increase the dimensionality
Use of Fourier
Transforms
for fast
convolution
Calculations for circular matrix
Let us consider a circular matrix
 c 0 c1            c N 1 
c                             
c1  c N  2 
 N 1 c 0
C                       
                              
                       
 c1 
                   c0      

Let w *  exp( j 2N )
N


  is one of the N distinct roots of unity.
then w   * k
N

w 
* kN
N       1 , k  0,1,, N  1
    c
 k   c0  c1 w                                      w 
* k              * 2k                    N 1k
By
multiplying                                             N         c2 w  N      N 1
*
N

w   k   c w   c w     c
* k
N                 0
* k
N              1
* 2k
N             N 1   w * Nk
N

 c  c w     c                                      w 
* k                              
*    N 1k
N 1          0      N                    N 2        N

w   k   c  c w     c
* 2k
N                  N 2          N 1
* k
N              N 3   w 
*
N
N 1k



w   *
N
 N 1k
 k   c1  c2 w         
* k
N           c0 w      *
N
 N 1k
In a matrix form,
    1                                            1    
 w* k   c0 c1
                                 cN 1       * k  
     N        c                              wN     
 
 k  wN * 2k
  N 1 c0            cN 2 
 wN * 2k
 

                                                 
              c                                
c0  *  N 1k 
 

*  N 1k 
 wN          
 1 c2        
 wN
       


  (k )w k   Cw k   x  Ax 
*          *

  (k ) is an eigen value and w k , is an eigen vector of C
*

there are N distinct eigen vectors w k , thus
*

w *  Cw*
W * =
In matrix form next slide
Cw*
1 1
1 w1
1        11  1             w = Cw
*                  *
          w   2


1 w 2     w4                 
                                                                  1
where w *  1                            ; inverse Fourier transform matrix(F )
1                            
                             
                            
1                           
                             
 0  0                    0 
 0      1         0       0 
                                   
Λ       0                      ; diagonal matrix
                                   
                            
 0
        0             0   N  1


C  w  w  *
      * 1
 F 1 AF
 
or   w Cw*      * 1

 circular matrix C is diagonaliz ed by DFT
Here is the formula for linear
linear convolution :         convolution, we already discussed for
1D and 2D data, images
n 1
y n    x j hn  j 
j 0

assume:  x0, x1, x2, x3, x4 N  5
h0, h1, h2, h3       L4
M  N  L 1  8

y 0  x0h0
y 1  x0h1  x1h0
y 2  x0h2  x1h1  x2h0

Linear convolution can be presented in
matrix form as follows:

 y 0 h0      0      0      0      0 
 y 1   h1   h0    0      0      0 
                                            x0
 y 2 h2     h1 h0 0           0 
                                             x1 
 y 3  h3    h2 h1 h0        0          
                                      x2 ; y  H x
 y 4  0       h3 h2 h1       h0        
                                            x3
 y 5  0        0     h3 h2     h1 
 x4
 y 6  0                                         
0      0     h3   h2
                                          
 y 7   0
                 0      0      0     h3
8 1                    8 5                   5 1
Circular convolution                                              As we see, circular
N 1                                                     convolution can be also
y n    x j hn  j N , n  0,1,  N  1                represented in matrix form
j 0

where hn N  hnmod N 
For N  4,
 y 0  h0    h3    h2    h1   x0 
 y 1   h1   h0    h3    h2   x1 
                                            ; y  Hc x
 y 2  h2    h1    h0    h3  x2 
                                            
 y 3  h3    h2    h1    h0   x3

In general
 h0     h N  1                    h1 
 h1        h0                      h2
Hc                                               
                                      

h N  1 h N  2                      0
 h 
remember
λk   c0  c1 w      
* k
N      c2 w  
* 2k
N                  
   c N 1 w
*
N
 N 1k

Thus the first row of H c is h j , since h j  is a periodic sequence
N 1
H k    h j  exp 2N kj


j 0
  N 1
     h j  exp
j 0
 2
N     kj ; sum from 0 to   N  1 is equal to 0 to  N-1

N 1
  h j  exp j 2N kj


j 0

 H k 
 eigen value of H c is the DFT of hi 

Important
result
now,
y  Hc x       and         F 1 ΛF  H c
y  F 1 ΛFx
where
 H 0   0         0     
 0      H 1            
Λ                          
        0         0     
                          
 0       0     H  N  1

1F x ; forwardDFT of x
2 Λ F x  ; H k  X k 
Inverse DFT of
3F 1 ΛFx  ; inverse DFT of H k  X k    convolution
 fast algorithm for DFT
linear convolution                                            •Thus we derived a fast
algorithm for linear
 xi , i  0,1,, N  1
                                                             convolution which we
 hi , i  0,1, L  1
illustrated earlier and
y i   xi   hi , i  0,1,,  N  L  2
discussed its
let M  N  L  2  2 m
importance.
then
•This result is very
~ i    xi ,   i  0,1, , N  1
x        
 0 ,      i  N ,, M  1 ; zero padding          fundamental since it
allows to use DFT with
hi ,    i  0,1,, L  1
h i   
~
inverse DFT to do all
 0 ,      i  L,  , M  1    ; zero padding 
kinds of image
then perform ~  H c ~
y       x                                 processing based on
 the first  N  L  2 componentsof ~ are exactly
y
convolution, such as
equal to the  N  L  2 componentsof y
edge detection,
thinning, filtering, etc.
 fast algorithm for linear convolution filtering 
2-D DFT
2-D DFT
2  D unitary DFT is
           1 N 1 N 1
v(k , l )   u (m, n) wN wN , 0  k , l  N  1
km ln

           N m 0 n 0

u (m, n)   1 N 1 N 1



N k 0 l 0
    
v(k , l ) wNkm wNln , 0  m, n  N  1

1.V  FUF since F  F t
U  F * VF*
vector  space notation
v  Fu and u  F* v
where F  F  F
2.separable  2 N 1 - D DFT 2 NO Nlog 2 N 
3.conjugate symmetry,if u(m,n)is real
vk , l   v*  N  k , N  l , 0  k , l  N  1
4. 2  D circular convolution theorem
y m, n   hm, n  u m, n , M  N  L  1
M M        L L       N N                         Circular
hm, n , 0  m, n  N  1
h m, n   
~                                                     convolution
 0 , N  m, n  M  1                   works for 2D

~ m, n   u m, n , 0  m, n  L  1
images
u            
 0 , N  m, n  M  1                       N+L-1
Y k , l   H k , l U k , l 
~            ~         ~

          
~ m, n   DFT 1 Y k , l  , 0  m, n  M  1
y
~
N+L-1
M

M
Circular convolution works for 2D images
So we can do all kinds of edge-detection, filtering etc
very efficiently

• 2-Dim. DFT (cont.)
– example

(a) Original Image       (b) Magnitude               (c) Phase

2  dim DFT of a 512 512 Lena image
• 2-Dim. DFT (cont.)
– Properties of 2D DFT
• Separability
N 1
 1   N 1

W                  
1
F (k , l )               km
N

            f (m, n)W ,
ln
N             k , l  0,, N  1
N   m0          N   n0              

 km  1                                
N 1                N 1

 WN  N               F (k , l )W
1
f (m, n)                   
 ln
N
,
      m,n  0,, N  1
N k 0                 l 0                   
• 2-Dim. DFT (cont.)
– Properties of 2D DFT (cont.)
• Rotation
m  r cos , n  r sin  ,         k   cos , l   sin 
f (r,   0 )  F ( ,    0 )

(a) a sample image        (b) its spectrum         (c) rotated image   (d) resulting spectrum
• 2-Dim. DFT (cont.)
– Properties of 2D DFT
• Circular convolution and DFT
f (m, n)  g (m, n)   f ( p, q) g C (m  p, n  q)
p   q

f (m, n) * g (m, n)  F (k , l )G(k , l )
f (m, n) g (m, n)  F (k , l ) * G(k , l )
• Correlation
R fg (m, n)  f (m, n)  g (m, n)   f * ( p, q) g C (m  p, n  q)
p   q

f (m, n)  g (m, n)  F * (k , l )G (k , l )
f * (m, n) g (m, n)  F (k , l )  G (k , l )
• 2-Dim. DFT (cont.)                                          Three ways of
calculating 2-D
– Calculation of 2-dim. DFT                                 DFT
• Direct calculation
 2          2 
N1 1 N 2 1      j     km  j      ln       0  k  N1  1,
F (k , l )       f (m, n)e
m 0 n 0
 N 
e     N 
,
0  l  N2 1

– Complex multiplications & additions :            ( N12 N 22 )
• Using separability
                     ln   j 
 2            2 
  f (m, n)e  N  e  N  , 0  k  N1  1,
N1 1 N 2 1     j                     km
F (k , l )  

m 0  n 0
               0  l  N2 1

– Complex multiplications & additions :          ( N1 N 2 ( N1  N 2 ))
• Using 1-dim FFT
– Complex multiplications & additions : ???
Questions to Students
1.   You do not have to remember derivations but you have to understand the
main concepts.
2.   Much software for all discussed transforms and their uses is available on
internet and also in Matlab, OpenCV, and similar packages.

1. How to create an algorithm for edge detection based
on FFT?
2. How to create a thinning algorithm based on DCT?
3. How to use DST for convolution – show example.
4. Low pass filter based on Hadamard.
5. Texture recognition based on Walsh

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 18 posted: 11/9/2011 language: English pages: 67