The Story of Wavelets: Theory and Engineering Applications by wTSxt1e


                Robi Polikar
Dept. of Electrical & Computer Engineering
             Rowan University
                    The Story of Wavelets

 Technical Overview
    But…We cannot do that with Fourier Transform….
    Time - frequency representation and the STFT
    Continuous wavelet transform
    Multiresolution analysis and discrete wavelet transform (DWT)
 Application Overview
    Conventional Applications: Data compression, denoising, solution of PDEs,
     biomedical signal analysis.
    Unconventional applications
    Yes…We can do that with wavelets too…
 Historical Overview
    1807 ~ 1940s: The reign of the Fourier Transform
    1940s ~ 1970s: STFT and Subband Coding
    1980s & 1990s: The Wavelet Transform and MRA
                                    What is a Transform
                               and Why Do we Need One ?

 Transform: A mathematical operation that takes a function or sequence
  and maps it into another one
 Transforms are good things because…
    The transform of a function may give additional /hidden information
      about the original function, which may not be available /obvious
    The transform of an equation may be easier to solve than the original
      equation (recall your fond memories of Laplace transforms in DFQs)
    The transform of a function/sequence may require less storage, hence
      provide data compression / reduction
    An operation may be easier to apply on the transformed function, rather
      than the original function (recall other fond memories on convolution).
                                         December, 21, 1807

                          “An arbitrary function, continuous or with
                         discontinuities, defined in a finite interval by an
                         arbitrarily capricious graph can always be
                         expressed as a sum of sinusoids”
                                                               J.B.J. Fourier

Jean B. Joseph Fourier
 Complex function representation through simple building blocks
    Basis functions

 Using only a few blocks  Compressed representation

 Using sinusoids as building blocks  Fourier transform
    Frequency domain representation of the function
                        How Does FT Work Anyway?

 Recall that FT uses complex exponentials (sinusoids) as building

 For each frequency of complex exponential, the sinusoid at that
 frequency is compared to the signal.

 If the signal consists of that frequency, the correlation is high 
  large FT coefficients.

 If the signal does not have any spectral component at a frequency,
  the correlation at that frequency is low / zero,  small / zero FT
FT At Work
    FT At Work



    FT At Work

                                                        FT At Work

                                                         Complex exponentials
                                                          (sinusoids) as basis


An ultrasonic A-scan using 1.5 MHz transducer, sampled at 10 MHz
                    Stationary and Non-stationary
 FT identifies all spectral components present in the signal, however it
  does not provide any information regarding the temporal (time)
  localization of these components. Why?
 Stationary signals consist of spectral components that do not change in
   all spectral components exist at all times
   no need to know any time information
   FT works well for stationary signals
 However, non-stationary signals consists of time varying spectral
   How do we find out which spectral component appears when?
   FT only provides what spectral components exist , not where in time
     they are located.
   Need some other ways to determine time localization of spectral
                    Stationary and Non-stationary
 Stationary signals’ spectral characteristics do not change with time

 Non-stationary signals have time varying spectra

                       Stationary vs. Non-Stationary


Perfect knowledge of what
frequencies exist, but no
information about where
these frequencies are     X5(ω)
located in time
                      Shortcomings of the FT
 Sinusoids and exponentials
    Stretch into infinity in time,      no time localization
    Instantaneous in frequency,         perfect spectral localization
    Global analysis does not allow analysis of non-stationary signals
 Need a local analysis scheme for a time-frequency representation
  (TFR) of nonstationary signals
    Windowed F.T. or Short Time F.T. (STFT) : Segmenting the signal into
     narrow time intervals, narrow enough to be considered stationary, and
     then take the Fourier transform of each segment, Gabor 1946.
    Followed by other TFRs, which differed from each other by the
     selection of the windowing function
                  Short Time Fourier Transform

1.   Choose a window function of finite length
2.   Place the window on top of the signal at t=0
3.   Truncate the signal using this window
4.   Compute the FT of the truncated signal, save.
5.   Incrementally slide the window to the right
6.   Go to step 3, until window reaches the end of the signal
    For each time location where the window is centered, we
     obtain a different FT
      Hence, each FT provides the spectral information of a
       separate time-slice of the signal, providing simultaneous time
       and frequency information

               Time       Frequency    Signal to                 FT Kernel
             parameter    parameter   be analyzed             (basis function)

STFT of signal x(t):          Windowing             Windowing function
Computed for each              function               centered at t=t’
window centered at t=t’

t’=-8   t’=-2

t’=4     t’=8
STFT at Work
STFT At Work
STFT At Work

 STFT provides the time information by computing a different FTs for
  consecutive time intervals, and then putting them together
    Time-Frequency Representation (TFR)
    Maps 1-D time domain signals to 2-D time-frequency signals
 Consecutive time intervals of the signal are obtained by truncating the
  signal using a sliding windowing function
 How to choose the windowing function?
    What shape? Rectangular, Gaussian, Elliptic…?
    How wide?
        Wider window require less time steps  low time resolution
        Also, window should be narrow enough to make sure that the portion of
         the signal falling within the window is stationary
        Can we choose an arbitrarily narrow window…?
                          Selection of STFT Window

Two extreme cases:
 W(t) infinitely long:               STFT turns into FT, providing
   excellent frequency information (good frequency resolution), but no time
 W(t) infinitely short:

    STFT then gives the time signal back, with a phase factor. Excellent
    time information (good time resolution), but no frequency information
Wide analysis window poor time resolution, good frequency resolution
Narrow analysis windowgood time resolution, poor frequency resolution
Once the window is chosen, the resolution is set for both time and frequency.
                                        Heisenberg Principle

  Time resolution: How well              Frequency resolution: How
  two spikes in time can be              well two spectral components
  separated from each other in           can be separated from each
  the transform domain                   other in the transform domain

Both time and frequency resolutions cannot be arbitrarily high!!!
 We cannot precisely know at what time instance a frequency component is
located. We can only know what interval of frequencies are present in which time
                               The Wavelet Transform

 Overcomes the preset resolution problem of the STFT by using a
  variable length window
 Analysis windows of different lengths are used for different
   Analysis of high frequencies Use narrower windows for
    better time resolution
   Analysis of low frequencies  Use wider windows for better
    frequency resolution
 This works well, if the signal to be analyzed mainly consists of slowly
  varying characteristics with occasional short high frequency bursts.
 Heisenberg principle still holds!!!
 The function used to window the signal is called the wavelet
                                   The Wavelet Transform

Translation parameter, Scale parameter,   A normalization
                                                          Signal to be
measure of time        measure of frequency constant

Continuous wavelet transform           The mother wavelet. All kernels are
of the signal x(t) using the           obtained by translating (shifting) and/or
analysis wavelet (.)                  scaling the mother wavelet

                           Scale = 1/frequency
                                WT at Work
       Low frequency (large
High frequency (small scale) scale)
WT at Work
WT at Work
WT at Work
Matlab Demos on CWT
                       Discrete Wavelet Transform

 CWT computed by computers is really not CWT, it is a discretized
  version of the CWT.
 The resolution of the time-frequency grid can be controlled (within
  Heisenberg’s inequality), can be controlled by time and scale step
 Often this results in a very redundant representation
 How to discretize the continuous time-frequency plane, so that the
  representation is non-redundant?
    Sample the time-frequency plane on a dyadic (octave) grid
                  Discrete Wavelet Transform

 Dyadic sampling of the time –frequency plane results in a
  very efficient algorithm for computing DWT:
    Subband coding using multiresolution analysis
    Dyadic sampling and multiresolution is achieved through a
     series of filtering and up/down sampling operations

              x[n]            H            y[n]
                             Discrete Wavelet Transform


       G                                                                 G   +
                  2                                                  2

       ~          2         ~                                        2
       H                    G          2       2     G        +          H

                           ~           2       2
                           H                         H

           Decomposition                              Reconstruction

              G   Half band high pass filter       2 Down-sampling
              H   Half band low pass filter        2 Up-sampling

       2-level DWT decomposition. The decomposition can be continues as long
       as there are enough samples for down-sampling.
                                                                            DWT - Demystified

                                        Length: 512                                  |H(jw)|
                                        B: 0 ~ 

                     g[n]                  h[n]
                                                         Length: 256         -/2           /2
Length: 256             2                     2          B: 0 ~ /2 Hz
B: /2 ~  Hz
                                               a1                                              |G(jw)|
                   d1: Level 1
                     Coeff.      g[n]                 h[n]
                                                                 Length: 128
Length: 128                        2                       2     B: 0 ~  /4 Hz     -    -/2    /2    
B: /4 ~ /2 Hz                                            a2
                             d2: Level 2
                                DWT        g[n]                  h[n]
                                               2                    2    Length: 64
 Length: 64
                                                                         B: 0 ~ /8 Hz
 B: /8 ~ /4 Hz
                                           d3: Level 3            …a3….Level 3 approximation
                 Implementation of DWT on MATLAB

                                                        Choose wavelet
                                                        and number
 Load signal                                            of levels
                                                          Hit Analyze

Approx. coef.
 at level 5

Level 1 coeff.                     (Wavedemo_signal1)
Highest freq.
                       Applications of Wavelets

 Compression
 De-noising
 Feature Extraction
 Discontinuity Detection
 Distribution Estimation
 Data analysis
    Biological data
    NDE data
    Financial data

 DWT is commonly used for compression, since most
  DWT are very small, can be zeroed-out!
ECG- Compression
Denoising Implementation
               in Matlab

                             First, analyze
                             the signal with


           (Noisy Doppler)
Denoising Using Matlab

               Choose thresholding

                 Choose noise type

                 Choose thrsholds

Denosing Using Matlab
Discontinuity Detection

Discontinuity Detection
              with CWT

                                      Application Overview

 Data Compression
 Wavelet Shrinkage Denoising
 Source and Channel Coding
 Biomedical Engineering
    EEG, ECG, EMG, etc analysis
    MRI
 Nondestructive Evaluation
    Ultrasonic data analysis for nuclear power plant pipe inspections
    Eddy current analysis for gas pipeline inspections
 Numerical Solution of PDEs
 Study of Distant Universes
    Galaxies form hierarchical structures at different scales
                                        Application Overview

 Wavelet Networks
    Real time learning of unknown functions
    Learning from sparse data
 Turbulence Analysis
    Analysis of turbulent flow of low viscosity fluids flowing at high speeds
 Topographic Data Analysis
    Analysis of geo-topographic data for reconnaissance / object identification
 Fractals
    Daubechies wavelets: Perfect fit for analyzing fractals
 Financial Analysis
    Time series analysis for stock market predictions
                           History Repeats Itself…

 1807, J.B. Fourier:
    All periodic functions can be expressed as a weighted sum of
     trigonometric function
    Denied publication by Lagrange, Legendre and Laplace
    1822: Fourier’s work is finally published
   …                       143 years
    1965, Cooley & Tukey: Fast Fourier Transform
                                  History Repeats Itself:
                                          Morlet’s Story

 1946, Gabor: STFT analysis:
    high frequency components using a narrow window, or
    low frequency components using a wide window, but not both
 Late 1970s, Morlet’s (geophysical engineer) problem:
    Time - frequency analysis of signals with high frequency components for short
      time spans and low frequency components with long time spans
    STFT can do one or the other, but not both Solution: Use different
      windowing functions for sections of the signal with different frequency content
    Windows to be generated from dilation / compression of prototype small,
      oscillatory signals  wavelets
 Criticism for lack of mathematical rigor !!!
 Early 1980s, Grossman (theoretical physicist): Formalize the transform and
  devise the inverse transformation  First wavelet transform !
 Rediscovery of Alberto Calderon’s 1964 work on harmonic analysis

 1984, Yeves Meyer :
    Similarity between Morlet’s and Colderon’s work, 1984
    Redundancy in Morlet’s choice of basis functions
    1985, Orthogonal wavelet basis functions with better time and
     frequency localization
 Rediscovery of J.O. Stromberg’s 1980 work the same basis
  functions (also a harmonic analyst)
 Yet re-rediscovery of Alfred Haar’s work on orthogonal
  basis functions, 1909 (!).
    Simplest known orthonormal wavelets
                Transition to the
         Discrete Signal Analysis

Ingrid Daubechies:
     Discretization of time and scale parameters
       of the wavelet transform
     Wavelet frames, 1986
     Orthonormal bases of compactly supported
       wavelets (Daubechies wavelets), 1988
     Liberty in the choice of basis functions at
       the expense of redundancy

Stephane Mallat:
     Multiresolution analysis w/ Meyer, 1986
          Ph.D. dissertation, 1988
     Discrete wavelet transform
     Cascade algorithm for computing DWT

 Decomposition of a discrete into dyadic frequencies (MRA) , known to
  EEs under the name of “Quadrature Mirror Filters”, Croisier, Esteban and
  Galand, 1976 (!)
Transition to the Discrete Signal

     Martin Vetterli & Jelena Kovacevic
         Wavelets and filter banks, 1986
         Perfect reconstruction of signals
            using FIR filter banks, 1988
         Subband coding
         Multidimensional filter banks, 1992

 Equivalence of QMF and MRA, Albert Cohen, 1990
 Compactly supported biorthogonal wavelets, Cohen, Daubechies, J.
  Feauveau, 1993
 Wavelet packets, Coifman, Meyer, and Wickerhauser, 1996
 Zero Tree Coding, Schapiro 1993 ~ 1999
 Search for new wavelets with better time and frequency localization
 Super-wavelets
 Matching Pursuit, Mallat, 1993 ~ 1999
                                     New & Noteworthy

 Zero crossing representation
    signal classification
    computer vision
    data compression
    denoising
 Super wavelet
    Linear combination of known basic wavelets
 Zero Tree Coding, Schapiro
 Matching Pursuit , Mallat
    Using a library of basis functions for decomposition
 New MPEG standard

To top