Preprocessing of NIR

Document Sample
Preprocessing of NIR Powered By Docstoc
					     Preprocessing
       With focus on NIR

Åsmund Rinnan , Frans van den Berg,
Rasmus Bro, Søren Balling Engelsen,
  Lars Nørgaard , Jonas Thygesen
                                   Introduction



               Introduction
                                   Techniques
                                   Summary




•   How often is it used?
•   When is it used?
•   What methods are used?
•   Why is it used?

• NIR is a measurement technique
  “screaming” for preprocessing
                   Introduction
                                  Introduction
                                  Techniques
                                  Summary




                        NIR


Fructose Glucose
    0      100
   50       50
   75       25
                 Introduction
                                                Introduction
                                                Techniques
                                                Summary




                  Techniques
• Correction of light   •   Baseline-correction
  scatter               •   Normalization
   – MSC/ ISC           •   Detrend
   – PMSC
                        •   Use of reference values
   – EMSC/ EISC
                            –   O-PLS
• SNV                       –   OSC
• Derivation                –   OS
   – Norris&Williams        –   SIS
   – Savitsky&Golay
                    Introduction
                                                  Introduction
                                                  Techniques
                                                  Summary




                         Why

                   Baseline with a slope/ curve

   Nonlinearity




A “fat” baseline
                                                                                                                                                    Introduction




                                                           ISC
                                                           MSC
                                                                                                                                                    Techniques
                                                                                                                                                    Summary




                                                                                                                         • Offset
                 Raw spectrum                                                                                            • Normalization




                                                           Reference

1.   P Geladi, D MacDougal, H Martens (1985): Linearization and scatter correction for near-infrared reflectance spectra of meat, Applied Spectroscopy, 39, 491-500
                    Introduction




MSC/ ISC
                    Techniques
                    Summary




           • Offset
           • Normalization
                                                                                                                                                           Introduction




                                                                    EISC
                                                                                                                                                           Techniques
                                                                                                                                                           Summary




                                                                                                                                 • Offset
                                                                                                                                 • Baseline
                                                                                                                                 • Normalization
x new  f x org   f λ   f x known   e

f xorg   a  1  b  xorg  c  xorg
                                                                   2




f λ   a  λ  b  λ  c  log λ     2



f x known   h T  G
        1.   H. Martens and E. Stark (1991): Extended multiplicative signal correction and spectral interference subtraction: new preprocessing methods for near infrared
             spectroscopy, Journal of Pharmaceutical and Biomedicinal Analysis, 9, 625-635
        2.   H Martens, JP Nielsen, SB Engelsen (2003): Light scattering and light absorbance separated by extended multiplicative signal correction. The application to
             near-infrared transmission analysis of powder mixtures, Analytical chemistry, 75, 394-404
        3.   DK Pedersen, H Martens, JP Nielsen, SB Engelsen (2002): Near-Infrared Absorption and Scattering Separated by Extended Inverse Signal Correction (EISC):
             Analysis of Near-Infrared Transmittance Spectra of Single Wheat, Applied Spectroscopy, 56, 1206-1214
        4.   Thennadil, SN, Martin, EB (2005): Empirical preprocessing methods and their impact on NIR calibrations - a simulation study, Journal of Chemometrics, 19, 77-89
           EMSC
                              Introduction

                              Techniques
                              Summary




(w/ 2. degree & wavelength)
                      • Offset
                      • Baseline
                      • Normalization
                                                                                                                                                               Introduction




                                                                    PxSC
                                                                                                                                                               Techniques
                                                                                                                                                               Summary




                                                                                                                                     • Offset
X = EM
    EI                                                                                                                               • Baseline
    M                                                                                                                                • Normalization
    I




         1.   Isaksson, T.; Kowalski, B. R. (1993): Piece-wise multiplicative scatter correction applied to Near-Infrared diffuse transmittance data from meat products, Applied
              Spectroscopy, 47, 702-09
      PMSC
                              Introduction

                              Techniques
                              Summary




(window size = 35)
                     • Offset
                     • Normalization
                                                                                                                                                         Introduction




                                                                   SNV
                                                                                                                                                         Techniques
                                                                                                                                                         Summary




                                                                                                                              • Offset
           x org  xorg                                                                                                       • Normalization
x new 
                    x




      1.   R.J. Barnes, M.S. Dhanoa, S.J. Lister (1989): Standard Normal Variate Transformation and De-trending of Near-Infrared Diffuse Reflectance Spectra, Applied
           Spectroscopy, 43, 772-777
                                   Introduction




      SNV vs MSC
                                   Techniques
                                   Summary




                 x org  a
       x new 
                    b
MSC                          SNV
                                                                                                                                                        Introduction




                                                       Detrend
                                                                                                                                                        Techniques
                                                                                                                                                        Summary




                                 n                                                                                           • Offset
xorg  xnew   ai  i 1                                                                                                 • Baseline
                               i 1




     1.   R.J. Barnes, M.S. Dhanoa, S.J. Lister (1989): Standard Normal Variate Transformation and De-trending of Near-Infrared Diffuse Reflectance Spectra, Applied
          Spectroscopy, 43, 772-777
                                    Derivation
                                                                                                                                                  Introduction

                                                                                                                                                  Techniques
                                                                                                                                                  Summary




                                  Savitsky-Golay
                                                                                                                       • Offset
                          xnew, f  x org,f m, f m  o                                                               • Baseline
                                                                               T


                               CT
                         o T        d!
                            C C
                              m                                       ...            m             i

                                                                                              i
                              m  1                                 ...          m  1 
                         C   ...                                       0                      ...               
                                                                                                                 
                              m 1                                     ...             m  1              i
                                                                                                                  
                              m                                                               mi                 
                                                                       ...                                       
1.   A Savitsky, MJE Golay (1964): Smoothing and differentiation of data by simplified least squares procedures, Analytical Chemistry, 36 (8), 1627-1639
2.   P.A. Gorry (1990): General Least-Squares smoothing and differentiation by the convolution (Savitsky-Golay) method, Analytical Chemistry, 62, 570-573
  Derivation
                              Introduction

                              Techniques
                              Summary




Savitsky-Golay
                 • Offset
                 • Baseline
  Derivation
                              Introduction

                              Techniques
                              Summary




Savitsky-Golay
                 • Offset
                 • Baseline
                                                   Derivation
                                                                                                                                                                  Introduction

                                                                                                                                                                  Techniques
                                                                                                                                                                  Summary




                                                 Norris-Williams
                                                                                                                                        • Offset
                                        xnew, f                                                                                         • Baseline
optional : xnew, f                 n                                                                                                   • Normalization
                                  x
                                   i 1
                                             new, f , i


                                                                 xorg , f  2  xorg , f 1
xnew, f ,1st  xsmooth, f 1  xsmooth, f 
                                                                                  3
                                                                                                   xorg , f  2  xorg , f 1  xorg , f 1  xorg , f  2
xnew, f , 2 nd  xsmooth, f 1  xsmooth, f 1  2  xsmooth, f 
                                                                                                                                          3
                   1

                 x       org , f  i
xsmooth, f      i  1
                          3


            1.     Norris, KH (1983): in Food Research and Data Analysis (Eds: Martens H and Russwurm H Jr), Applied Science, London, 95-114
            2.     Norris KH, Williams PC (1984): Optimization of mathematical treatments of raw near-infrared signal in the measurement of protein in hard red spring wheat: I.
                   Influence of particle size, Cereal Chemistry, 61, 158-165
  Derivation
                          Introduction

                          Techniques
                          Summary




Norris-Williams
                  • Offset
                  • Baseline
                  • Normalization
  Derivation
                          Introduction

                          Techniques
                          Summary




Norris-Williams
                  • Offset
                  • Baseline
                  • Normalization
                                           Introduction




            Preprocessing
                                           Techniques

                                           Summary




• PMSC: Sensitivite to window size
• EMSC: Small effect on baseline  Detrend
  is better
• MSC and SNV are practical the same
• Savitsky-Golay is more robust than Norris-
  Williams
                                                   Introduction




               Preprocessing
                                                   Techniques

                                                   Summary




                  Offset Baselinje Normalization
(P)MSC/ISC/SNV      x                   x
(P)EMSC/EISC        x                   x
Detrend             x       x
Deriveration        x       x           x
Thanks for the attention

www.models.life.ku.dk

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:13
posted:3/1/2012
language:English
pages:23