sbe

Document Sample
sbe Powered By Docstoc
					  Successive Bayesian Estimation
             Alexey Pomerantsev
           Semenov Institute of Chemical Physics
              Russian Chemometrics Society




23.02.03                                           1
Agenda

           1. Introduction. Bayes Theorem
           2. Successive Bayesian Estimation
           3. Fitter Add-In
           4. Spectral Kinetics Example
           5. New Idea (Method ?)
           6. More Applications of SBE
           7. Conclusions

23.02.03                                       2
1. Introduction




23.02.03          3
The Bayes Theorem, 1763
                  Posterior Probability   Prior Probabilities



           Where to take
             the prior
  Thomas Bayes


           probabilities?
   (1702-1761)


                      Likelihood Function

                 L(a,s 2)=h(a,s 2)L0(a,s 2)
23.02.03                                                        4
Jam Sampling & Blending Theory


            Now we know
             the origin of
           0.20  0.30 0.50

           0.50
                a worm in
                 0.20 0.05


                 the jam!
23.02.03                         5
2.Successive Bayesian Estimation (SBE)




23.02.03                                 6
SBE Concept
     SBE principles    Whole data set
                       Data subset 1 Data subset 2                                      Data subset k
1)    Split up


       How to eat away
                        y1         X1            y2         X2                 ...       yk         Xk
      whole data set
                       f 1 (X 1 , a 0 , a 1 )   f 2 (X 2 , a 0 , a 2 )        ...       f k (X k , a 0 , a k )
2)    Process each


         an elephant?
      subset alone
                                  Post          Prior        Post                       Prior       Post
3)    Make posterior              a0, a1         a0          a0, a2           ...        a0         a0, ak
                                      2            2            2
      information                 s1 N1 s1 N1 s2 N2                                     sk 2Nk sk 2 Nk

4)
        Slice by slice!
      Build prior
      information                                                       Result
                                                                    a 0 , a 1 ,…, a k
5)    Use it for the                                                     s2 N
      next subset


23.02.03                                                                                                   7
OLS & SBE Methods for Two Subsets
    OLS


             Quadratic
           approximation
             near the
    SBE




             minimum!
23.02.03                            8
Posterior & Prior Information
           Subset 1. Posterior Information




           Make Posterior,
             Rebuilding (common & partial parameters)


           rebuild it and
           apply as Prior!
           Subset 2. Prior Information



23.02.03                                                9
Prior Information of Type I
            Posterior Information            Prior Information



           The same error
           Parameter estimates          Prior parameter values b

           Matrix A                     Recalculated matrix H

           variance in the
                 s        s
           Variance estimate    2       Prior variance value   0
                                                                   2




            each subset
           NDF Nf                       Prior NDF N0



               of data!
                               Objective Function




23.02.03                                                               10
Prior Information of Type II
            Posterior Information         Prior Information


            Different error
           Parameter estimates   ˆ
                                 a   Prior parameter values b



            variances in the
              A
           Matrix          H         Recalculated matrix

                            Objective Function


               each subset
                 of data!
23.02.03                                                        11
SBE Main Theorem
     Different order of subsets processing


                    SBE
                 agree with
                    OLS!
     Theorem (Pomerantsev & Maksimova , 1995)




23.02.03                                        12
3. Fitter Add-In




23.02.03           13
Fitter Workspace
         A     B            C            D        E            F        G         H          I      J       K      L         M       N        O          P            Q       R   S   T
     1
    2        Data                                                                                       General
                                                                                                        y=exp(p)*A+exp(q)*B+exp(r)*C
    3          x            t            y        w            f        A         B          C          Date       01.08.01 19:07
                                                                                                        A=A0*exp(-k1*t)
    4          13                0   0.047            1       0.047        1         0          0       Data    Bayes!rData
                                                                                                        B=k1*A0/(k1-k2)*[exp(-k2*t)-exp(-k1*t)]+B0*exp(-k2*t)
    5          13                2   0.553            1        0.56    0.125     0.448     0.4266       Model Bayes!ABCbayes
                                                                                                        C=A0+B0+C0+A0/(k1-k2)*[k2*exp(-k1*t)-k1*exp(-k2*t)]-B0*exp(-k2*t)
    6          13                4   0.412            1       0.403    0.016     0.209      0.775       Param eters
                                                                                                                Bayes!rParam
                                                                                                                A0="cA0"




                                      Fitter is a
    7          13                6   0.304            1       0.308    0.002     0.079     0.9194       Bayes Bayes!rBayes
                                                                                                                B0="cB0"
    8          13                8   0.275            1        0.27    2E-04     0.028      0.972       Precision
                                                                                                                1E-11
                                                                                                                C0="cC0"
    9          13               10   0.253            1       0.257    3E-05      0.01     0.9904       Convergence
                                                                                                                0.001 k1=?
   10                                                                                                           Relative
                                                                                                        Error type k2=?
   11        Bayesian Inform ation                                                                              0.05
                                                                                                        Significance
                                                                                                                      p=?
   12        Nam e Value                Matrix                                            Exclude               0.95
                                                                                                        Confidence
                                                                                                                      q=?
   13        k1      1.07 265.3 146.5        0                              0         0                         Linearization
                                                                                                        Prediction




                                     tool for SBE!
                                                                                                                       r=?
   14        k2     0.554 146.5    1117      0                              0         0
   15                    0      0     0      0                              0         0                 Param eters estim ation
   16                    0      0     0      0                              0         0                 Nam e Initial    Final    Deviation
   17                    0      0     0      0                              0         0                 k1      1.06969 1.03907 0.060509
   18                                                                                                   k2      0.55366 0.53661 0.027954
   19                                                                                                   p      -3.05769 -3.05769 0.019371
                    y                        x = 13
   20                                                                                                   q         -0.002    -0.002 0.039994
   21
               1.0                                                                                      r      -1.38757 -1.38757 0.017332
                                A                 C
   22
   23                                                                                                   Param eters                               Search Progress
   24                                                                                                   k1      1.03907                           Objective value    0.0139
               0.5
   25                                             y                                                     k2      0.53661                           Com pleteness       100%
   26                                                                                                   p       -3.05769                          Objective change   -1E-07
   27                                             B                                                     q          -0.002                         Iteration               2
   28          0.0                                                                                      r       -1.38757
   29
                        0            2        4           6        8        10    t
   30




23.02.03                                                                                                                                                                              14
Data & Model Prepared for Fitter

                                                                         Values
              Predictor       Response                                                Comment
                                                   Weight

      A   B    C   D      E   F     G         H              I           J    K        L         M
  1



                          Apply Fitter!
  2       BoxBod Data             Parameters                     y
  3        x   y w     f          a     100
                                        213.80941           200
  4         0     0   0.00        b     0.5472375
                                        0.4
  5
  5         1 109 1 90.11
  6
  6         2 149 1 142.24      'BoxBOD model               100
  7
  7         3 149 1 172.41    y=a*[1-exp(-b*x)]
  8
  8         5 191 1 199.95          a=?
  9         7 213 1 209.17          b=?                          0
 10
 10       10 224 1 212.91                                            0            4        8     x
 11
 11

                                         Fitting                                      Equation
                                                                 Parameters
23.02.03                                                                                             15
                                     'Цикл "увлажнение-сушка"



Model f(x,a)
                      M=Sor*hev(t1-t)+Des*[hev(t-t1)+imp(t-t1)]
                      'Кинетика "увлажнения"
                          Sor=Sor1*hev(USESor1)+Sor2*[hev(-USESor1)+imp(-USESor1)]
                      'Кинетика "сушки"
                          Des=Des1*hev(USEDes1)+Des2*[hev(-USEDes1)+imp(-USEDes1)]
                      'Условие применимости асимптотик
                          USESor1=Sor2-Sor1


                 Different shapes of the same model
                          USEDes1=Des1-Des2
                      'константы и промежуточные величины
                          t3=(t-t1)*hev(t-t1)
                          t4=t*hev(t1-t)+t1*[hev(t-t1)+imp(t-t1)]
                          P2=PI*PI
                          P12=(PI)^(-0.5)


                                    y = a + (b – a)*exp(–c*x)
                          R=r*(M1-M0)*exp(-r*t4)
           Explicit model K=M1+(M0-M1)*exp(-r*t4)




                         Rather
                          V0=M0-C0
                          V1=M1-C0
                      'асимптотика сорбции при 0<t<tau
                          Sor1=C0+4*P12*(d*t)^0.5*[M0-C0+(M1-M0)*beta]


                                    0 = a + (b – a)*exp(–c*x) – y
                             beta=1-exp(-z)

           Implicit model    x=r*t
                             z=(a1*x+a2*x*x+a3*x*x*x)/(1+b1*x+b2*x*x+b3*x*x*x)
                                a1=0.6666539250029
                                a2=0.0121051017749




                       complex
                                a3=0.0099225322428


                                    d[y]/d[x] = – c*(y –a); y(0) = b
                                b1=0.0848006232519

           Diff. equation       b2=0.0246634591223
                                b3=0.0017549947958
                      'кинетика сорбции при tau<t<t1
                         Sor2=K-8*S1
                             S1=U01/n0+U11/n1+U21/n2+U31/n3+U41/n4
                                n0=P2

                      Presentation at worksheet

                          model!
                                U01=[(V0*n0*d-V1*r)*exp(-n0*d*t4)+R]/(n0*d-r)
                                n1=P2*9
                                U11=[(V0*n1*d-V1*r)*exp(-n1*d*t4)+R]/(n1*d-r)
                                n2=P2*25
                                U21=[(V0*n2*d-V1*r)*exp(-n2*d*t4)+R]/(n2*d-r)
                                n3=P2*49
                                U31=[(V0*n3*d-V1*r)*exp(-n3*d*t4)+R]/(n3*d-r)
                                n4=P2*81
                                U41=[(V0*n4*d-V1*r)*exp(-n4*d*t4)+R]/(n4*d-r)
                      'асимптотика десорбции при t1<t<t1+tau
                          Des1=K*[1-4*P12*(d*t3)^0.5]-8*S1
                      'кинетика десорбции при t1+tau<t
                          Des2=8*S2
                             S2=U02/n0+U12/n1+U22/n2+U32/n3+U42/n4
                                U02=(K-U01)*exp(-n0*d*t3)
                                U12=(K-U11)*exp(-n1*d*t3)
                                U22=(K-U21)*exp(-n2*d*t3)
                                U32=(K-U31)*exp(-n3*d*t3)
                                U42=(K-U41)*exp(-n4*d*t3)
                      'неизвестные параметры
                                   d=?
                                   M0=?
                                   M1=?
                                   C0=?
                                   r=?
23.02.03                           t1=?
                                                                                     16
4. Spectral Kinetics Modeling




           1
               7
                   13
                        19
                             25
                                  31
                                       37
                                            43                      8   10
                                                            4   6
                                                 49 0   2



23.02.03                                                                     17
Spectral Kinetic Data
                        Y(t,x,k)=C(t,k)P(x)+E
               This is large
   spectral signal          conc.                pure spectra             errors


              Y non-linear E
          wavelengths        species               wavelengths            wavelengths




                                       species
                            time
   time




                 = C  P  +




                                                                   time
                regression
              K constants    L wavelengths           M species   N time points




                  problem!
           Y is the (NL) known data matrix
           C is the (NM) known matrix depending on unknown parameters k
           P is the (ML) unknown matrix of pure component spectra
           E is the (NL) unknown error matrix

23.02.03                                                                                18
How to Find Parameters k?
   Method             Idea                Dimension     Problem

  Full OLS                                               Large
                                         K+ML >> 1
   (hard)                                              dimension


 Short OLS
   (hard)
              This is a                  K+MS  10
                                                          Small
                                                        precision


   WCR
(hard&soft)
              challenge!                    K  10
                                                         Matrix
                                                       degradation


                 1  e  kt          
   GRAM       k  ln  k (t  s )      K+MA  100
                                                        Just one
   (soft)        s e
                                     
                                                        model


23.02.03                                                           19
Simulated Example Goals


     Compare SBE estimates with ‘true’ values



     Compare SBE estimates for different order



     Compare SBE estimates with OLS estimates




23.02.03                                         20
Model. Two Step Kinetics
                 k1    k2
              A  B  C

           Standard
           dA
           dt
               k1 A;       A(0)  A0  1


           ‘training’
           dB
           dt
               k1 A  k2 B; B(0)  B0  0


             model
           dC
               k2 B;        C(0)  C0  0
           dt

            ‘True’ parameter values
                k1=1          k2=0.5

23.02.03                                     21
Data Simulation
                  Simulated concentration profiles                   Simulated pure component spectra

                  1.0                                                           1.0
                                               C                                                        B                   C
                            A                                                             A




                                 Usual way of
                  0.8                                                           0.8
 concentrations




                                                              spectral signal
                  0.6                                                           0.6

                  0.4               B                                           0.4




                                data simulation
                  0.2                                                           0.2

                  0.0                                                           0.0
                        0       2   4          6     8   10                           0   5   10   15   20   25   30   35       40   45   50
                                        time                                                  conventional wavelengths


                   C1(t) = [A](t)                                                                                 P1(x) = pA (x)
                                                   Y(t,x)=C(t)P(x)(I+E)
                   C2(t) = [B](t)                                                                                 P2(x) = pB (x)
                   C3(t) = [C](t)                    STDEV(E)=0.03                                                P3(x) = pC (x)

23.02.03                                                                                                                                       22
Simulated Data. Spectral View

                             1.0       t=0                              t=10
                             0.9                                         t=8
                                                                         t=6




                               Spectral
                             0.8

                             0.7
           spectral signal




                             0.6                        t=2




                             view of data
                             0.5
                                                  t=4
                             0.4

                             0.3

                             0.2

                             0.1

                             0.0
                                   0         10               20   30          40   50
                                                    conventional wavelengths



23.02.03                                                                                 23
Simulated Data. Kinetic View


                              1




                             Kinetic view
                             0.8
           spectral signal




                             0.6




                               of data
                             0.4



                             0.2



                              0
                                   0   2   4      6   8   10
                                               time



23.02.03                                                       24
One Wavelength Estimates
           Conventional wavelength 3              Conventional wavelength 14
           y                                          y
 1.0                                        1.0

 0.8                                        0.8

 0.6                                        0.6

 0.4                                        0.4

 0.2                                        0.2                                               14
                                       3




                       Bad accuracy!
 0.0                                        0.0
       0       2   4    6    8    10 time         0           2   4     6      8        10 time

       Conventional wavelength 51                                 Estimates

           y                                4.0
 1.0

 0.8                                        3.0

 0.6
                                            2.0
 0.4
                                                                                   k1
 0.2                                        1.0
                                       51                                          k2
 0.0
                                            0.0
       0       2   4    6    8    10 time                 3        14         51          O
23.02.03                                                                                           25
Four Wavelengths Estimates
                    Direct order                                       Inverse order
           y                                               y
 1.0                                             1.0

 0.8                                             0.8




                    Bad accuracy,
 0.6                                             0.6
                                           4                                                      50
 0.4                                             0.4
                                           3                                                      51
 0.2                                       2     0.2                                              52
                                           1                                                      53
 0.0                                             0.0




                        again!
       0       2     4     6       8   10 time         0           2     4       6     8        10 time

                   Random order                                         Estimates
           y                                     1.5
 1.0

 0.8                                                                                       k1
                                                 1.0
 0.6                                      29
                                           8                                               k2
 0.4
                                                 0.5
                                          16
 0.2
                                           5
 0.0                                             0.0
       0       2     4     6       8   10 time                 D             I       R           O
23.02.03                                                                                               26
SBE Estimates at the Different Order
                   Direct 1, 2, 3, ….                                    Inverse 53, 52, 51, ….
    1.25
    1.50
                        k1                             Inverse
                                                           1.50
                                                                          k1



     SBE (practically)
    1.25                                                   1.25
                        k1
    1.00                                                   1.00

    1.15
    0.75
                    Direct        k2
                                                           0.75
                                                                                                    k2
    0.50                                                   0.50




   doesn’t depend on
    0.25                                                   0.25
           1   8    15       22    29   36   43   50              53 49 45 41 37 33 29 25 21 17 13 9 5 1
   1.05        conventional wavelengths                                  conventional wavelengths




   the subsets order!
               Random 16, 5, 29, ….                                  0.95 Confidence Ellipses
    1.50        'True'                                     1.25     k1           Inverse
   0.95
    1.25
                   k1
                                                           1.15                                Direct
    1.00
        Random                     k2
    0.75                                                   1.05
                                                                                                         k2
    0.85
    0.50
                                                           0.95
                                                                      Random
                                                                                           'True'
    0.25
                                                                                                         k2
             0.42 0.44 0.46 0.48
         16 41 27 33 19 2 15 51 21 9 24 50 12 22
               conventional wavelengths
                                                             0.5
                                                           0.85                   0.54 0.56
                                                                            0.52 0.5 0.52 0.540.56
                                                               0.42 0.44 0.46 0.48

23.02.03                                                                                                      27
SBE Estimates and OLS Estimates
           1.25
                  k1
           1.20




            SBE estimates
           1.15

           1.10




             are close to
           1.05
                  OLS
           1.00                                    'True'



            OLS estimates!
           0.95
                               SBE
           0.90
                                                                   k2
           0.85
               0.42     0.44   0.46   0.48   0.5     0.52   0.54   0.56




23.02.03                                                                  28
Pure Spectra Estimating
                             1.2       Spectrum A
                                                B
                                                C                              0.6

                              1


                               SBE gives
                                                                               0.4
                                                                
           spectral signal
           spectral signal




                             0.8




                                                                                      accuracy
                                                                                      accuracy
                                       

                              good spectra
                             0.6                                              0.2

                                                                 
                                                                 
                             0.4


                               estimates!
                                       
                                                                              0
                                                
                             0.2
                             0.2                                
                                                                 
                              0
                              0                                                -0.2
                                                                               -0.2
                                   1
                                   1       11
                                           11        21
                                                     21      31
                                                             31      41
                                                                     41   51
                                                                          51
                                                conventional wavelength
                                                conventional wavelength
23.02.03                                                                                         29
Real World Example Goals

     Apply SBE for real world data



     Compare SBE with other known methods




23.02.03                                    30
Data
     Bijlsma S, Smilde AK. J.Chemometrics 2000; 14: 541-560
          8
     Epoxidation of 2,5-di-tert-butyl-1,4-benzoquinone
        6


                         Preprocessed
     SW-NIR spectra
   spectral signal




                     4
                     2


                             Data
                                              8
                     0
                                              6
        -2
                            spectral signal
     240 spectra                              4
                                              2
        -4
     1200 time points                         0
     21 wavelengths
        -6                                    -2
                                              -4
     Preprocessing:
        -8                                    -6
     Savitzky-Golay filter865
           860                                -8        870          875         880
                                                   860    865    870       875   880
                                                     wavelength
                                                              wavelength

23.02.03                                                                               31
Progress in SBE Estimates
           0.4



                   SBE works
                                            k1
           0.3




                  with the real
           0.2

                                             k2


                   world data!
           0.1


           0.0
                 860 862 864 866 868 870 872 874 876 878 880
                               wavelength (nm)

23.02.03                                                       32
SBE and the Other Methods
           0.40    k1                          GRAM


         SBE gives the
           0.35




       lowest deviations
           0.30                                  SBE

           0.25


        and correlation!
           0.20

           0.15
                        WCR

                               LM-PAR                         k2
           0.10
               -0.05    0.00    0.05    0.10   0.15    0.20   0.25

23.02.03                                                             33
5. New Idea




23.02.03      34
Bayesian Step Wise Regression
Ordinarily Step Wise Regression      Bayesian Step Wise Regression

                 y=a1x1+a2x2+a3x3
           BSWR accounts
            correlations of
           variables in step
            wise estimation
                Objective function




23.02.03                                                        35
BSW Regression & Ridge Regression


      BSWR is RR with
       a moving center
      and non-Euclidean
           metric
23.02.03                            36
Example. RMSEC & RMSEP



       BSWR gives
    typical U-shape of
     the RMSEP curve

23.02.03                 37
Linear Model. RMSEC & RMSEP


                 BSWR is not
                 y=a1x1+a2x2+a3x3+a4x4+a5x5


                 worse then
           0.5
                         RMSEC
           0.4
                         RMSEP



                 PLS or PCR
           0.3




                  and better
           0.2


           0.1




                  then SWR
           0.0

                   PLS   PCR     OLS   SWR   BSWR

23.02.03                                            38
Non-Linear Model. RMSEC & RMSEP

      y  a1e k1x1  a2 e k2 x2  a3 e k3 x3  a4 e k4 x4  a5 e k5 x5

           1.2
                 For non-linear
                     1.2
                                      RMSEC
                                    RMSEC


                 model BSWR is
           1.0       1.0
                                      RMSEP
                                    RMSEP
           0.8       0.8




                  better then
           0.6       0.6

           0.4       0.4




                  PLS or PCR
           0.2       0.2

           0.0       0.0

                 PLS       PCR
                            PLS     OLS
                                    PCR       SWR BSWR
                                               OLS SWR           BSWR

23.02.03                                                                 39
Variable selection


           BSWR is just
            an idea, not
           the method so
           any criticism is
            welcomed now!
23.02.03                      40
6. More Practical Applications of SBE




23.02.03                                41
   Antioxidants Activity by DSC
                               DSC Data                                      Oxidation Initial Temperature (OIT)
             4                  2                                          570                                    C=0.1
             3                                 5                                                                  C=0.05
             2                                                             550




                                To test
                                                    10
DSC signal




             1                                           15




                                                                OIT T ,K
                                                                           530
             0
             -1                                           20                                                      C=0.025
                                                                           510
             -2




                              antioxidants!
             -3                                                            490
             -4
             -5                                                            470
                  460   470     480     490        500    510                    0    5        10        15        20
                              Temperature, K                                          Heating rate v , grad/min




   23.02.03                                                                                                             42
 Network Density of Shrinkable PE by TMA
                                      TMA Data                                                             Network density

                                                                                                  14




                                        To solve
                  1.5                            1                                                                               A




                                                                         Chemical modulus, gmm2
                                                                     2                            12                                      B
Elongation L/Lo




                  1.4                                                                             10
                                                                                                   8                                      C




                                      technological
                  1.3
                                                     3         4                                   6

                  1.2                                                5                             4
                                                                                                   2
                  1.1                                                                                          D1      D2   D3




                                        problem!
                                                                                                   0
                        0   10   20   30   40   50   60   70   80   90
                                                                                                       0   5         10    15        20       25
                                        Time, min                                                                   Dose, MRad




 23.02.03                                                                                                                                     43
       PVC Isolation Service Life by TGA
                                 TGA Data                                           Service life prediction

                 1.00                                     490

                                               




                                                                Temperature T , K
Mass chamge, y




                 0.98




                                      To predict
                                                          450
                 0.96

                 0.94
                                                          410




                                      durability!
                 0.92

                 0.90                                     370
                        0   10   20     30      40   50
                                 Time t , min




       23.02.03                                                                                               44
                Tire Rubber Storage
                                  Elongation at break                                                Tensile strength

                                              Time, yr                                                      Time, yr
                         0             15             26 30         45                        0        15      23       30          45
                     6                                                                   30
Elongation @ break




                     5                                                                   25
                                                                                                      T=20 C




                                                                          Tensile, KPa
                     4                      T=20 C                                       20                                   Critical
                                                               Critical



                                                 To predict
                                                                                                                               level
                     3                                          level                    15
                     2                                                                   10
                     1                                                                    5
                             T=140 C        T=125 C           T=110 C                             T=140 C T=125 C            T=110 C




                                                 reliability!
                     0                                                                    0
                         0             20                40      60                           0       20               40        60
                                              Time, hr                                                      Time, hr




                23.02.03                                                                                                          45
7. Conclusions

      1    SBE is of general nature and it can be used for any model




            Thanks!
      2    SBE agrees with OLS


      3    SBE gives small deviations and correlations


      4    SBE uses no subjective a priori information

      5    SBE may be useful for non-linear modeling (BWSR?)




23.02.03                                                               46

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:9
posted:11/8/2011
language:English
pages:46