# sbe

Document Sample

```					  Successive Bayesian Estimation
Alexey Pomerantsev
Semenov Institute of Chemical Physics
Russian Chemometrics Society

23.02.03                                           1
Agenda

1. Introduction. Bayes Theorem
2. Successive Bayesian Estimation
4. Spectral Kinetics Example
5. New Idea (Method ?)
6. More Applications of SBE
7. Conclusions

23.02.03                                       2
1. Introduction

23.02.03          3
The Bayes Theorem, 1763
Posterior Probability   Prior Probabilities

Where to take
the prior
Thomas Bayes

probabilities?
(1702-1761)

Likelihood Function

L(a,s 2)=h(a,s 2)L0(a,s 2)
23.02.03                                                        4
Jam Sampling & Blending Theory

Now we know
the origin of
0.20  0.30 0.50

0.50
a worm in
0.20 0.05

the jam!
23.02.03                         5
2.Successive Bayesian Estimation (SBE)

23.02.03                                 6
SBE Concept
SBE principles    Whole data set
Data subset 1 Data subset 2                                      Data subset k
1)    Split up

How to eat away
y1         X1            y2         X2                 ...       yk         Xk
whole data set
f 1 (X 1 , a 0 , a 1 )   f 2 (X 2 , a 0 , a 2 )        ...       f k (X k , a 0 , a k )
2)    Process each

an elephant?
subset alone
Post          Prior        Post                       Prior       Post
3)    Make posterior              a0, a1         a0          a0, a2           ...        a0         a0, ak
2            2            2
information                 s1 N1 s1 N1 s2 N2                                     sk 2Nk sk 2 Nk

4)
Slice by slice!
Build prior
information                                                       Result
a 0 , a 1 ,…, a k
5)    Use it for the                                                     s2 N
next subset

23.02.03                                                                                                   7
OLS & SBE Methods for Two Subsets
OLS

approximation
near the
SBE

minimum!
23.02.03                            8
Posterior & Prior Information
Subset 1. Posterior Information

Make Posterior,
Rebuilding (common & partial parameters)

rebuild it and
apply as Prior!
Subset 2. Prior Information

23.02.03                                                9
Prior Information of Type I
Posterior Information            Prior Information

The same error
Parameter estimates          Prior parameter values b

Matrix A                     Recalculated matrix H

variance in the
s        s
Variance estimate    2       Prior variance value   0
2

each subset
NDF Nf                       Prior NDF N0

of data!
Objective Function

23.02.03                                                               10
Prior Information of Type II
Posterior Information         Prior Information

Different error
Parameter estimates   ˆ
a   Prior parameter values b

variances in the
A
Matrix          H         Recalculated matrix

Objective Function

each subset
of data!
23.02.03                                                        11
SBE Main Theorem
Different order of subsets processing

SBE
agree with
OLS!
Theorem (Pomerantsev & Maksimova , 1995)

23.02.03                                        12

23.02.03           13
Fitter Workspace
A     B            C            D        E            F        G         H          I      J       K      L         M       N        O          P            Q       R   S   T
1
2        Data                                                                                       General
y=exp(p)*A+exp(q)*B+exp(r)*C
3          x            t            y        w            f        A         B          C          Date       01.08.01 19:07
A=A0*exp(-k1*t)
4          13                0   0.047            1       0.047        1         0          0       Data    Bayes!rData
B=k1*A0/(k1-k2)*[exp(-k2*t)-exp(-k1*t)]+B0*exp(-k2*t)
5          13                2   0.553            1        0.56    0.125     0.448     0.4266       Model Bayes!ABCbayes
C=A0+B0+C0+A0/(k1-k2)*[k2*exp(-k1*t)-k1*exp(-k2*t)]-B0*exp(-k2*t)
6          13                4   0.412            1       0.403    0.016     0.209      0.775       Param eters
Bayes!rParam
A0="cA0"

Fitter is a
7          13                6   0.304            1       0.308    0.002     0.079     0.9194       Bayes Bayes!rBayes
B0="cB0"
8          13                8   0.275            1        0.27    2E-04     0.028      0.972       Precision
1E-11
C0="cC0"
9          13               10   0.253            1       0.257    3E-05      0.01     0.9904       Convergence
0.001 k1=?
10                                                                                                           Relative
Error type k2=?
11        Bayesian Inform ation                                                                              0.05
Significance
p=?
12        Nam e Value                Matrix                                            Exclude               0.95
Confidence
q=?
13        k1      1.07 265.3 146.5        0                              0         0                         Linearization
Prediction

tool for SBE!
r=?
14        k2     0.554 146.5    1117      0                              0         0
15                    0      0     0      0                              0         0                 Param eters estim ation
16                    0      0     0      0                              0         0                 Nam e Initial    Final    Deviation
17                    0      0     0      0                              0         0                 k1      1.06969 1.03907 0.060509
18                                                                                                   k2      0.55366 0.53661 0.027954
19                                                                                                   p      -3.05769 -3.05769 0.019371
y                        x = 13
20                                                                                                   q         -0.002    -0.002 0.039994
21
1.0                                                                                      r      -1.38757 -1.38757 0.017332
A                 C
22
23                                                                                                   Param eters                               Search Progress
24                                                                                                   k1      1.03907                           Objective value    0.0139
0.5
25                                             y                                                     k2      0.53661                           Com pleteness       100%
26                                                                                                   p       -3.05769                          Objective change   -1E-07
27                                             B                                                     q          -0.002                         Iteration               2
28          0.0                                                                                      r       -1.38757
29
0            2        4           6        8        10    t
30

23.02.03                                                                                                                                                                              14
Data & Model Prepared for Fitter

Values
Predictor       Response                                                Comment
Weight

A   B    C   D      E   F     G         H              I           J    K        L         M
1

Apply Fitter!
2       BoxBod Data             Parameters                     y
3        x   y w     f          a     100
213.80941           200
4         0     0   0.00        b     0.5472375
0.4
5
5         1 109 1 90.11
6
6         2 149 1 142.24      'BoxBOD model               100
7
7         3 149 1 172.41    y=a*[1-exp(-b*x)]
8
8         5 191 1 199.95          a=?
9         7 213 1 209.17          b=?                          0
10
10       10 224 1 212.91                                            0            4        8     x
11
11

Fitting                                      Equation
Parameters
23.02.03                                                                                             15
'Цикл "увлажнение-сушка"

Model f(x,a)
M=Sor*hev(t1-t)+Des*[hev(t-t1)+imp(t-t1)]
'Кинетика "увлажнения"
Sor=Sor1*hev(USESor1)+Sor2*[hev(-USESor1)+imp(-USESor1)]
'Кинетика "сушки"
Des=Des1*hev(USEDes1)+Des2*[hev(-USEDes1)+imp(-USEDes1)]
'Условие применимости асимптотик
USESor1=Sor2-Sor1

Different shapes of the same model
USEDes1=Des1-Des2
'константы и промежуточные величины
t3=(t-t1)*hev(t-t1)
t4=t*hev(t1-t)+t1*[hev(t-t1)+imp(t-t1)]
P2=PI*PI
P12=(PI)^(-0.5)

y = a + (b – a)*exp(–c*x)
R=r*(M1-M0)*exp(-r*t4)
Explicit model K=M1+(M0-M1)*exp(-r*t4)

Rather
V0=M0-C0
V1=M1-C0
'асимптотика сорбции при 0<t<tau
Sor1=C0+4*P12*(d*t)^0.5*[M0-C0+(M1-M0)*beta]

0 = a + (b – a)*exp(–c*x) – y
beta=1-exp(-z)

Implicit model    x=r*t
z=(a1*x+a2*x*x+a3*x*x*x)/(1+b1*x+b2*x*x+b3*x*x*x)
a1=0.6666539250029
a2=0.0121051017749

complex
a3=0.0099225322428

d[y]/d[x] = – c*(y –a); y(0) = b
b1=0.0848006232519

Diff. equation       b2=0.0246634591223
b3=0.0017549947958
'кинетика сорбции при tau<t<t1
Sor2=K-8*S1
S1=U01/n0+U11/n1+U21/n2+U31/n3+U41/n4
n0=P2

Presentation at worksheet

model!
U01=[(V0*n0*d-V1*r)*exp(-n0*d*t4)+R]/(n0*d-r)
n1=P2*9
U11=[(V0*n1*d-V1*r)*exp(-n1*d*t4)+R]/(n1*d-r)
n2=P2*25
U21=[(V0*n2*d-V1*r)*exp(-n2*d*t4)+R]/(n2*d-r)
n3=P2*49
U31=[(V0*n3*d-V1*r)*exp(-n3*d*t4)+R]/(n3*d-r)
n4=P2*81
U41=[(V0*n4*d-V1*r)*exp(-n4*d*t4)+R]/(n4*d-r)
'асимптотика десорбции при t1<t<t1+tau
Des1=K*[1-4*P12*(d*t3)^0.5]-8*S1
'кинетика десорбции при t1+tau<t
Des2=8*S2
S2=U02/n0+U12/n1+U22/n2+U32/n3+U42/n4
U02=(K-U01)*exp(-n0*d*t3)
U12=(K-U11)*exp(-n1*d*t3)
U22=(K-U21)*exp(-n2*d*t3)
U32=(K-U31)*exp(-n3*d*t3)
U42=(K-U41)*exp(-n4*d*t3)
'неизвестные параметры
d=?
M0=?
M1=?
C0=?
r=?
23.02.03                           t1=?
16
4. Spectral Kinetics Modeling

1
7
13
19
25
31
37
43                      8   10
4   6
49 0   2

23.02.03                                                                     17
Spectral Kinetic Data
Y(t,x,k)=C(t,k)P(x)+E
This is large
spectral signal          conc.                pure spectra             errors

Y non-linear E
wavelengths        species               wavelengths            wavelengths

species
time
time

= C  P  +

time
regression
K constants    L wavelengths           M species   N time points

problem!
Y is the (NL) known data matrix
C is the (NM) known matrix depending on unknown parameters k
P is the (ML) unknown matrix of pure component spectra
E is the (NL) unknown error matrix

23.02.03                                                                                18
How to Find Parameters k?
Method             Idea                Dimension     Problem

Full OLS                                               Large
K+ML >> 1
(hard)                                              dimension

Short OLS
(hard)
This is a                  K+MS  10
Small
precision

WCR
(hard&soft)
challenge!                    K  10
Matrix

1  e  kt          
GRAM       k  ln  k (t  s )      K+MA  100
Just one
(soft)        s e

                   model

23.02.03                                                           19
Simulated Example Goals

Compare SBE estimates with ‘true’ values

Compare SBE estimates for different order

Compare SBE estimates with OLS estimates

23.02.03                                         20
Model. Two Step Kinetics
k1    k2
A  B  C

Standard
dA
dt
 k1 A;       A(0)  A0  1

‘training’
dB
dt
 k1 A  k2 B; B(0)  B0  0

model
dC
 k2 B;        C(0)  C0  0
dt

‘True’ parameter values
k1=1          k2=0.5

23.02.03                                     21
Data Simulation
Simulated concentration profiles                   Simulated pure component spectra

1.0                                                           1.0
C                                                        B                   C
A                                                             A

Usual way of
0.8                                                           0.8
concentrations

spectral signal
0.6                                                           0.6

0.4               B                                           0.4

data simulation
0.2                                                           0.2

0.0                                                           0.0
0       2   4          6     8   10                           0   5   10   15   20   25   30   35       40   45   50
time                                                  conventional wavelengths

C1(t) = [A](t)                                                                                 P1(x) = pA (x)
Y(t,x)=C(t)P(x)(I+E)
C2(t) = [B](t)                                                                                 P2(x) = pB (x)
C3(t) = [C](t)                    STDEV(E)=0.03                                                P3(x) = pC (x)

23.02.03                                                                                                                                       22
Simulated Data. Spectral View

1.0       t=0                              t=10
0.9                                         t=8
t=6

Spectral
0.8

0.7
spectral signal

0.6                        t=2

view of data
0.5
t=4
0.4

0.3

0.2

0.1

0.0
0         10               20   30          40   50
conventional wavelengths

23.02.03                                                                                 23
Simulated Data. Kinetic View

1

Kinetic view
0.8
spectral signal

0.6

of data
0.4

0.2

0
0   2   4      6   8   10
time

23.02.03                                                       24
One Wavelength Estimates
Conventional wavelength 3              Conventional wavelength 14
y                                          y
1.0                                        1.0

0.8                                        0.8

0.6                                        0.6

0.4                                        0.4

0.2                                        0.2                                               14
3

0.0                                        0.0
0       2   4    6    8    10 time         0           2   4     6      8        10 time

Conventional wavelength 51                                 Estimates

y                                4.0
1.0

0.8                                        3.0

0.6
2.0
0.4
k1
0.2                                        1.0
51                                          k2
0.0
0.0
0       2   4    6    8    10 time                 3        14         51          O
23.02.03                                                                                           25
Four Wavelengths Estimates
Direct order                                       Inverse order
y                                               y
1.0                                             1.0

0.8                                             0.8

0.6                                             0.6
4                                                      50
0.4                                             0.4
3                                                      51
0.2                                       2     0.2                                              52
1                                                      53
0.0                                             0.0

again!
0       2     4     6       8   10 time         0           2     4       6     8        10 time

Random order                                         Estimates
y                                     1.5
1.0

0.8                                                                                       k1
1.0
0.6                                      29
8                                               k2
0.4
0.5
16
0.2
5
0.0                                             0.0
0       2     4     6       8   10 time                 D             I       R           O
23.02.03                                                                                               26
SBE Estimates at the Different Order
Direct 1, 2, 3, ….                                    Inverse 53, 52, 51, ….
1.25
1.50
k1                             Inverse
1.50
k1

SBE (practically)
1.25                                                   1.25
k1
1.00                                                   1.00

1.15
0.75
Direct        k2
0.75
k2
0.50                                                   0.50

doesn’t depend on
0.25                                                   0.25
1   8    15       22    29   36   43   50              53 49 45 41 37 33 29 25 21 17 13 9 5 1
1.05        conventional wavelengths                                  conventional wavelengths

the subsets order!
Random 16, 5, 29, ….                                  0.95 Confidence Ellipses
1.50        'True'                                     1.25     k1           Inverse
0.95
1.25
k1
1.15                                Direct
1.00
Random                     k2
0.75                                                   1.05
k2
0.85
0.50
0.95
Random
'True'
0.25
k2
0.42 0.44 0.46 0.48
16 41 27 33 19 2 15 51 21 9 24 50 12 22
conventional wavelengths
0.5
0.85                   0.54 0.56
0.52 0.5 0.52 0.540.56
0.42 0.44 0.46 0.48

23.02.03                                                                                                      27
SBE Estimates and OLS Estimates
1.25
k1
1.20

SBE estimates
1.15

1.10

are close to
1.05
OLS
1.00                                    'True'

OLS estimates!
0.95
SBE
0.90
k2
0.85
0.42     0.44   0.46   0.48   0.5     0.52   0.54   0.56

23.02.03                                                                  28
Pure Spectra Estimating
1.2       Spectrum A
B
C                              0.6

1

SBE gives
0.4
                     
spectral signal
spectral signal

0.8

accuracy
accuracy


good spectra
0.6                                              0.2



0.4

estimates!

                 0

0.2
0.2                                

0
0                                                -0.2
-0.2
1
1       11
11        21
21      31
31      41
41   51
51
conventional wavelength
conventional wavelength
23.02.03                                                                                         29
Real World Example Goals

Apply SBE for real world data

Compare SBE with other known methods

23.02.03                                    30
Data
Bijlsma S, Smilde AK. J.Chemometrics 2000; 14: 541-560
8
Epoxidation of 2,5-di-tert-butyl-1,4-benzoquinone
6

Preprocessed
SW-NIR spectra
spectral signal

4
2

Data
8
0
6
-2
spectral signal
240 spectra                              4
2
-4
1200 time points                         0
21 wavelengths
-6                                    -2
-4
Preprocessing:
-8                                    -6
Savitzky-Golay filter865
860                                -8        870          875         880
860    865    870       875   880
wavelength
wavelength

23.02.03                                                                               31
Progress in SBE Estimates
0.4

SBE works
k1
0.3

with the real
0.2

k2

world data!
0.1

0.0
860 862 864 866 868 870 872 874 876 878 880
wavelength (nm)

23.02.03                                                       32
SBE and the Other Methods
0.40    k1                          GRAM

SBE gives the
0.35

lowest deviations
0.30                                  SBE

0.25

and correlation!
0.20

0.15
WCR

LM-PAR                         k2
0.10
-0.05    0.00    0.05    0.10   0.15    0.20   0.25

23.02.03                                                             33
5. New Idea

23.02.03      34
Bayesian Step Wise Regression
Ordinarily Step Wise Regression      Bayesian Step Wise Regression

y=a1x1+a2x2+a3x3
BSWR accounts
correlations of
variables in step
wise estimation
Objective function

23.02.03                                                        35
BSW Regression & Ridge Regression

BSWR is RR with
a moving center
and non-Euclidean
metric
23.02.03                            36
Example. RMSEC & RMSEP

BSWR gives
typical U-shape of
the RMSEP curve

23.02.03                 37
Linear Model. RMSEC & RMSEP

BSWR is not
y=a1x1+a2x2+a3x3+a4x4+a5x5

worse then
0.5
RMSEC
0.4
RMSEP

PLS or PCR
0.3

and better
0.2

0.1

then SWR
0.0

PLS   PCR     OLS   SWR   BSWR

23.02.03                                            38
Non-Linear Model. RMSEC & RMSEP

y  a1e k1x1  a2 e k2 x2  a3 e k3 x3  a4 e k4 x4  a5 e k5 x5

1.2
For non-linear
1.2
RMSEC
RMSEC

model BSWR is
1.0       1.0
RMSEP
RMSEP
0.8       0.8

better then
0.6       0.6

0.4       0.4

PLS or PCR
0.2       0.2

0.0       0.0

PLS       PCR
PLS     OLS
PCR       SWR BSWR
OLS SWR           BSWR

23.02.03                                                                 39
Variable selection

BSWR is just
an idea, not
the method so
any criticism is
welcomed now!
23.02.03                      40
6. More Practical Applications of SBE

23.02.03                                41
Antioxidants Activity by DSC
DSC Data                                      Oxidation Initial Temperature (OIT)
4                  2                                          570                                    C=0.1
3                                 5                                                                  C=0.05
2                                                             550

To test
10
DSC signal

1                                           15

OIT T ,K
530
0
-1                                           20                                                      C=0.025
510
-2

antioxidants!
-3                                                            490
-4
-5                                                            470
460   470     480     490        500    510                    0    5        10        15        20
Temperature, K                                          Heating rate v , grad/min

23.02.03                                                                                                             42
Network Density of Shrinkable PE by TMA
TMA Data                                                             Network density

14

To solve
1.5                            1                                                                               A

Chemical modulus, gmm2
2                            12                                      B
Elongation L/Lo

1.4                                                                             10
8                                      C

technological
1.3
3         4                                   6

1.2                                                5                             4
2
1.1                                                                                          D1      D2   D3

problem!
0
0   10   20   30   40   50   60   70   80   90
0   5         10    15        20       25

23.02.03                                                                                                                                     43
PVC Isolation Service Life by TGA
TGA Data                                           Service life prediction

1.00                                     490

         

Temperature T , K
Mass chamge, y

0.98

To predict
450
0.96

0.94
410

durability!
0.92

0.90                                     370
0   10   20     30      40   50
Time t , min

23.02.03                                                                                               44
Tire Rubber Storage
Elongation at break                                                Tensile strength

Time, yr                                                      Time, yr
0             15             26 30         45                        0        15      23       30          45
6                                                                   30
Elongation @ break

5                                                                   25
T=20 C

Tensile, KPa
4                      T=20 C                                       20                                   Critical
Critical

To predict
level
3                                          level                    15
2                                                                   10
1                                                                    5
T=140 C        T=125 C           T=110 C                             T=140 C T=125 C            T=110 C

reliability!
0                                                                    0
0             20                40      60                           0       20               40        60
Time, hr                                                      Time, hr

23.02.03                                                                                                          45
7. Conclusions

1    SBE is of general nature and it can be used for any model

Thanks!
2    SBE agrees with OLS

3    SBE gives small deviations and correlations

4    SBE uses no subjective a priori information

5    SBE may be useful for non-linear modeling (BWSR?)

23.02.03                                                               46

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 9 posted: 11/8/2011 language: English pages: 46