Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out
Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Introduction to Stochastic Spectral Methods

VIEWS: 45 PAGES: 133

									Introduction
RVs and RPs

Several UQ methods



Spectral
                            Introduction to Stochastic Spectral Methods
representation
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial
Chaos                                                                    Didier LUCOR
Post-processing



Resolution for a                                                 Institut Jean Le Rond d’Alembert
general SPDE
Stochastic Galerkin                                                          e
                                                             Equipe Fluides R´actifs et Turbulence
Method (SGM)

Stochastic Collocation                                             e
                                                          Universit´ Pierre et Marie Curie (Paris VI)
Method (SCM)



Multivariate
quadratures
Full                                                                       ECODOQUI
Sparse
                                                                         25 Novembre 2008
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                    25 Novembre 2008   1 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        2 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        3 / 133
                                    Introduction
Introduction
RVs and RPs

Several UQ methods
                                    Modeling uncertainties, numerical errors and data uncertainties can
Spectral
representation
                                    interact.
           e
Karhunen-Lo`ve

Polynomial Chaos
                                    Aleatoric/stochastic (not reducible) or epistemic (incomplete
generalized Polynomial              knowledge) uncertainty.
Chaos

Post-processing
                                    Need to quantitatively access the impact of uncertain data on
Resolution for a                    simulation outputs.
general SPDE
Stochastic Galerkin
Method (SGM)
                                    The validity of the model can be established only if uncertainty in
Stochastic Collocation              numerical predictions due to uncertain input parameters can be
Method (SCM)
                                    quantified.
Multivariate
quadratures                         Difficulty: not looking for the unique solution.
Full

Sparse
                                    Possible sources: simulation constants/parameters, transport
Intrude or not
                                    coefficients, physical properties, boundary/initial conditions, geometry,
intrude ?                           models, numerical schemes, ...
Open issues


                         Introduction to Stochastic Spectral Methods         25 Novembre 2008             4 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        5 / 133
                                   Random variables (RVs)
Introduction

                            Complete probability space: Ω, A, P , where Ω is the event space, A ⊂ 2Ω
RVs and RPs
                                                        `        ´
Several UQ methods
                            the σ-algebra and P the probability measure.
Spectral                        `        ´
representation
                            X : Ω, A, P → R, with probability density function (pdf) fX and
           e
Karhunen-Lo`ve

Polynomial Chaos
                            cumulative density function (cdf) FX .
generalized Polynomial
Chaos                       Random vector : X = {Xi (ω)}N , N ∈ N.
                                                        i=1
Post-processing

                                                                        ˜    ˜
                                    uncorrelated if : cov(X1 , X2 ) = E(X1 ⊗ X2 ) = 0
Resolution for a
general SPDE
Stochastic Galerkin
                                    independent if : E[φ1 (X1 ) φ2 (X2 )] ≡ E[φ1 (X1 )] E[φ2 (X2 )]
Method (SGM)

Stochastic Collocation
Method (SCM)                Expectation operator : if u ∈ L1 (Ω),
                                                           P
                                                         Z                Z
Multivariate
quadratures
                                         E[u] =< u >=        u(ω)dP (ω) =   u(x )fX (x )dx
                                                                        Ω             R
Full

Sparse
                                    ¯
                                    u = E[u]
Intrude or not
intrude ?
                                    varu = E[˜ 2 ]
                                             u                 where   u =u −u
                                                                       ˜     ¯
                                    P (u ≤ u0 ) = P ({ω ∈ Ω : u(ω) ≤ u0 }) = E[1{u≤u0 } ]
Open issues


                         Introduction to Stochastic Spectral Methods             25 Novembre 2008     6 / 133
                                   Random processes (RPs)
Introduction

                            Random process (RP) u(x´ ω) indexed by a bounded domain D ⊂ Rd on the
RVs and RPs

Several UQ methods                           `        ,
                            probability space Ω, A, P .
Spectral
representation
                               1. It is a set of RVs indexed by x ∈ D. For each x = x0 , u(x = x0 , ω) is a
           e
Karhunen-Lo`ve

Polynomial Chaos
                                  RV on Ω. It is defined by all its finite-dimensional distributions
generalized Polynomial            functions. Fx1 ,...,xN = P {u(x1 ) ≤ x1 . . . u(xN ) ≤ xN }
                                                                       ˆ                 ˆ
Chaos

Post-processing
                               2. It is a function-valued RV. u is a function of D × Ω with value u(x , ω).
Resolution for a                  for given x ∈ D and ω ∈ Ω. For each fixed ω ∈ Ω, u(x , ω) is a function
general SPDE
                                  - a realization - of x in D.
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)
                            A random field (RF) is a measurable function : u : D × Ω → R.

Multivariate
                            Expectation operator :                     Z                      Z
quadratures
Full                                       E[u] = u(x , ω) =               u(x , ω)dP (ω) =           u(x , x )fX (x )dx
Sparse
                                                                       Ω                          R


Intrude or not
                                    u (x ) = E[u(x )], varu (x ) = E[˜ (x )2 ] where u (x ) = u(x ) − u (x )
                                    ¯                                u               ˜                ¯
intrude ?
                                    P (u(x ) ≤ u0 ) = P ({ω ∈ Ω : u(x , ω) ≤ u0 }) = E[1{u(x )≤u0 } ]
Open issues


                         Introduction to Stochastic Spectral Methods                   25 Novembre 2008                    7 / 133
                                   Special case of Gaussian random fields
Introduction
RVs and RPs

Several UQ methods
                            A random field, for which all finite-dimensional distributions are jointly
Spectral                    Gaussian. it is sufficient to specify its second order statistics, i.e. its mean
representation
                            and its covariance function.
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial
                            Convenient properties:
Chaos

Post-processing                     Uncorrelated Gaussian RVs → independent
Resolution for a
general SPDE
                                    Linear combinations of Gaussian RVs are also Gaussian
Stochastic Galerkin
Method (SGM)                        Gaussian fields occur naturally due to the CLT
Stochastic Collocation
Method (SCM)
                                    Second order statistical information is often the only one available in
Multivariate                        applications
quadratures
Full
                                    Gaussian random fields are the maximum entropy model
Sparse



Intrude or not              However, they may hence be inappropriate for describing positive or strictly
intrude ?
                            positive RPs.
Open issues


                         Introduction to Stochastic Spectral Methods         25 Novembre 2008             8 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        9 / 133
                                   Statistical methods
Introduction
RVs and RPs

Several UQ methods

                                                                                     √
Spectral
representation
                                    Brute-force Monte Carlo method : Converges as 1/ N ; Convergence
Karhunen-Lo`ve
           e                        rate independent of number of RVs. Robust. Parallelizable.
Polynomial Chaos

generalized Polynomial
Chaos

Post-processing
                                    Monte Carlo based methods :

Resolution for a                             QMC (Quasi-MC)
general SPDE
Stochastic Galerkin                          MCMC (Markov chain MC)
Method (SGM)

Stochastic Collocation
Method (SCM)
                                             Variance reduction technique : limitation with large number of RVs
                                             (importance sampling, correlated sampling, conditional sampling, ...)
Multivariate
quadratures
Full
                                    Response Surface Method (RSM) : realizations reduced by interpolation
Sparse
                                    in state space; same limitation with large number of RVs.
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods               25 Novembre 2008                  10 / 133
                                   Non-statistical methods
Introduction
RVs and RPs

Several UQ methods
                                    Indirect methods :

Spectral                                     Fokker-Planck equation : Solves for distribution function; Challenging in
representation                               high dimensions (computational cost), BCs
           e
Karhunen-Lo`ve

Polynomial Chaos                             Moments equations : Closure of equations is key. Good for linear
generalized Polynomial
Chaos
                                             problems with Gaussian RVs
Post-processing



Resolution for a                    Direct methods (e.g. SFEM, stochastic finite element method) :
general SPDE
Stochastic Galerkin
Method (SGM)
                                             Interval analysis : maximum output bounds
Stochastic Collocation
Method (SCM)                                 Perturbation-based methods : Taylor expansion around means. Differ at
                                             the local representation of randomness: mid-point, local average,
Multivariate                                 piecewise polynomial, etc
quadratures
Full                                         Operator-based methods : Weighted integral method; Neumann
Sparse
                                             expansion
Intrude or not                                                                                           e
                                             Stochastic spectral methods : Polynomial chaos, Karhunen-Lo`ve,
intrude ?
                                             Wiener-Askey/gPC chaos & representation [Wiener 1938, Lo`ve 1977,
                                                                                                     e
Open issues                                  Cameron & Martin 1947, Ghanem & Spanos 1991, Xiu & Karniadakis 2002].

                         Introduction to Stochastic Spectral Methods                25 Novembre 2008                 11 / 133
                                   Stochastic spectral methods for UQ
Introduction
RVs and RPs

Several UQ methods



Spectral                            Means of representing 2nd order RPs parametrically through a finite set
representation
Karhunen-Lo`ve
           e
                                    of RVs (finite dimensional noise assumption).
Polynomial Chaos

generalized Polynomial
Chaos
                                    Projection of the solution onto the space spanned by those RVs.
Post-processing

                                    Pros: not limited to small uncertainties with Gaussian distributions,
Resolution for a
general SPDE                        convergence rate, richness of the information (not only solution stats),
Stochastic Galerkin
Method (SGM)
                                    does not require high skills in prob. or stat.
Stochastic Collocation
Method (SCM)
                                    Cons: dimensionality, robustness (stability issues for non-linear and
Multivariate                        instationary problems), solvers modification.
quadratures
Full

Sparse                              Future efforts: computational efficiency, error estimates, adaptivity,
                                    sparsity, reduced basis.
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods         25 Novembre 2008               12 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        13 / 133
                                   Finite dimensional noise assumption
Introduction
RVs and RPs

Several UQ methods
                            Need to reduce the infinite-dimensional probability space to a
Spectral                    finite-dimensional space.
representation
           e
Karhunen-Lo`ve

Polynomial Chaos
                                    Characterizing the probability space by a countable number N of
generalized Polynomial
Chaos                               mutually independent RVs.
Post-processing



Resolution for a
                                    Approximation of the target random process (decomposition).
general SPDE
Stochastic Galerkin
Method (SGM)                        The random field has been represented in a finite number of
Stochastic Collocation
Method (SCM)
                                    independent random variables −→ function on a high dimensional
                                    space.
Multivariate
quadratures
Full                                The independence of the underlying RVs allows to see each of them as
Sparse                              the axis of a coordinate system (Doob-Dynkin lemma).
Intrude or not
intrude ?                                                                e
                                    One possible choice : the Karhunen Lo´ve type expansion.
Open issues


                         Introduction to Stochastic Spectral Methods       25 Novembre 2008           14 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        15 / 133
                                              KL representation - Derivation of the expansion (1)
Introduction
RVs and RPs

Several UQ methods
                            The Karhunen-Lœve (KL) expansion of a RP is a Fourier-type series based
                            on the spectral expansion of its covariance function of a RP.
Spectral
representation
Karhunen-Lo`ve
           e
                            We consider a second-order RP : u(x , ω) ∈ L2 (D × Ω) ⇒ E[u(x )2 ] < +∞,
Polynomial Chaos            ∀x ∈ D ⊂ Rd , and its covariance Ru (x 1 , x 2 ) ∈ (D × D) :
generalized Polynomial
Chaos

Post-processing
                                                                   Ru (x 1 , x 2 )        =          E(u(x 1 , ω) ⊗ u(x 2 , ω))

                                             1                                                                           1
Resolution for a
general SPDE                                0.9                                                                         0.9

                                            0.8                                                                         0.8
Stochastic Galerkin
Method (SGM)                                0.7                                                                         0.7
                               Covariance




                                                                                                           Covariance
                                            0.6                                                                         0.6
Stochastic Collocation
Method (SCM)                                0.5                                                                         0.5

                                            0.4                                                                         0.4

                                            0.3                                      Exponential                        0.3                                 Exponential
Multivariate                                                                         Gaussian                                                               Gaussian
                                            0.2                                                                         0.2
quadratures                                                                          Sine                                                                   Sine
                                            0.1                                      Triangular                         0.1                                 Triangular
Full                                                                                 Linear Exponential                                                     Linear Exponential
                                             0                                                                           0
                                             −1       −0.5    0       0.5     1                                          !1        !0.5      0    0.5   1
Sparse                                                       Lag                                                                            Lag


                                                  1D covariance kernels : L = 2 and correlation length of Cl = L/2 and Cl = L/8.
Intrude or not
intrude ?
                            They demonstrate how fast the correlation drop between two distant points.
Open issues


                         Introduction to Stochastic Spectral Methods                                                          25 Novembre 2008                        16 / 133
                                    KL representation - Derivation and expansion (2)
Introduction
RVs and RPs

Several UQ methods
                            We can define a covariance operator such that:
                                                                Z
Spectral
representation              Ru : L2 (D) → L2 (D), (Ru u)(x ) :=   Ru (x 1 , x 2 )u(x 2 )d x 2 , ∀u ∈ L2 (D)
Karhunen-Lo`ve
           e                                                                 D
Polynomial Chaos

generalized Polynomial      The covariance operator is real, symmetric and positive-definite.
Chaos

Post-processing
                            It has a countable sequence of eigenpairs with: eigenvalues {λi } ⊂ R+ and
Resolution for a            orthogonal eigenfunctions φi (x ) (complete basis).
general SPDE
Stochastic Galerkin
Method (SGM)
                            Spectral representation of the kernel :
Stochastic Collocation                                             ∞
                                                                   X
Method (SCM)
                                                 Ru (x 1 , x 2 ) =   λi φi (x 1 )φi (x 2 ),
Multivariate                                                           i=1
quadratures
Full                        Second-order Fredholm equation (for the eigenvalues and eigenfunctions):
Sparse                         Z                                                    Z
Intrude or not
                                  Ru (x 1 , x 2 )φi (x 2 )d x 2 = λi φi (x 1 ) with   φi (x )φj (x )d x = δij
intrude ?                            D                                                   D

Open issues


                         Introduction to Stochastic Spectral Methods             25 Novembre 2008           17 / 133
                                    KL representation - Derivation and expansion (3)
Introduction
RVs and RPs

Several UQ methods
                            The KL representation of u(x , ω) is:
Spectral                                                                                     ∞
representation                                                                               X√
           e
Karhunen-Lo`ve                                                       ¯
                                                          u(x , ω) = u (x ) + σu               λi φi (x )Xi (ω),
Polynomial Chaos                                                                             i=1
generalized Polynomial
Chaos                       with Xi : centred, normalized, uncorrelated RVs (but not necessarily
Post-processing
                            independent); EXi = 0, E(Xi Xj ) = δij .
Resolution for a
general SPDE                Xi (ω) are evaluated from:
Stochastic Galerkin
                                                                            Z
Method (SGM)
                                                                       1            `                    ´
Stochastic Collocation                                    Xi (ω) =                      u(x , ω) − u (x ) φi (x )d x
                                                                                                   ¯
Method (SCM)                                                           λi       D

Multivariate
quadratures                 In practice : different ways of dealing with those RVs. Due to the fact that
Full
                            there exists an infinity of second-order RPs sharing the same given
Sparse
                            covariance kernel (they all hold the same decomposition. But, they do not
Intrude or not              have necessarily the same RVs).
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                                25 Novembre 2008   18 / 133
                                    KL representation - Derivation and expansion (4)
Introduction
RVs and RPs
                            Practically, we can only deal with finite representation and the KL expansion
Several UQ methods
                            has to be truncated. We have:
Spectral
representation
                                                                                    N
                                                                                    X√
Karhunen-Lo`ve
           e                                            u N (x , ω) = u (x ) + σu
                                                                      ¯               λi φi (x )Xi (ω),
Polynomial Chaos                                                                    i=1
generalized Polynomial
Chaos

Post-processing
                            The more modes N we add into the KL expansion, the more small-scale
                            details we capture with the representation.
Resolution for a
general SPDE
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)



Multivariate
quadratures
Full

Sparse



Intrude or not
intrude ?                   KL representation of groundwater flow stochastic conductivity for different level of resolution
                            (Courtesy of Hermann Matthies, TU Braunschweig)
Open issues


                         Introduction to Stochastic Spectral Methods                      25 Novembre 2008             19 / 133
                                    KL representation - Eigenvalues distributions
Introduction
RVs and RPs
                            Eigenvalues of the exponential kernel and Gaussian kernel for different
Several UQ methods
                            correlation lengths. (a): Cl = 1; (b): Cl = 0.1
Spectral
                                                                               (a)                                                               (b)
representation                                                  10
                                                                  0
                                                                                                                             10
                                                                                                                               0


                                                                                     Exponential Kernel                                                   Exponential Kernel
Karhunen-Lo`ve
           e                                                                                                                                              Gaussian Kernel
                                                                                     Gaussian Kernel
                                                                                                                               −1
                                                                                                                             10
Polynomial Chaos
                                                                  −5
                                                                10
generalized Polynomial                                                                                                         −2




                                                                                                               Eigenvalues
                                                  Eigenvalues

                                                                                                                             10
Chaos

Post-processing                                                                                                                −3
                                                                  −10                                                        10
                                                                10


Resolution for a                                                                                                             10
                                                                                                                               −4


general SPDE                                                      −15
                                                                10
                                                                                                                               −5
Stochastic Galerkin                                                    0   5   10      15                 20
                                                                                                                             10
                                                                                                                                   0     5       10        15              20
Method (SGM)                                                                   n                                                                 n

Stochastic Collocation
Method (SCM)
                                    More modes are needed as the correlation decreases (i.e. the noise level
Multivariate
quadratures
                                    increases).
Full

Sparse
                                    In the asymptotic limit of white noise, an infinity number of modes will
                                    be needed.
Intrude or not
intrude ?
                                    For a given correlation length, the smoothness of the covariance kernel
Open issues                         impacts the convergence rate of the eigenvalues.

                         Introduction to Stochastic Spectral Methods                                                                   25 Novembre 2008                         20 / 133
                                    KL representation - Example of KL eigenfunctions
Introduction
RVs and RPs                 Distributions of the eigenfunctions of the exponential kernel (top row) and
Several UQ methods
                            Gaussian kernel (bottom row) for different correlation lengths Cl .
Spectral                                                               Cl = 1                                     Cl = 0.1
                                                        1.5                                    1.5
representation
           e
Karhunen-Lo`ve                                              1                                   1

Polynomial Chaos
                                                        0.5                                    0.5
generalized Polynomial
Chaos
                                               fn(y)




                                                                                      fn(y)
                                                            0                                   0
Post-processing
                                                       −0.5      N=1                          −0.5     N=1
                                                                 N=2                                   N=2
Resolution for a                                                 N=3                                   N=3
                                                        −1       N=4                           −1      N=4
general SPDE                                                     N=5                                   N=5
                                                                 N=6                                   N=6
Stochastic Galerkin                                    −1.5                                   −1.5
                                                         −0.5             0     0.5             −0.5                 0       0.5
Method (SGM)                                                              y                                          y

Stochastic Collocation                                                 Cl = 1
                                                                                                                  Cl = 0.1
                                                        4
Method (SCM)                                                                                   1.5

                                                        3
                                                                                                1
Multivariate                                            2
quadratures                                                                                    0.5
                                                        1
Full
                                               f (y)




                                                                                      fn(y)
                                                        0                                       0
                                                  n




Sparse
                                                       −1
                                                                N=1                           −0.5     N=1
                                                       −2       N=2                                    N=2
Intrude or not                                                  N=3                                    N=3
intrude ?                                                       N=4                            −1      N=4
                                                       −3
                                                                N=5                                    N=5
                                                                N=6                                    N=6
                                                       −4                                     −1.5
                                                       −0.5              0      0.5             −0.5                 0       0.5
Open issues                                                              y                                           y




                         Introduction to Stochastic Spectral Methods                                    25 Novembre 2008           21 / 133
                                   Error minimizing property
Introduction
RVs and RPs

Several UQ methods

                                    If we truncate after the N largest eigenvalues, we have an optimal - in
Spectral
representation
                                    variance - expansion in N RVs.
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial
                                    There is no other linear representation with N terms that has a smaller
Chaos
                                    mean square quadratic error 2 .N
Post-processing



Resolution for a
general SPDE                        u N converges to u in variance uniformly :
Stochastic Galerkin
Method (SGM)                                                              ∞
                                                                          X
                                          sup E (u(x ) − u N (x ))2 = sup   λi ui2 (x ) → 0, as m → ∞
Stochastic Collocation
                                               `                   ´
Method (SCM)
                                          x ∈D                         x ∈D
                                                                              i=N +1
Multivariate
quadratures
Full
                                    The KL representation based on the eigenfunctions of the covariance
Sparse
                                    kernel is the only representation that will involved non-correlated
Intrude or not                      (orthogonal) RVs.
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods              25 Novembre 2008       22 / 133
                                   Convergence rate of the spectrum (1)
Introduction
RVs and RPs

Several UQ methods

                                    The truncation error decreases monotonically with the number of terms
Spectral
representation                      in the expansion.
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial              The convergence is inversely proportional to the correlation length and
Chaos

Post-processing
                                    depends on the regularity of the covariance kernel.

Resolution for a
general SPDE                        if Ru is piecewise analytic on D × D, with D ⊂ Rd [Frauenfelder 2005] :
Stochastic Galerkin
Method (SGM)

Stochastic Collocation                           λn ≤ c1 exp(−c2 n 1/d ),                  ∀m ≥ 1, c1 , c2 > 0 ind. of n
Method (SCM)



Multivariate
                                    where c1 , c2 > 0 are constants independent of n;
quadratures
Full

Sparse                              if Ru is piecewise H k ,0 ≡ H k ⊗ L2 with k > 0, we have :
Intrude or not                                                         λn ≤ c3 n −k /d ,     ∀n ≥ 1, c3 > 0
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                           25 Novembre 2008            23 / 133
                                   Convergence rate of the spectrum (2)
Introduction
RVs and RPs

Several UQ methods
                            Decay of eigenvalues for the Gaussian kernel function.

Spectral
representation                      Left : Eigenvalues in 3D domain and theoretical estimate.
           e
Karhunen-Lo`ve

Polynomial Chaos                    Right : Eigenvalue decays in 1D, 2D, and 3D domains.
generalized Polynomial
Chaos                               Exponential decay of eigenvalues in all three cases with smaller decay
Post-processing                     rate at higher dimensions.
Resolution for a
general SPDE                                1
                                                           Largest 2000 Eigenvalues of Factorizable Kernel exp(!10|x!x!|2), x " (!1,1)3                              2
                                                                                                                                                                    10
                                           10
                                                                                                               computed eigenvalues #
                                                                                                                                      m
Stochastic Galerkin                                                                                            estimated eigenvalues 1/ $(m1/3/2)
                                                                                                                                                                     0
Method (SGM)                                0
                                                                                                                                                                    10
                                           10

Stochastic Collocation                                                                                                                                               !2
                                                                                                                                                                    10
Method (SCM)                                !1
                                           10

                                                                                                                                                                     !4
                                                                                                                                                                    10
                                            !2
                                           10




                                                                                                                                                       Eigenvalue
Multivariate                                                                                                                                                         !6
                                                                                                                                                                    10
                                       m




quadratures
                                      #




                                            !3
                                           10
                                                                                                                                                                     !8
Full                                                                                                                                                                10

                                            !4
Sparse                                     10                                                                                                                        !10
                                                                                                                                                                    10


                                            !5
                                           10                                                                                                                        !12
                                                                                                                                                                    10
Intrude or not                                                                                                                                                                                                                                    1D Gaussian kernel
                                                                                                                                                                                                                                                  2D Gaussian kernel
intrude ?                                   !6                                                                                                                       !14
                                                                                                                                                                                                                                                  3D Gaussian kernel
                                           10                                                                                                                       10
                                                 0   200        400       600       800       1000      1200      1400      1600      1800      2000                       1   2   3   4   5   6   7   8   9   10     11 12   13   14   15   16   17   18   19   20    21
                                                                                               m                                                                                                                    Index


Open issues


                         Introduction to Stochastic Spectral Methods                                                                                                                   25 Novembre 2008                                                                     24 / 133
                                   Convergence rate of the spectrum (3)
Introduction
RVs and RPs
                                                                                                           14 Largest Eigenvalues of exp(!|x!x!|1+"), x# (!1,1)
Several UQ methods                                                                            1
                                                                                             10
                                                                                                                                                                  "=0.00, observed
                                                                                                                                                                  "=0.00, predicted
                                                                                                                                                                  "=0.25, observed


Spectral                          Decay of eigenvalues for the covariance                     0
                                                                                             10
                                                                                                                                                                  "=0.25, predicted
                                                                                                                                                                  "=0.50, observed
                                                                                                                                                                  "=0.50, predicted
representation
                                  kernel function Ru = exp(−|x1 − x2 |1+δ ).                                                                                      "=0.75, observed
                                                                                                                                                                  "=0.75, predicted

           e
Karhunen-Lo`ve                                                                                !1
                                                                                             10

Polynomial Chaos

generalized Polynomial
                                  For smaller values of δ, the eigenvalues
                                                                                              !2
                                                                                             10
Chaos                             decay slower, as predicted by the estimate.
Post-processing
                                                                                              !3
                                                                                             10



Resolution for a
general SPDE                                                                                  !4
                                                                                             10
                                                                                                   0   2      4              6             8             10          12               14

Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)
                            In another study [Lucor 2004], the convergence rate for a RP with an
                            exponential covariance function over a temporal domain T :
Multivariate
quadratures                                                                         2
                                                                h 2 (x ; ω) − hN (x ; ω)          T 1
Full
                                                       ε≡                   2 (x ; ω)
                                                                                         ∼ 0.4053      .
Sparse                                                                    h                       Cl N
Intrude or not              Numerical simulations for random oscillators established that this indeed is a
intrude ?
                            very sharp error estimate.
Open issues


                         Introduction to Stochastic Spectral Methods                   25 Novembre 2008                                                             25 / 133
                                   Special case of Gaussian processes
Introduction
RVs and RPs

Several UQ methods          A very convenient special case is the one where the RP to represent u(x , ω)
                            is a Gaussian RP.
Spectral
representation
Karhunen-Lo`ve
           e
                            In this case, it has a KL representation with RVs ξi (ω) : Gaussian vector.
Polynomial Chaos

generalized Polynomial
                            These Gaussian RVs are uncorrelated ⇒ independent.
Chaos

Post-processing
                                                                                   N
                                                                                   X√
                                                                     ¯
                                                          u(x , ω) = u (x ) + σu     λi φi (x )ξi (ω),
Resolution for a
                                                                                   i=1
general SPDE
Stochastic Galerkin
Method (SGM)
                            The task of numerically representing continuous non-Gaussian random fields
Stochastic Collocation      via a KL decomposition that the random variables involved are independent,
Method (SCM)
                            falls into the topic of numerical representation of non-Gaussian processes is
Multivariate                not a trivial task.
quadratures
Full                        That it is possible to construct multidimensional functional spaces based on
Sparse
                            finite number of dependent random variables [Ghanem & Soize].
Intrude or not
intrude ?                   However, such a construction does not, in its current form, allow
                            straightforward numerical implementations.
Open issues


                         Introduction to Stochastic Spectral Methods                     25 Novembre 2008   26 / 133
                                   Choices of covariance kernel functions (1)
Introduction
RVs and RPs                         A covariance function can be generated through first-order
Several UQ methods
                                    autoregression model or first-order Markov process.
Spectral
representation
                                                        ui = αui−1 + βXi ,             α = e −1/γ ,        α2 + β 2 = 1,
           e
Karhunen-Lo`ve

Polynomial Chaos
                                    Correspondence of random processes and their covariance functions :
generalized Polynomial              [Su 2006]
Chaos

Post-processing                Dimension d                             Random process u(x ; ω)                       Covariance function Ru
                                                                                                                              “       ”
                                   d =1                                 ui = αui−1 + βXi                                 exp − C   r
Resolution for a                                                                                                                   l
general SPDE                       d =1                                   b
                                                                    ui = 2 (ui−1 + ui+1 ) + Xi                      (1 + r /Cl ) exp(−r /Cl )
                                                                                                                                  “    ”
Stochastic Galerkin
                                   d =2                                 b
                                                                ui,j = 4 (ui±1,j + ui,j ±1 ) + Xi,j                        r K      r
Method (SGM)                                                                                                               Cl “ 1 C
                                                                                                                                      l”
Stochastic Collocation             d =3                      b
                                                   ui,j ,k = 6 (ui±1,j ,k + ui,j ±1,k + ui,j ,k ±1 ) + Xi,j ,k            exp − C   r
Method (SCM)                                                                                                                       l
                                d = 1, 2, 3                                     N/A                                        exp(−r 2 )
Multivariate
quadratures
Full
                                    The corresponding continuous process satisfies approximation the
Sparse                              Helmholtz equation :   ∂2u     u        β
                                                                − 2 =2         X.
Intrude or not
                                                           ∂x 2   Cl      ∆x 2
intrude ?
                                    Solving the equation with periodic boundary conditions produces the
Open issues                         solution u(x , X ) from which the covariance can then be computed.
                         Introduction to Stochastic Spectral Methods                             25 Novembre 2008                       27 / 133
                                   Choices of covariance kernel functions (2)
Introduction
RVs and RPs

Several UQ methods



Spectral
representation
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial
Chaos

Post-processing



Resolution for a
general SPDE
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)



Multivariate
quadratures
Full

Sparse



Intrude or not
intrude ?
                            KL realizations of RPs in two spatial dimensions for different covariance kernels (rows) and
Open issues
                            different correlation lengths (columns). (Courtesy of Andreas Keese, TU Braunschweig).
                         Introduction to Stochastic Spectral Methods                  25 Novembre 2008                    28 / 133
                                   Rational spectra
Introduction
RVs and RPs

Several UQ methods
                            If we consider a certain class of RPs, that are stationary output of linear
Spectral
representation
                            filters to white noise excitation, they have a spectral density that takes the
Karhunen-Lo`ve
           e                general form:
Polynomial Chaos                                                    N (ω 2 )
                                                            S (ω) =          ,
generalized Polynomial
Chaos                                                               D(ω 2 )
Post-processing
                            where N and D are polynomials of the angular frequency ω of order n and d
Resolution for a            respectively.
general SPDE
Stochastic Galerkin
Method (SGM)                The nice property is that the Fredholm equation can be transformed in this
Stochastic Collocation
Method (SCM)
                            case into a second order homogeneous differential equation that yields:

Multivariate
                                                                       ˆ d2 ˜             ˆ d2 ˜
quadratures                                                     λD            φ(x 1 ) = N        φ(x 1 )
Full
                                                                        dx21               dx21
Sparse
                            This equation can be solved in terms of λ. There exists in fact some analytic
Intrude or not              or semi-analytic solutions for a few spectrums [Ghanem & Spanos 1991].
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                        25 Novembre 2008   29 / 133
                                   Rational spectra - exponential kernel (1)
Introduction
RVs and RPs

Several UQ methods
                            Let us consider the first-order, stationary, Markovian process in space
Spectral                    u(x , ω) (or in time) with exponential correlation function defined as
representation
Karhunen-Lo`ve
           e                                       Ru (x1 , x2 ) = σu e −|x2 −x1 |/Cl
                                                                    2
                                                                                        ∀(x1 , x2 ) ⊂ D × D,
Polynomial Chaos
                                         2
generalized Polynomial      where   is the variance, Cl is the correlation length and
                                        σu
Chaos

Post-processing
                            D = [−L/2, L/2] ∈ R.

Resolution for a            The u(x , ω) process can be shown to be the stationary solution of the
general SPDE
Stochastic Galerkin
                            differential equation:
Method (SGM)
                                                                       r
                                                           1              2
Stochastic Collocation                            u(x ) = − u(x ) + σu
                                                  ˙                          W (x ),
Method (SCM)
                                                           Cl             Cl
Multivariate                in which W (x ) is the zero-mean stationary white noise with covariance
quadratures
Full
                            function δ(x ).
Sparse
                            It has been shown that higher order Markovian kernels may be expressed as
Intrude or not
intrude ?
                            linear combinations of first order ones. This model is widely used in the
                            literature.
Open issues


                         Introduction to Stochastic Spectral Methods                     25 Novembre 2008      30 / 133
                                   Rational spectra - exponential kernel (2)
Introduction
RVs and RPs

Several UQ methods
                            The integral Fredholm equation can be differentiated twice which gives a
Spectral                    differential equation of the form:
representation
                                                                                                                        1
           e
Karhunen-Lo`ve                                                                                                   (2 −   Cl
                                                                                                                             λ)
Polynomial Chaos                    φ (x ) + γ 2 φ(x ) = 0               ∀x ∈ [−L/2, L/2],       with     γ2 =                    ,
generalized Polynomial                                                                                             Cl λ
Chaos

Post-processing             where γ can be numerically solved by evaluating the differential equation at
Resolution for a
                            the boundaries of the domain.
general SPDE
                                                  ∗
Stochastic Galerkin         The variables γn and γn are the first n roots of those non-linear equations
Method (SGM)

Stochastic Collocation
                                                                          1           L
Method (SCM)
                                                                             − γ tan(γ )     =    0
                                                                          Cl          2
Multivariate
quadratures                                                             ∗    1       ∗L
                                                                       γ −      tan(γ )      =    0,
Full                                                                         Cl       2
Sparse

                            where L is the reference length of the spatial domain.
Intrude or not
intrude ?
                            Typically, one can use a Newton-Raphson algorithm to evaluate the roots.
Open issues


                         Introduction to Stochastic Spectral Methods                       25 Novembre 2008                           31 / 133
                                   Rational spectra - exponential kernel (3)
Introduction
RVs and RPs

Several UQ methods

                            Once this is done, the eigenvalues are:
Spectral
representation
                                                                        2                                    2
           e
Karhunen-Lo`ve                                   λn      =                        and         λ∗ =
                                                                                               n                     .
Polynomial Chaos                                                 Cl (γn + Cl2 )
                                                                      2                               Cl (γn + Cl2 )
                                                                                                           ∗2

generalized Polynomial
Chaos
                              The corresponding eigenfunctions are thus
Post-processing



Resolution for a                                                                      cos(γn x )
general SPDE                                                       φn (x )   =    r                         ,
Stochastic Galerkin                                                                   L       sin(2γn L )
                                                                                                      2
Method (SGM)
                                                                                      2
                                                                                          +      2γn
Stochastic Collocation
Method (SCM)                                                                               ∗
                                                                                      sin(γn y)
                                                                   φ∗ (x )
                                                                    n        =    r                         ,
Multivariate                                                                                  sin(2γn L )
                                                                                                    ∗
                                                                                      L
quadratures
                                                                                      2
                                                                                          −      2γn∗
                                                                                                      2
Full

Sparse
                            and they form the symmetric and the anti-symmetric components in the KL
Intrude or not
intrude ?
                            expansion.

Open issues


                         Introduction to Stochastic Spectral Methods                           25 Novembre 2008          32 / 133
                                   Rational spectra - exponential kernel (4)
Introduction
RVs and RPs                 The 4-term and 10-term KL approximation of the exponential covariance
Several UQ methods          surface versus x1 and x2 with L = 1 and Cl = L/2 (left column), and the
Spectral
                            corresponding relative error surfaces (right column).
representation
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial
Chaos

Post-processing



Resolution for a
general SPDE
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)



Multivariate
quadratures
Full

Sparse



Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods   25 Novembre 2008           33 / 133
                                         Rational spectra - exponential kernel (5)
Introduction
RVs and RPs

Several UQ methods

                            Variance computed from the KL representation of the RP with a total of N
Spectral
representation
                            modes. Left : Cl = 1.0 and right : Cl = 0.1.
           e
Karhunen-Lo`ve

Polynomial Chaos
                                                                      Cl=1.0                                                                         Cl=0.1
generalized Polynomial                     1                                                                              1
Chaos
                                                                                                                        0.95
Post-processing
                                         0.99
                                                                                                                         0.9

Resolution for a                                                                                                        0.85
                                         0.98
general SPDE
                                                                                                                         0.8
                               Var(CN)




                                                                                                              Var(CN)
Stochastic Galerkin
Method (SGM)                             0.97                                                                           0.75

Stochastic Collocation                                                                                                   0.7
Method (SCM)                             0.96                                                    N=8                                                                            N=8
                                                                                                                        0.65
                                                                                                 N=16                                                                           N=16
                                                                                                 N=24                    0.6                                                    N=24
Multivariate                             0.95                                                    N=32                                                                           N=32
quadratures                                                                                      N=40                   0.55                                                    N=40
Full                                     0.94                                                                            0.5
                                           −0.5 −0.4 −0.3 −0.2 −0.1     0      0.1   0.2   0.3   0.4    0.5               −0.5 −0.4 −0.3 −0.2 −0.1     0      0.1   0.2   0.3   0.4    0.5
Sparse                                                                 lag                                                                            lag


Intrude or not
intrude ?                   Maximum error is at the boundaries of the domain.
Open issues


                         Introduction to Stochastic Spectral Methods                                                           25 Novembre 2008                                        34 / 133
                                   Non-rational spectra
Introduction
RVs and RPs                         Much more difficult to derive the solution of non-rational spectra.
Several UQ methods
                                    Possible to derive an ODE from the Fredholm integral equation, but we
Spectral
representation
                                    only have analytical solutions for a few RPs.
Karhunen-Lo`ve
           e
                                    The Fredholm integral equation can be solved by standard techniques
Polynomial Chaos

generalized Polynomial
                                    (means of a variational formulation, Galerkin-type or collocation-type
Chaos
                                    methods).
Post-processing

                                    In projection methods, approximation of the eigenfunctions :
Resolution for a                                         κn
general SPDE                                             X
Stochastic Galerkin                            φn (x ) =
                                                 m          am,k vk (x ),  x ∈ D, m = 1, 2, · · · .
Method (SGM)
                                                                       k =1
Stochastic Collocation
Method (SCM)                        The Galerkin approximation : find λm = 0 and φn ∈ Vn such that
                                                                                      m
                                     Z                                          Z
                                          Rh (x1 , x2 )φn (x2 )v (x )dxdx2 = λm   φn (x )v (x )dx , ∀v ∈ Vn .
Multivariate
quadratures                                             m                          m
Full                                    D×D                                       D
Sparse
                                    Matrix eigenvalue problem : Av = λMv, where matrices A and M are
Intrude or not                      symmetric and positive definite.
intrude ?
                                    Fast Multipole Method with O(κn log(κn )) instead of O(κ2 )
                                                                                            n
Open issues                         operations.
                         Introduction to Stochastic Spectral Methods          25 Novembre 2008            35 / 133
                                   Non-rational spectra - Gaussian kernel (1)
Introduction
RVs and RPs                 A smooth kernel, the Gaussian kernel (analytic fct. mean-square
Several UQ methods          differentiable of any order) :
Spectral                                                                   −(x2 −x1 )2
representation                                                       2       L2 C 2
Karhunen-Lo`ve
           e                                        Ru (x1 , x2 ) = σu e         l       ∀(x1 , x2 ) ⊂ D × D,
Polynomial Chaos                                                              2
generalized Polynomial
                            where L is the diameter of the domain D and σu and Cl are the standard
Chaos
                            deviation and the correlation length, respectively.
Post-processing



Resolution for a
general SPDE
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)



Multivariate
quadratures
Full

Sparse



Intrude or not
intrude ?
                            The 4-term KL approximation of the Gaussian covariance surface versus x1 and x2 with L = 1
Open issues
                            and Cl = L/2, and the corresponding relative error surfaces of these approximations.

                         Introduction to Stochastic Spectral Methods                       25 Novembre 2008       36 / 133
                                                                    Non-rational spectra - Gaussian kernel (2)
Introduction
RVs and RPs

Several UQ methods



Spectral                    The convergence in the relative L2 -norm error of the Gaussian kernel and its
representation
                            variance with increasing number of expansion, N.
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial                                               0                                                                                        0
                                                                    10                                                                                       10
Chaos
                                                                                                           Cl=0.01                                                                                       Cl=0.01
Post-processing
                                                                                                           Cl=0.1                                                                                        Cl=0.1
                               Relative L2−norm error of variance




                                                                                                                          Relative L −norm error of kernel
                                                                     −4                                    Cl=1                                               −4
                                                                                                                                                                                                         Cl=1
                                                                    10                                     Cl=10                                             10                                          Cl=10
Resolution for a
                                                                                                           Cl=100                                                                                        Cl=100
general SPDE
Stochastic Galerkin
                                                                     −8                                                                                       −8
Method (SGM)                                                        10                                                                                       10
Stochastic Collocation




                                                                                                                          2
Method (SCM)

                                                                     −12                                                                                      −12
                                                                    10                                                                                       10
Multivariate
quadratures
Full                                                                 −16                                                                                      −16
                                                                    10                                                                                       10
Sparse                                                                    0   5   10   15   20   25   30   35        40                                            0      5    10    15   20   25   30   35        40
                                                                                            N                                                                                             N


Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                                                                                                   25 Novembre 2008                        37 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        38 / 133
                                   Definitions and convergence properties (1)
Introduction
RVs and RPs

Several UQ methods          The Polynomial Chaos (PC) is an orthogonal development for non-linear
                            functionals with Gaussian measure.
Spectral
representation

                            Let Θ ⊆ L2 (Ω, B, P ) be a separable Gaussian Hilbert space. The space Θ is
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial      a subspace of L2 (Ω, B, P ) that only contains centered Gaussian RVs
Chaos

Post-processing
                            ξ := {ξj }N and that is complete (the covariance as scalar product).
                                      j =1

Resolution for a
general SPDE                The space PlN of all N -variate polynomials of degree l is denoted by:
Stochastic Galerkin
Method (SGM)

Stochastic Collocation                 PlN (Θ) := {p(ξ) | p is polynomial of degree l ; ξ ∈ Θ, N < ∞}
Method (SCM)



Multivariate
                            The space of all polynomials is called P(Θ) := ∞ PlN (Θ) and we denote
                                                                          S
quadratures                                                                 l=0
Full
                               ¯
                            by PlN the closure with respect to L2 and define
Sparse
                                                               N
                                                              H=0      :=   ¯N
                                                                            P0 ,
Intrude or not
intrude ?                                                       N
                                                               H=l     :=   ¯    ¯N
                                                                            PlN Pl−1    for     l ∈N
Open issues


                         Introduction to Stochastic Spectral Methods                   25 Novembre 2008   39 / 133
                                   Definitions and convergence properties (2)
Introduction
RVs and RPs

Several UQ methods
                                     N
                                    H=l is called the homogeneous chaos of order l
Spectral
representation                                    Sl
                                     N              N
Karhunen-Lo`ve
           e                        H≤l :=        H=l is called the polynomial chaos of order l
                                                     i=0
Polynomial Chaos                                       ∞
                                                       M N
generalized Polynomial
Chaos                               L2 (Ω, Σ(Θ), P ) =     H=l is called the polynomial chaos
Post-processing                                                        l=0
                                    decomposition
Resolution for a
general SPDE
Stochastic Galerkin
                            An orthogonal basis of the PC space are the multivariate Hermite
                                          N
Method (SGM)
                            polynomials H≤P of degree P < ∞ in N independent Gaussian RVs.
Stochastic Collocation
Method (SCM)



Multivariate
                            Those polynomials can be constructed as tensor-products of univariate
                                                  1
quadratures                 Hermite polynomials HP ≡ HP .
Full

Sparse
                            The entire family of polynomials can be written in a compact fashion by
Intrude or not
intrude ?
                            means of multi-indices.

Open issues


                         Introduction to Stochastic Spectral Methods         25 Novembre 2008         40 / 133
                                   Definitions and convergence properties (3)
Introduction
RVs and RPs

Several UQ methods
                            Based on the previous PC decomposition, any Σ(Θ)-measurable RV on Ω
                            (not necessarily Gaussian) with finite variance has a L2 convergent
Spectral
representation
                            approximation in the multivariate Hermite polynomial space [Cameron & Martin
Karhunen-Lo`ve
           e                1947]. This convergence is in the mean square sense.
Polynomial Chaos

generalized Polynomial
Chaos
                            The RV u(ω) can be represented in terms of orthogonal Hermite polynomials
Post-processing             of independent normalized Gaussian RVs, ξ = {ξi (ω)}∞ :
                                                                                i=1
                                                                                  ∞
                                                                                  X
Resolution for a
general SPDE                                                             u(ω) =          ˆ
                                                                                         uk Hk (ξ(ω)),
Stochastic Galerkin                                                               k =0
Method (SGM)

Stochastic Collocation                                         ˆ
                            where the deterministic coefficients uk are defined as:
Method (SCM)
                                                                                                            Z
Multivariate                            u, Hk
quadratures                 ˆ
                            uk =                where u, Hk = E[u(ω)Hk (ξ(ω))] =                                   u(ξ)Hk (ξ(ω))ρ(ξ)d ξ,
                                        Hk , Hk                                                               Γ
Full

Sparse                                                           2
                                                            e −ξi /2
                                                 QN
                            with ρ(ξ) =              i=1
                                                              √
                                                                2π
                                                                     .
Intrude or not
intrude ?
                            Orthogonality condition : < Hi , Hj >= E[Hi , Hj ] = E[Hi2 ] δij .
Open issues


                         Introduction to Stochastic Spectral Methods                            25 Novembre 2008                   41 / 133
                                   Truncated PC representation : Curse of dimensionality
Introduction
RVs and RPs
                            In practice, stochastic fields can be approximated by a PC of a certain finite
Several UQ methods          order, corresponding to a finite set of N random variables and a maximum
                            polynomial order P .
Spectral
representation
Karhunen-Lo`ve
           e                Due to the tensor form construction of the PC multi-dimensional basis, an
Polynomial Chaos            expansion in N independent stochastic dimensions, and of highest order P
generalized Polynomial
Chaos                       has a total number of terms M :
Post-processing                                                   „         «
                                                      M              P +N
                                                dim(HP ) = M =
Resolution for a                                                        N
general SPDE
Stochastic Galerkin
Method (SGM)
                            Vector space dimensions of the polynomial chaos of degree P in N
Stochastic Collocation      independent RVs :
Method (SCM)

                                             P=2          P=3          P=4     P=5     P=6          P=7         P=8        P=9        P = 10
Multivariate                   N   =   2           6          10          15      21        28            36          45         55          66
quadratures
                               N   =   3          10          20          35      56        84           120         165        220         286
Full                           N   =   4          15          35          70     126       210           330         495        715     1   001
Sparse                         N   =   5          21          56         126     252       462           792     1   287    2   002     3   003
                               N   =   6          28          84         210     462       924       1   716     3   003    5   005     8   008
Intrude or not                 N   =   7          36         120         330     792   1   716       3   432     6   435   11   440    19   448
intrude ?                      N   =   8          45         165         495   1 287   3   003       6   435    12   870   24   310    43   758
                               N   =   9          55         220         715   2 002   5   005      11   440    24   310   48   620    92   378
                               N   =   10         66         286       1 001   3 003   8   008      19   448    43   758   92   378   184   756
Open issues


                         Introduction to Stochastic Spectral Methods                             25 Novembre 2008                       42 / 133
                                   One-dimensional Hermite polynomials
Introduction
RVs and RPs                 We consider the univariate Hermite polynomials H (ξ) that will serve as the
Several UQ methods
                            foundation for the construction of the multi-dimensional Hermite
Spectral                    polynomials (orthogonal with respect to the Gaussian measure).
representation
Karhunen-Lo`ve
           e
                                                                             1   ξ2
Polynomial Chaos            The pdf of a Gaussian RV ξ is defined as :ρ(ξ) = √ e − 2 .
generalized Polynomial                                                       2π
Chaos

Post-processing                                             30


Resolution for a                                            20
general SPDE
Stochastic Galerkin
Method (SGM)
                                                            10
Stochastic Collocation
Method (SCM)

                                                             0
Multivariate
quadratures
Full
                                                           −10
Sparse

                                                           −20
Intrude or not
intrude ?
                                                           −30
                                                             −3        −2   −1   0    1         2       3
                                                                                 ξ
Open issues


                         Introduction to Stochastic Spectral Methods                 25 Novembre 2008       43 / 133
                                   Multi-dimensional Hermite polynomials construction (1)
Introduction
RVs and RPs

Several UQ methods          For (N > 1), one can build the multi-dimensional PC basis as
Spectral
                            tensor-products of univariate polynomials.
representation
Karhunen-Lo`ve
           e                For Hermite polynomials, a simple way to construct the k th polynomial
                              N                                                           1
Polynomial Chaos
                            Hk (ξ) is to tensorize one-dimensional Hermite polynomials Hki (ξi ).
generalized Polynomial
Chaos

Post-processing             We define the multi-index k = {k1 . . . ki , . . . kN }, such that we have:
Resolution for a                                                                  N
                                                                                  Y
general SPDE                                                            N                1
                                                                       Hk (ξ) =         Hki (ξi ),
Stochastic Galerkin
Method (SGM)                                                                      i=1
Stochastic Collocation
Method (SCM)
                            and k :=         {ki }N
                                               is a sequence whose each component refers to the
                                                  i=1
Multivariate
                                                                                          1
                            polynomial degree p of the i th one-dimensional polynomial Hp≤P (ξi )
quadratures                                    N
Full
                            contributing to Hk (ξ).
Sparse
                            If we define the modulus of k by |k | = N ki , the set of all Hk (ξ) with
                                                                                             N
                                                                   P
                                                                      i=1
                                                                N                     N
Intrude or not
intrude ?
                            |k | = p is an orthogonal basis of Hp and the set of all Hk (ξ) with |k | ≤ P
                                                                 N
                            is a polynomial chaos of order P : H≤P .
Open issues


                         Introduction to Stochastic Spectral Methods                        25 Novembre 2008   44 / 133
                                   Multi-dimensional Hermite polynomials construction (2)
Introduction
RVs and RPs
                            Examples: We consider first a multi-dimensional Hermite polynomial of order
Several UQ methods
                            at most P = 3 in N = 2 Gaussian independent RVs {ξ1 , ξ2 }.
Spectral
representation              We list the M = 10 multi-indices k by order of appearance for this case:
           e
Karhunen-Lo`ve
                                                   »                                             –
Polynomial Chaos
                                                       0 1 0 2 1 0 3 2 1 0
generalized Polynomial          k (N = 2; P = 3) =                                                 .
Chaos                                                  0 0 1 0 1 2 0 1 2 3
Post-processing

                            The first row corresponds to the k1 values and the second one to the k2
Resolution for a
general SPDE                values.
Stochastic Galerkin
Method (SGM)
                            Vertical lines correspond to the limitations between different polynomial
Stochastic Collocation      order representations.
Method (SCM)                                                       2
                            The first polynomial is a constant: H0 (ξ1 , ξ2 ) = 1.
Multivariate
quadratures                 We have, for instance for the 8-th polynomial of the basis:
Full

Sparse
                                                                   N =2
                                                                   Y
                                      2                                   1           1          1
                                     H7 (ξ1 , ξ2 )         =           Hki (ξi ) = Hk1 (ξ1 ) ⊗ Hk2 (ξ2 )
Intrude or not
intrude ?
                                                                   i=1
                                                                     1           1                 2               2
                                                           =       H1 (ξ1 ) × H2 (ξ2 ) = (ξ1 ) × (ξ2 − 1)    = ξ1 ξ2 − ξ1 .
Open issues


                         Introduction to Stochastic Spectral Methods                      25 Novembre 2008                    45 / 133
                                   Multi-dimensional Hermite polynomials construction (3)
Introduction
RVs and RPs
                                              2                                                                                                                                3                                                                                                                     3

Several UQ methods                        1.5
                                                                                                                                                                               2                                                                                                                     2

                                                                                                                                                                               1                                                                                                                     1

                                              1                                                                                                                                0                                                                                                                     0

                                                                                                                                                                              !1                                                                                                                    !1
                                          0.5
                                                                                                                                                                              !2                                                                                                                    !2
Spectral                                      0
                                              3
                                                                                                                                                                              !3
                                                                                                                                                                               3
                                                                                                                                                                                                                                                                                                    !3
                                                                                                                                                                                                                                                                                                     3

representation                                            2
                                                                      1
                                                                                0
                                                                                                                                            0
                                                                                                                                                    1
                                                                                                                                                            2
                                                                                                                                                                    3                2
                                                                                                                                                                                          1
                                                                                                                                                                                              0
                                                                                                                                                                                                                                                 0
                                                                                                                                                                                                                                                             1
                                                                                                                                                                                                                                                                     2
                                                                                                                                                                                                                                                                             3                                2
                                                                                                                                                                                                                                                                                                                       1
                                                                                                                                                                                                                                                                                                                               0
                                                                                                                                                                                                                                                                                                                                                                                       0
                                                                                                                                                                                                                                                                                                                                                                                                    1
                                                                                                                                                                                                                                                                                                                                                                                                            2
                                                                                                                                                                                                                                                                                                                                                                                                                    3

                                                                                     !1                                                                                                            !1                                                                                                                               !1
                                                                                              !2                                  !1                                                                    !2                             !1                                                                                                    !2                              !1
                                                                                                                        !2                                                                                                   !2                                                                                                                                    !2

Karhunen-Lo`ve
           e
                                                                                x2                  !3        !3                       x1
                                                                                                                                                                        (a)                   x2             !3    !3                       x1
                                                                                                                                                                                                                                                                                 (b1 )                                         x2                  !3    !3                       x1
                                                                                                                                                                                                                                                                                                                                                                                                                        (b2 )
Polynomial Chaos                      8                                                                                                                                                                                                                                                                   8
                                                                                                                                                                                    10

                                      6                                                                                                                                                                                                                                                                   6
generalized Polynomial                4
                                                                                                                                                                                     5
                                                                                                                                                                                                                                                                                                          4
Chaos                                 2
                                                                                                                                                                                     0
                                                                                                                                                                                                                                                                                                          2

                                      0                                                                                                                                             !5                                                                                                                    0
Post-processing                      !2                                                                                                                                            !10                                                                                                                   !2
                                      3                                                                                                                                              3                                                                                                                    3
                                                  2                                                                                                             3                         2                                                                                                                        2                                                                                                    3
                                                              1                                                                                                                               1                                                                                  3                                         1
                                                                                                                                                        2                                                                                                                2                                                                                                                                      2
                                                                          0                                                                     1                                                  0                                                             1                                                                  0                                                                   1
                                                                                !1                                                     0                                                                !1                                               0                                                                                  !1                                                  0
                                                                                         !2                                  !1                                                                              !2                             !1                                                                                                    !2                              !1
                                                                                                                   !2                                                                                                             !2                                                                                                                                    !2

Resolution for a
                                                                          x2                   !3    !3                           x1
                                                                                                                                                                    (c1 )                          x2             !3    !3                       x1
                                                                                                                                                                                                                                                                                     (c2 )                                          x2                  !3    !3                       x1
                                                                                                                                                                                                                                                                                                                                                                                                                            (c3 )
general SPDE
                                      30                                                                                                                                            100                                                                                                                   150
                                      20                                                                                                                                                                                                                                                                  100
Stochastic Galerkin                   10
                                                                                                                                                                                     50
                                                                                                                                                                                                                                                                                                              50

Method (SGM)                           0                                                                                                                                              0                                                                                                                        0
                                     !10                                                                                                                                                                                                                                                                  !50
                                                                                                                                                                                   !50
                                     !20                                                                                                                                                                                                                                                                 !100
Stochastic Collocation               !30                                                                                                                                           !100                                                                                                                  !150
                                       3
Method (SCM)                                      2
                                                              1                                                                                         2
                                                                                                                                                                3
                                                                                                                                                                                      3
                                                                                                                                                                                          2
                                                                                                                                                                                              1                                                                                  3
                                                                                                                                                                                                                                                                                                            3
                                                                                                                                                                                                                                                                                                                       2
                                                                                                                                                                                                                                                                                                                               1                                                                                        3
                                                                                                                                                                                                                                                                         2                                                                                                                                      2
                                                                          0                                                                     1                                                  0                                                             1                                                                      0                                                               1
                                                                                !1                                                     0                                                                !1                                               0                                                                                   !1                                                 0
                                                                                         !2                                  !1                                                                              !2                             !1                                                                                                     !2                             !1
                                                                                                                   !2                                                                                                             !2                                                                                                                                     !2
                                                                          x2                   !3    !3                           x1
                                                                                                                                                                    (d1 )                          x2             !3    !3                       x
                                                                                                                                                                                                                                                     1                               (d2 )                                              x2               !3    !3                      x
                                                                                                                                                                                                                                                                                                                                                                                            1                               (d3 )
Multivariate
quadratures                           150

                                      100
                                                                                                                                                                                    100                                                                                                                   30

                                                                                                                                                                                                                                                                                                          20
                                                                                                                                                                                     50
                                       50                                                                                                                                                                                                                                                                 10

Full                                      0                                                                                                                                           0                                                                                                                       0

                                      !50                                                                                                                                                                                                                                                                !10
                                                                                                                                                                                   !50
Sparse                               !100                                                                                                                                                                                                                                                                !20

                                     !150                                                                                                                                          !100                                                                                                                  !30
                                        3                                                                                                                                             3                                                                                                                    3
                                                      2                                                                                                                                   2                                                                                                                            2                                                                                                3
                                                                  1                                                                                             3                             1                                                                                  3                                             1
                                                                                                                                                        2                                                                                                                2                                                                                                                                      2
                                                                           0                                                                    1                                                  0                                                             1                                                                   0                                                                  1
                                                                                    !1                                                 0                                                                !1                                               0                                                                                   !1                                                 0
                                                                                          !2                                 !1                                                                              !2                             !1                                                                                                    !2                              !1
                                                                                                                   !2                                                                                                             !2                                                                                                                                    !2

Intrude or not
                                                                           x2                  !3        !3                       x
                                                                                                                                   1                                (d4 )                          x2             !3    !3                       x
                                                                                                                                                                                                                                                     1                               (d5 )                                           x2                 !3    !3                       x1
                                                                                                                                                                                                                                                                                                                                                                                                                            (d6 )
intrude ?
                                 Hermite polynomials in a truncated domain. (a): constant; (b1−2 ): 1st -order; (c1−3 ):
Open issues
                                  2nd -order; (d1−6 ): 5th -order.
                         Introduction to Stochastic Spectral Methods                                                                                                                                                                                                                   25 Novembre 2008                                                                                                                             46 / 133
                                    Multi-dimensional Hermite polynomials construction (4)
Introduction                For a multi-dimensional Hermite polynomial of order at most P = 3 in
RVs and RPs

Several UQ methods
                            N = 3 Gaussian independent RVs, we have:

Spectral
                            k
                           » (N = 3; P = 3) =                                                                                                  –
                                0      1     0      0      2     1     1   0   0   0   3   2     2     1      1   1     0     0    0       0
representation
                                0      0     1      0      0     1     0   2   1   0   0   1     0     2      1   0     3     2    1       0
           e
Karhunen-Lo`ve
                                0      0     0      1      0     0     1   0   1   2   0   0     1     0      1   2     0     1    2       3
Polynomial Chaos
                                                                                                 k     p      Polynomial decomposition
generalized Polynomial
Chaos                                                                                            0     0      H0 (ξ1 ) H0 (ξ2 ) H0 (ξ3 )
Post-processing                                                                                  1     1      H1 (ξ1 ) H0 (ξ2 ) H0 (ξ3 )
                                                                                                 2            H0 (ξ1 ) H1 (ξ2 ) H0 (ξ3 )
Resolution for a            Multi-indices values and ordering                                    3            H0 (ξ1 ) H0 (ξ2 ) H1 (ξ3 )
general SPDE                                                                                     4     2      H2 (ξ1 ) H0 (ξ2 ) H0 (ξ3 )
                            described in this section are totally                                5            H1 (ξ1 ) H1 (ξ2 ) H0 (ξ3 )
Stochastic Galerkin
Method (SGM)                independent from the type of random                                  6            H1 (ξ1 ) H0 (ξ2 ) H1 (ξ3 )
Stochastic Collocation
                                                                                                 7            H0 (ξ1 ) H2 (ξ2 ) H0 (ξ3 )
Method (SCM)                distributions and polynomial basis that                              8            H0 (ξ1 ) H1 (ξ2 ) H1 (ξ3 )
                                                                                                 9            H0 (ξ1 ) H0 (ξ2 ) H2 (ξ3 )
                            are considered.                                                     10     3      H3 (ξ1 ) H0 (ξ2 ) H0 (ξ3 )
Multivariate
quadratures                                                                                     11            H2 (ξ1 ) H1 (ξ2 ) H0 (ξ3 )
Full
                            They merely reflect the adopted ranking                              12            H2 (ξ1 ) H0 (ξ2 ) H1 (ξ3 )
                                                                                                13            H1 (ξ1 ) H2 (ξ2 ) H0 (ξ3 )
Sparse                      logic together with the tensor-product                              14            H1 (ξ1 ) H1 (ξ2 ) H1 (ξ3 )
                            construction.                                                       15            H1 (ξ1 ) H0 (ξ2 ) H2 (ξ3 )
Intrude or not                                                                                  16            H0 (ξ1 ) H3 (ξ2 ) H0 (ξ3 )
intrude ?                                                                                       17            H0 (ξ1 ) H2 (ξ2 ) H1 (ξ3 )
                                                                                                18            H0 (ξ1 ) H1 (ξ2 ) H2 (ξ3 )
Open issues                                                                                     19            H0 (ξ1 ) H0 (ξ2 ) H3 (ξ3 )


                         Introduction to Stochastic Spectral Methods                       25 Novembre 2008                         47 / 133
                                   Examples of PC representation of non-gaussian RFs
Introduction
RVs and RPs

Several UQ methods
                            Convergence of any functional in L2 (Ω, A, P ) does not mean necessarily a
Spectral
                            fast convergence for non-Gaussian random fields.
representation
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial
Chaos

Post-processing



Resolution for a
general SPDE
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)



Multivariate
quadratures
Full

Sparse

                            PDF from Hermite PC approximations of Beta(α, β) distributions. (left): α = β = 0; (right):
Intrude or not              α = 2, β = 0 (Courtesy of Donbin Xiu, Purdue university).
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods             25 Novembre 2008                  48 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        49 / 133
                                   Definitions (1)
Introduction
RVs and RPs
                                    Second-order random field u : Ω → V over the Hilbert space V :
Several UQ methods
                                                                               2
                                                                         E u       = E(u, u) < ∞.
Spectral
representation                      The RV u(ω) can be represented by gPC :
Karhunen-Lo`ve
           e                                                      ∞
                                                                  X
Polynomial Chaos
                                                    u(x , t, ω) =   ˆ
                                                                    uk (x , t)Φk (X (ω))
generalized Polynomial
Chaos                                                                              k =0
Post-processing
                                                     ˆ
                                    The knowledge of uk fully determines the RP.
Resolution for a
general SPDE                        {Φj (X (ω))} are orthogonal polynomials in terms of a zero-mean
Stochastic Galerkin
Method (SGM)
                                    random vector X := {Xj (ω)}N , satisfying the orthogonality relation
                                                                 j =1
Stochastic Collocation
Method (SCM)                                                                 Φi Φj = Φ2 δij ,
                                                                                      i

Multivariate                        Numerically, we have to truncate the representation :
quadratures
Full                                                                               M
                                                                                   X
Sparse                                                             u(x , t, ω) =          ˆ
                                                                                          uk (x , t)Φk (X (ω))
                                                                                   k =0
Intrude or not
intrude ?
                                    where M depends of the number of random dimensions N and the
Open issues                         highest polynomial order P of the polynomial basis.
                         Introduction to Stochastic Spectral Methods                           25 Novembre 2008   50 / 133
                                   Definitions (2)
Introduction
RVs and RPs

Several UQ methods                  The inner product is in the Hilbert space determined by the measure of
Spectral
                                    the random variables :
representation                                        Z                           Z
           e
Karhunen-Lo`ve

Polynomial Chaos
                                        f (X )g(X ) =        f (X )g(X )dP (ω) = f (X )g(X )w (X )d X
                                                                       ω∈Ω
generalized Polynomial
Chaos

Post-processing                     with w (X ) denoting the density of the law dP (ω) with respect to the
                                    Lebesgue measure d X and with integration taken over a suitable
Resolution for a
general SPDE                        domain, determined by the range of the random vector X .
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)
                                    In the discrete case, the above orthogonal relation takes the form :
                                                                      X
Multivariate
quadratures
                                                       f (X )g(X ) =      f (X )g(X )w (X ).
Full
                                                                             X
Sparse



Intrude or not
                                    Correspondence between the type of the orthogonal polynomials {Φ}
intrude ?                           and the law of the random variables X .
Open issues


                         Introduction to Stochastic Spectral Methods             25 Novembre 2008          51 / 133
                                   The Askey scheme (1)
Introduction
RVs and RPs                 The Askey scheme of hypergeometric polynomials
Several UQ methods


                                                    4
                                                        F3(4)                Wilson                              Racah
Spectral
representation
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial
Chaos
                                                        F2(3)   Continuous       Continuous
                                                    3
                                                                dual Hahn          Hahn                   Hahn           Dual Hahn
Post-processing



Resolution for a
general SPDE
Stochastic Galerkin
Method (SGM)                                                    Meixner
                                                    2
                                                        F1(2)       -                 Jacobi             Meixner         Krawtchouk
Stochastic Collocation
                                                                Pollaczek
Method (SCM)



Multivariate
quadratures
Full                                                1
                                                        F1(1)            Laguerre                             Charlier         2
                                                                                                                                   F0(1)
Sparse



Intrude or not
intrude ?
                                                    2   F0(0)                                  Hermite
Open issues


                         Introduction to Stochastic Spectral Methods                                     25 Novembre 2008                  52 / 133
                                   The Askey scheme (2)
Introduction
RVs and RPs

Several UQ methods



Spectral
representation              The correspondence between the polynomial of the Wiener-Askey type and
Karhunen-Lo`ve
           e
                            the associated probability distribution (N ≥ 0 is a finite number).
Polynomial Chaos

generalized Polynomial
Chaos

Post-processing                                           Random variable ξ    Wiener-Askey PC {φ(ξ)}          Support
                                     Continuous                Gaussian            Hermite-chaos             (−∞, ∞)
Resolution for a                     distribution               gamma              Laguerre-chaos              [0, ∞)
general SPDE
                                                                 beta               Jacobi-chaos                [a, b]
Stochastic Galerkin
Method (SGM)
                                                               uniform             Legendre-chaos               [a, b]
Stochastic Collocation
                                       Discrete                Poisson              Charlier-chaos         {0, 1, 2, . . . }
Method (SCM)                         distribution              binomial           Krawtchouk-chaos         {0, 1, . . . , N }
                                                           negative binomial       Meixner-chaos           {0, 1, 2, . . . }
Multivariate                                                hypergeometric           Hahn-chaos            {0, 1, . . . , N }
quadratures
Full

Sparse



Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                    25 Novembre 2008                        53 / 133
                                   gPC basis choice
Introduction
RVs and RPs

Several UQ methods



Spectral
representation                      Optimal choice of the gPC basis for non-Gaussian RP with irregular
Karhunen-Lo`ve
           e                        and/or unknown measures is an open question.
Polynomial Chaos

generalized Polynomial
Chaos

Post-processing
                                    For non-linear problems, optimal gPC representation (from Askey
                                    scheme) of the inputs     optimality for the solution.
Resolution for a
general SPDE
Stochastic Galerkin
Method (SGM)
                                    Possibility of handing RPs whose underlying RVs have differents
Stochastic Collocation              distributions =⇒ concept of gPC mixed basis.
Method (SCM)



Multivariate
quadratures                         Possibility of tailoring a basis =⇒ Concept of gPC reprsentation for
Full                                arbitrary measures.
Sparse



Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods        25 Novembre 2008               54 / 133
                                   Mixed basis (1)
Introduction
RVs and RPs

Several UQ methods
                            Example:
Spectral
representation
                            Let us consider a multi-dimensional Hermite-Legendre polynomial Φk of
           e
Karhunen-Lo`ve

Polynomial Chaos
                            order at most P = 3 in N = 2 independent RVs, where ξ is Gaussian RV
generalized Polynomial      and ζ is a uniform RV.
Chaos

Post-processing



Resolution for a
                            We tensorize the 1D Hermite polynomial with the 1D Legendre polynomial
general SPDE                corresponding to the Gaussian and the uniform RVs, respectively.
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)                We have, for instance for the 8-th polynomial of the basis:
Multivariate
quadratures
Full
                                       Φ2 (ξ, ζ)
                                        7                =       1
                                                                Hk1 (ξ) ⊗ L12 (ζ)
                                                                           k
                                                                                          1             3      1
                                                                 1
                                                                H1 (ξ) × L1 (ζ) = (ξ) ×     (3ζ 2 − 1) = ξζ 2 − ξ
Sparse
                                                         =                2
                                                                                          2             2      2
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                      25 Novembre 2008          55 / 133
                                   Mixed basis (2)
Introduction
RVs and RPs
                            Another example : the simple first-order linear ODE:
Several UQ methods
                                               dΘ
Spectral
                                                  = −C Θ                       with Θ (t = 0) = Θ0 and t ∈ [0, T ] .
representation
                                               dt
           e
Karhunen-Lo`ve

Polynomial Chaos            The deterministic and stochastic analytic solutions are, respectively:
                                                  ¯                                      ¯
generalized Polynomial
Chaos                               Θ(t) = Θ0 e −C t and Θ(t, ω) = (Θ0 + σΘ0 ξ)e (C −σC ζ)t .
                                             ¯                            ¯
Post-processing

                            The exact mean solution in this case is:
Resolution for a                          Z ∞ Z 1                                    2
general SPDE                                                                    e (−ξ /2)  1
Stochastic Galerkin             Θ(t) =              (Θ0 + σΘ0 ξ) e −(C +σC ζ)t × √        × d ζd ξ
Method (SGM)
                                            −∞ −1                                   2π     2
Stochastic Collocation
                                                                           ∗            ∗
Method (SCM)
                                                  (e 2t − 1) e −t                           (C /σC +1)
                                                 =       Θ0 /2                                           .
Multivariate                                                 t∗
quadratures
Full
                            The exact variance solution is:
                                                              Θ2 − Θ            2
Sparse
                               var [Θ] (t)           =
                                                                       2                        ∗              ∗
Intrude or not
                                                                                    !
                                                                      2
                                                                Θ0 + σΘ0                    (e 4t − 1) e −2t       (C /σC +1)
                                                                                                                                , t ∗ = σC t.
intrude ?
                                                     =                                  ×
                                                                   4                                    t∗
Open issues


                         Introduction to Stochastic Spectral Methods                                25 Novembre 2008                       56 / 133
                                   Mixed basis (3)
Introduction
RVs and RPs
                                          ¯                          ¯
                            Results with (C , σC ) = (1/6, 1/3) and (Θ0 , σΘ0 ) = (1/3, 1/4) :
Several UQ methods
                                         0.7                                                                                                                      0.35
                                                                                                                                                                                                                      !0
                                         0.6                                                                                                                       0.3                                                !
Spectral                                                                                                                                                                                                               1
                                                                                                                                                                                                                      !2
representation                           0.5                                                                                                                      0.25
                                                                                                                                                                                                                      !3
                                         0.4                                                                                                                       0.2                                                !4
           e
Karhunen-Lo`ve
                                         0.3                                                                                                                      0.15                                                !




                                                                                                                                                         Modes
                                                                                                                                                                                                                       5
Polynomial Chaos
                                    !




                                         0.2                                                                                                                       0.1
generalized Polynomial
Chaos                                    0.1                                                                                                                      0.05

Post-processing                           0                                                                                                                         0

                                        !0.1                                                                                                                     !0.05

                                        !0.2                                                                                                                      !0.1
Resolution for a                            0   0.5   1   1.5   2   2.5   3                                    3.5          4                                         0           0.5   1   1.5   2   2.5   3   3.5        4
                                                                t                                                                                                                                 t
general SPDE
Stochastic Galerkin
                                Time evolution of random realizations                                                                                   Convergence rate of the variance vs.
Method (SGM)                                                                                                                                                   polynomial order P
Stochastic Collocation                                                                                        10
                                                                                                                   0

Method (SCM)                                                                                                                                                             t=1.0
                                                                                                                                                                         t=2.0
                                                                                                                                                                         t=3.0
                                                                                                                                                                         t=4.0
                                                                              L !norm error of the variance




Multivariate                                                                                                       !5
                                                                                                              10
quadratures
Full

Sparse                                                                                                             !10
                                                                                                              10
                                                                              2




Intrude or not
intrude ?                                                                                                          !15
                                                                                                              10
                                                                                                                        1   2   3   4   5       6   7     8          9           10
                                                                                                                                            P

Open issues                                               Convergence rate of the variance vs. polynomial order P

                         Introduction to Stochastic Spectral Methods                                                                                     25 Novembre 2008                                                      57 / 133
                                   gPC for arbitrary distributions (1)
Introduction
RVs and RPs

Several UQ methods
                                    The idea is to build a space of solutions with functions orthogonals with
Spectral
representation
                                    respect to an arbitrary measure.
           e
Karhunen-Lo`ve

Polynomial Chaos                    Orthogonal polynomials satisfy a three-term recurrence relation,
generalized Polynomial
Chaos

Post-processing                            πi+1 (X )          =        (X − αi )πi (X ) − βi πi−1 (X ),         i = 0, 1, · · · , Np

Resolution for a
                                              π0 (X )         =        1,   π−1 (X ) = 0,
general SPDE
Stochastic Galerkin                 where {πi (X )} is a set of (monic) orthogonal polynomials,
Method (SGM)


                                                              πi (X ) = X i + O(X i−1 )
Stochastic Collocation
Method (SCM)                                                                                   i = 0, 1, · · · Np
Multivariate
quadratures
                                    and the coefficients αi and βi are uniquely determined by a positive
Full                                measure, which corresponds to our probability measure.
Sparse



Intrude or not
                                    For continuous measure : the Stieltjes procedure and the modified
intrude ?                           Chebyshev algorithm (less stable).
Open issues


                         Introduction to Stochastic Spectral Methods                         25 Novembre 2008                          58 / 133
                                   gPC for arbitrary distributions (2)
Introduction
RVs and RPs

Several UQ methods
                            The Stieltjes procedure uses the Darboux’s formulae to compute the
Spectral                    coefficients αi and βi :
representation
Karhunen-Lo`ve
           e                                                             (X πi , πi )
Polynomial Chaos                                                  αi =                , i = 0, 1, · · · Np
generalized Polynomial
                                                                          (πi , πi )
Chaos

Post-processing             and
                                                                                     (πi , πi )
Resolution for a
                                                    β0 = (π0 , π0 ),        βi =                  , i = 1, 2, · · · ,
general SPDE
                                                                                   (πi−1 , πi−1 )
Stochastic Galerkin
Method (SGM)                where (·, ·) denotes the inner product in terms of the measure ρ(X ).
Stochastic Collocation
Method (SCM)



Multivariate
                                    The inner product can be evaluated by Gauss-type quadrature rule.
quadratures
Full

Sparse                              Such a quadrature rule can be regarded as a discrete measure that
                                    yields the corresponding discrete versious of these procedures.
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                          25 Novembre 2008          59 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        60 / 133
                                   Post-processing (1)
Introduction
RVs and RPs

Several UQ methods
                                    Once the modal coefficients are computed, it is then possible to perform
                                    a number of analytical operations on the stochastic solution.
Spectral
representation                      Moments, sensitivity analysis, confidence intervals and pdf of the
Karhunen-Lo`ve
           e
                                    solution can be easily evaluated due to the orthogonality of the modes.
Polynomial Chaos

generalized Polynomial
Chaos
                                    Covariance between two fields u(x , t, ω) and v (x , t, ω) :
Post-processing



Resolution for a
                                    Ruv (x1 , x2 , t)           =      < u(x1 , t, ω)− < u(x1 , t, ω) >, v (x2 , t, ω)− < v (x2 , t, ω) >>
general SPDE                                                           M
                                                                       X
Stochastic Galerkin
Method (SGM)                                                    =             uk (x1 , t)ˆk (x2 , t) < Φ2 >
                                                                              ˆ          v              k
Stochastic Collocation                                                 k =0
Method (SCM)



Multivariate
                                    Auto-correlation Ru :
quadratures
Full                                                   Ru (x1 , x2 , t)        =    < u(x1 , t, ω), u(x2 , t, ω) >
Sparse
                                                                                    M
                                                                                    X
Intrude or not                                                                 =           uk (x1 , t)ˆk (x2 , t) < Φ2 >
                                                                                           ˆ          u              k
intrude ?
                                                                                    k =0

Open issues


                         Introduction to Stochastic Spectral Methods                             25 Novembre 2008                61 / 133
                                   Post-processing (2)
Introduction
RVs and RPs

Several UQ methods                  Expected values:

Spectral
representation                          1. Mean :
Karhunen-Lo`ve
           e                                                                                        ˆ
                                                                             µu = E[u(x , t, X )] = u0
Polynomial Chaos

generalized Polynomial
Chaos
                                        2. Variance :
                                                                                                  M
                                                                                                  X
Post-processing                                                         2
                                                                       σu = E[u(x , t, X )2 ] =          ˆ2
                                                                                                         uk E[Φ2 ]
                                                                                                               k
Resolution for a                                                                                  k =1
general SPDE
Stochastic Galerkin
Method (SGM)
                                        3. Skewness :
Stochastic Collocation                                                                    M M M
                                                                1                      1 XX X
                                                                   E[u(x , t, X )3 ] = 3
Method (SCM)
                                                      δu =       3
                                                                                                       ˆ ˆ ˆ
                                                                                                       ui uj uk E[Φi Φj Φk ]
                                                               σu                     σu i=1 j =1 k =1
Multivariate
quadratures
Full
                                        4. Kurtosis :
Sparse
                                                                                    M M M M
                                                          1                      1 XX XX
Intrude or not                                 κu =        4
                                                             E[u(x , t, X )4 ] = 4                   ˆ ˆ ˆ ˆ
                                                                                                     ui uj uk ul E[Φi Φj Φk Φl ]
intrude ?                                                σu                     σu i=1 j =1 k =1 l=1

Open issues


                         Introduction to Stochastic Spectral Methods                        25 Novembre 2008                   62 / 133
                                   Post-processing (3)
Introduction
RVs and RPs

Several UQ methods                  Sensitivity analysis :
Spectral                                                                                                     2
                                             Variance-based : Sobol’ sensitivity indices Si = var(E[u|Xi ])/σu
representation
Karhunen-Lo`ve
           e
                                             (analytically computed from gPC coefficients)
Polynomial Chaos

generalized Polynomial
Chaos
                                    Example :
                                                               5
                                                               X
Post-processing
                                    u(X1 , X2 )         =              ˆ
                                                                       uk φk (X1 , X2 )
                                                               k =0
Resolution for a
                                                                                              2                             2−1
general SPDE                                            =      u0 + u1 X1 + u2 X2 + u3 (X1 − 1) + u4 X1 X2 + u5 (X2
                                                               ˆ    ˆ       ˆ       ˆ             ˆ          ˆ                    )
Stochastic Galerkin
Method (SGM)
                                                               M
                                                  2
                                                               X        2     2
Stochastic Collocation
                                         =⇒ σu          =              ˆ
                                                                       uk E[Φk ]
Method (SCM)                                                   k =1
                                                                                     2    2                      2−1    2                        2
Multivariate
                                         =⇒             =      E[(ˆ1 X1 + u3 (X1 − 1)) ] + E[(ˆ2 X2 + u5 (X2
                                                                  u       ˆ                   u       ˆ                          u
                                                                                                                       )) ] + E[(ˆ4 X1 X2 )
quadratures
Full
                                         =⇒       1     =      S1 + S2 + S12
Sparse



Intrude or not
intrude ?
                            Hierarchical nature of gPC is an advantage here.
Open issues


                         Introduction to Stochastic Spectral Methods                          25 Novembre 2008                        63 / 133
                                   Post-processing (4)
Introduction
RVs and RPs

Several UQ methods
                                    Distributions and conditional densities :

Spectral                                1. Histogram
representation
Karhunen-Lo`ve
           e
                                        2. Kernel-smoothing density estimate
Polynomial Chaos                                            X       fX (Xn )
generalized Polynomial                  3. fu (x , t, x ) =   ˛ ∂u(x ,t,X )        ˛ with Xn roots of
Chaos
                                                            n
                                                              ˛
                                                                   ∂X
                                                                            |X =Xn ˛
Post-processing
                                                            M
                                                            X
Resolution for a                           u(x , t, X ) =     ˆ
                                                              uk Φk = x .
general SPDE                                                     i=0
Stochastic Galerkin
Method (SGM)

Stochastic Collocation              Reliability analysis :
Method (SCM)

                                        1. Probability failure Pf of u:
Multivariate                                    Z
quadratures
Full
                                           Pf =      fX (X )dX = E[1D (u)] with D = {G(X ) = R−u(x , t, X ) < 0}
                                                        D
Sparse



Intrude or not                          2. α-Quantile uα :
intrude ?                                                                                                 `        ´
                                                P(u(x , t, X ) ≤ uα (x , t) = α i.e. uα = inf {u(x , t), F u(x , t) > α}
Open issues


                         Introduction to Stochastic Spectral Methods                  25 Novembre 2008                64 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        65 / 133
                                   Computational approaches
Introduction
RVs and RPs

Several UQ methods
                                    Discretize spatial (and temporal) domains (e.g. finite differences, finite
Spectral
                                    elements, spectral methods)
representation
Karhunen-Lo`ve
           e                        Numerically represent with a finite number of RVs the random fields
Polynomial Chaos
                                    which are input to the model (e.g. KL expansion, PC/gPC expansion)
generalized Polynomial
Chaos

Post-processing                     Compute solution/statistics/functionals of the solution:
Resolution for a                             via Perturbation method : stochasticity considered as small perturbation
general SPDE
                                             around the mean value, then Taylor expand and truncate.
Stochastic Galerkin
Method (SGM)                                 ’directly’ via sampling-based high-dimensional numerical integration (e.g.
Stochastic Collocation
Method (SCM)
                                             Monte-Carlo & variants, FORM, Probabilistic Collocation Method
                                             (PCM), Stochastic Collocation Method (SCM)). Needs realizations at
Multivariate                                 collocation points.
quadratures
Full
                                             build a surrogate model of the solution (response surface) and then
Sparse
                                             integrate it numerically (cheap). Needs realizations at collocation points.
                                             PC/gPC representations are one possibility.
Intrude or not
intrude ?
                                             Stochastic Galerkin projection on PC/gPC space.

Open issues


                         Introduction to Stochastic Spectral Methods                25 Novembre 2008                66 / 133
                                   Stochastic differential equation
Introduction
RVs and RPs

Several UQ methods
                                                                     `        ´
                            We consider a complete probability space: Ω, A, P , where Ω is the event
Spectral
representation              space, A ⊂ 2Ω the σ-algebra and P the probability measure.
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial              Find u(x , t, ω) with t ∈ [0, T ], ω ∈ Ω, such that:
Chaos

Post-processing
                                                          L(x , t, ω; u)   =   f (x , t, ω)   with      x ∈ D,
Resolution for a
general SPDE
                                                          B(x , t, ω; u)   =   g(x , t, ω)    with      x ∈ ∂D.
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)
                                    Random inputs ← L (linear or non-linear operator), B (boundary
                                    operator), f , g, D ⊂ Rd bounded domain, random parameter R, ...
Multivariate
quadratures
Full
                                    We assume that the physical boundary δD and the forcing terms (f , g)
Sparse                              are sufficiently regular and smooth such that the stochastic problem
                                    aforementioned is well posed.
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                     25 Novembre 2008         67 / 133
                                   Strong & weak form of the stochastic ODE/PDE (1)
Introduction
RVs and RPs

Several UQ methods
                                    Finite dimensional noise assumption:
Spectral
                                    R(ω) = R(X1 (ω), X2 (ω), . . . , XN (ω))
representation
Karhunen-Lo`ve
           e
                                    Each random variable is a function Xi : ω ∈ Ω → R
Polynomial Chaos                    u(x , t, ω) ≈ u(x , t, X1 (ω), X2 (ω), . . . , XN (ω)) (Doob-Dynkin lemma)
generalized Polynomial
Chaos

Post-processing
                                    X (ω) = (X1 (ω), X2 (ω), . . . , XN (ω)): set of i.i.d continuous random
Resolution for a                    variables with PDF ρ and support Γ :
general SPDE
Stochastic Galerkin                                                                        N
                                                                                           Y                     N
                                                                                                                 Y
                                                                                                                       Xi (Ω) ⊂ RN .
Method (SGM)

Stochastic Collocation
                                     ρ(X ) = ρ1 (X1 )ρ2 (X2 ) · · · ρN (XN ) =                   ρi (Xi ), Γ ≡
Method (SCM)
                                                                                           i=1                   i=1

Multivariate
quadratures                         Strong form: find u(x , t, X ) from the (N + d )-dimensional differential
Full

Sparse
                                    system, such that:

Intrude or not                                   L(x , t, X ; u)       =   f (x , t, X )     with      (x , X ) ∈ D × Γ
intrude ?
                                                 B(x , t, X ; u)       =   g(x , t, X )      with      (x , X ) ∈ ∂D × Γ
Open issues


                         Introduction to Stochastic Spectral Methods                         25 Novembre 2008                   68 / 133
                                   Strong & weak form of the stochastic ODE/PDE (2)
Introduction
RVs and RPs

Several UQ methods



Spectral
                                    Finite dimensional subspace VΓ ⊂ L2 (Γ) of all square integrable
                                                                        ρ
representation                      function in Γ with respect to the measure ρ(X )d X .
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial
Chaos

Post-processing
                                    Weak form: find uV (x , t, X ) ∈ VΓ (X ), such that:

Resolution for a
general SPDE                        ∀ φ(X ) ∈ VΓ , x ∈ D :
Stochastic Galerkin                     Z                                  Z
Method (SGM)

Stochastic Collocation                    L(x , t, X ; uV )φ(X )ρ(X )d X =   f (x , t, X )φ(X )ρ(X )d X ,
Method (SCM)
                                             Γ                               Γ

Multivariate
quadratures                         ∀ φ(X ) ∈ VΓ , x ∈ ∂D :
Full
                                        Z                                  Z
Sparse
                                          B(x , t, X ; uV )φ(X )ρ(X )d X =   g(x , t, X )φ(X )ρ(X )d X .
Intrude or not                               Γ                               Γ
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods        25 Novembre 2008                69 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        70 / 133
                                   Stochastic Galerkin projection (1)
Introduction
RVs and RPs

Several UQ methods                  The Galerkin projection is a standard procedure in spectral methods :
Spectral                                1. The differents terms are spectrally expanded to a certain order.
representation
Karhunen-Lo`ve
           e                            2. The obtained equation are projected onto the spectral basis.
Polynomial Chaos

generalized Polynomial
Chaos
                                    gPC decomposition :
Post-processing

                                                                       M
                                                                       X                                       (N + P )!
Resolution for a
general SPDE
                                                 u(x , t, ω) =                ˆ
                                                                              uk (x , t)Φk (X ) with M =                 −1
                                                                       k =0
                                                                                                                 N ! P!
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)
                                    We assume the deterministic coefficient can be themselves treated with
Multivariate
                                    an appropriate representation.
quadratures
Full

Sparse
                                    For instance, if the evolution problem is normally space-discretized :
                                                                                      X
Intrude or not
intrude ?
                                                                       ˆ
                                                                       uk (x , t) =        ˜l
                                                                                           uk (t)Nl (x )
                                                                                       l
Open issues


                         Introduction to Stochastic Spectral Methods                            25 Novembre 2008              71 / 133
                                   Stochastic Galerkin projection (2)
Introduction
RVs and RPs

Several UQ methods                  Substitute in the weak form of the model problem. For i = 0, 2, . . . M :
                                            Z               M
Spectral
                                                 `       ´ X                    ´
representation                                 L x , t, X ;    ˆ
                                                               uk (x , t)Φk (X ) Φi (X )ρ(X )d X
           e
Karhunen-Lo`ve
                                                    Γ                          k =0
Polynomial Chaos
                                                                       Z
generalized Polynomial                                            =            f (x , t, X )Φi (X )ρ(X )d X .
Chaos
                                                                           Γ
Post-processing



Resolution for a
                                    Orthogonality condition ⇒ system of (M + 1) deterministic equations
general SPDE                            ˆ
                                    for ui (x , t).
Stochastic Galerkin
Method (SGM)                                 System is coupled unless the problem is linear (in random space).
Stochastic Collocation                                               ˆ
                                             Implicitly accounts for uk coupling
Method (SCM)
                                             Standard numerical method can be used to solve.
Multivariate
quadratures                                  Requires the resolution of a large system (integrals of residuals).
Full                                         Problems for non-linearities.
Sparse



Intrude or not
                                    Variations of the stochastic Galerkin method when poor convergence
intrude ?                           (discontinuity, stochastic bifurcation, long-time integration).
Open issues


                         Introduction to Stochastic Spectral Methods                              25 Novembre 2008   72 / 133
                                   Example 1 : 1st order stochastic ODE with random
Introduction
                                   variables decay rate and initial condition (1)
RVs and RPs

Several UQ methods
                                           dΘ
                                              = −C Θ; Θ (t = 0) = Θ0 , with C > 0 and t ∈ [0, T ] .
Spectral
representation
                                           dt
           e
Karhunen-Lo`ve

Polynomial Chaos                    The decay rate coefficient C and the initial condition Θ0 are considered
generalized Polynomial
Chaos                               to be random inputs to the system (uniform RV ζ = (ζ1 , ζ2 )).
Post-processing



Resolution for a                    The stochastic counterpart of the equation becomes non-linear due to
general SPDE
                                    the product between the random quantity C and the solution Θ.
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)                                  ¯                        ¯
                                    C (ζ1 ) = C + σC ζ1 and Θ0 (ζ2 ) = Θ0 + σΘ0 ζ2 where (σC , σΘ0 ) are the
                                    standard deviations of each random variables respectively.
Multivariate
quadratures
Full

Sparse
                                    The deterministic and stochastic analytic solutions are, respectively:
                                                           ¯                                             ¯
Intrude or not                                Θ(t) = Θ0 e −C t
                                                     ¯                 and              ¯
                                                                             Θ(t, ω) = (Θ0 + σΘ0 ζ2 ) e (C +σC ζ1 )t .
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                    25 Novembre 2008                 73 / 133
                                   Example 1 : 1st order stochastic ODE with random
Introduction
                                   variables decay rate and initial condition (2)
RVs and RPs

Several UQ methods



Spectral                    The exact mean solution in this case is :
representation
           e
Karhunen-Lo`ve                               Z 1 Z 1
                                                                                     1 1
Polynomial Chaos
                                  Θ(t) =              (Θ0 + σΘ0 ζ2 )e −(C +σC ζ1 )t × × d ζ1 d ζ2
generalized Polynomial
Chaos                                         −1 −1                                  2 2
                                                                          ∗
Post-processing
                                                                 e −C t       /σC
                                                                                     sinh (t ∗ )
                                                    =       Θ0                                     .
Resolution for a                                                                t∗
general SPDE
Stochastic Galerkin
Method (SGM)
                            The exact variance solution is :
Stochastic Collocation
Method (SCM)

                                           var [Θ] (t)           =        Θ2 − Θ           2
Multivariate
                                                                                    2                                ∗
quadratures
                                                                                               !
                                                                                     2
                                                                              3Θ0 + σΘ0                    e −2C t       /σC
                                                                                                                               sinh (2t ∗ )
Full
                                                                 =                                     ×                                      ,
Sparse                                                                            6                                        t∗

                            where t ∗ = σC t.
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                                       25 Novembre 2008                       74 / 133
                                   Example 1 : 1st order stochastic ODE with random
Introduction
                                   variables decay rate and initial condition (3)
RVs and RPs

Several UQ methods



Spectral
representation              The weak form of the problem consists in:
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial      Find ΘV (t, ζ) ∈ VΓ (ζ) ⊂ L2 (Γ), such that:
                                                       ρ
Chaos

Post-processing



Resolution for a            ∀ φ(ζ) ∈ VΓ and t ∈ ]0, T ] :
general SPDE
Stochastic Galerkin
                                         Z                   Z
Method (SGM)
                                            ˙
                                           ΘV φ(ζ)ρ(ζ)d ζ = − C (ζ)ΘV φ(ζ)ρ(ζ)d ζ,
Stochastic Collocation
Method (SCM)                                            Γ                        Γ

Multivariate
quadratures                 ∀ φ(ζ) ∈ VΓ :
Full

Sparse
                                                    Z                                Z
                                                            ΘV (0, ζ)φ(ζ)ρ(ζ)d ζ =           Θ0 (ζ)φ(ζ)ρ(ζ)d ζ.
Intrude or not                                          Γ                                Γ
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                         25 Novembre 2008     75 / 133
                                   Example 1 : 1st order stochastic ODE with random
Introduction
                                   variables decay rate and initial condition (4)
RVs and RPs

Several UQ methods
                            The Legendre polynomial basis is well fitted to uniform distributions.
Spectral
representation
                            We make the choice to use Legendre polynomials to represent both our
           e
Karhunen-Lo`ve

Polynomial Chaos
                            random inputs and stochastic solution.
generalized Polynomial
Chaos
                                                  M
                                                  X
Post-processing                    C (ω) =              ˆ                  ˆ                  ˆ                  ¯
                                                        Ci Li (ζ1 , ζ2 ) = C0 L0 (ζ1 , ζ2 ) + C1 L1 (ζ1 , ζ2 ) = C + σC ζ1
                                                  i=0
Resolution for a
general SPDE                   ˆ    ¯ ˆ             ˆ
                            =⇒ C0 = C , C1 = σC and Ci = 0 for i > 1.
Stochastic Galerkin
Method (SGM)

Stochastic Collocation                              M
Method (SCM)                                        X
                              Θ0 (ω)         =             ˆ
                                                           Θ0l Ll (ζ1 , ζ2 )
Multivariate                                        l=0
quadratures
Full                                         =      ˆ                   ˆ                   ˆ                   ¯
                                                    Θ00 L0 (ζ1 , ζ2 ) + Θ01 L1 (ζ1 , ζ2 ) + Θ02 L2 (ζ1 , ζ2 ) = Θ0 + σΘ0 ζ2
Sparse
                               ˆ     ¯ ˆ           ˆ             ˆ
                            =⇒ Θ00 = Θ0 , Θ01 = 0, Θ02 = σΘ0 and Θ0i = 0 for i > 2.
Intrude or not
intrude ?                                                                         M
                                                                                  X
                            For the solution, we write : Θ(t, ω) =                       ˆ
                                                                                         Θj (t)Lj (ζ1 , ζ2 )
Open issues
                                                                                  j =0

                         Introduction to Stochastic Spectral Methods                        25 Novembre 2008                 76 / 133
                                     Example 1 : 1st order stochastic ODE with random
Introduction
                                     variables decay rate and initial condition (5)
RVs and RPs

Several UQ methods

                            We inject all expansions into the following weak form :
Spectral                    Z        Z                                                 Z         Z
representation                                ˙
                                              ΘV L(ζ1 , ζ2 )ρ(ζ1 , ζ2 )dζ1 dζ2 = −                     C (ζ1 , ζ2 )ΘV L(ζ1 , ζ2 )ρ(ζ1 , ζ2 )dζ1 dζ2 ,
           e
Karhunen-Lo`ve                  ζ1       ζ2                                                 ζ1    ζ2
Polynomial Chaos            Z        Z                                                               Z      Z
generalized Polynomial
Chaos
                                              ΘV (0, ζ1 , ζ2 )L(ζ1 , ζ2 )ρ(ζ1 , ζ2 )dζ1 dζ2 =                     Θ0 (ζ1 , ζ2 )L(ζ1 , ζ2 )ρ(ζ1 , ζ2 )dζ1 dζ2 ,
                             ζ1      ζ2                                                                ζ1    ζ2
Post-processing

                            for all L(ζ1 , ζ2 ) ∈ VΓ and t ∈ [0, T ].
Resolution for a
general SPDE
Stochastic Galerkin
Method (SGM)
                            After dropping the (ζ1 , ζ2 )-dependency, for the sake of clarity, we get a
Stochastic Collocation      system of equations :
Method (SCM)                Z Z X   M                              Z Z X   M         M
                                       ˆ˙                                      ˆ
                                                                                     X
                                                                                         ˆ
Multivariate                           Θj (t)Lj Lk ρ d ζ1 d ζ2 = −             Ci Li    Θj (t)Lj Lk ρ d ζ1 d ζ2 ,
quadratures                     ζ1       ζ2 j =0                                            ζ1       ζ2 i=0            j =0
Full

Sparse
                            Z        Z         M
                                               X                                   Z        Z        M
                                                                                                     X
                                                    ˆ
                                                    Θj (0)Lj Lk ρ d ζ1 d ζ2 =                               ˆ
                                                                                                            Θ0l Ll Lk ρ d ζ1 d ζ2 ,
Intrude or not                  ζ1       ζ2 j =0                                       ζ1        ζ2 l=0
intrude ?

                            for k = 0 . . . M and t ∈ [0, T ].
Open issues


                         Introduction to Stochastic Spectral Methods                                     25 Novembre 2008                          77 / 133
                                   Example 1 : 1st order stochastic ODE with random
Introduction
                                   variables decay rate and initial condition (6)
RVs and RPs

Several UQ methods



Spectral
                            System of (M + 1) coupled ODEs (due to the orthogonality of the basis) :
representation
Karhunen-Lo`ve
           e                For k = 0 . . . M ,
Polynomial Chaos                                                       M    M
                                                   ˙
                                                   ˆ
                                                                 XX
                                                                    ˆ ˆ
generalized Polynomial
Chaos                                              Θk (t) L2 = −
                                                           k        Ci Θj (t) Li Lj Lk for t ∈ ]0, T ]
Post-processing                                                        i=0 j =0

Resolution for a                                   ˆ        ˆ
                                                   Θk (0) = Θ0k
general SPDE
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
                                    The inner product quantities L2 and Li Lj Lk involve
                                                                     k
Method (SCM)                        multi-dimensional integrals of know polynomials over know supports
Multivariate
                                    and can be numerically computed and tabulated for each type of
quadratures                         distribution prior to the numerical simulations.
Full

Sparse
                                                         ˆ
                                    The random modes Θk are smooth and thus any standard ordinary
Intrude or not                      differential equation solver can be employed here. In the following, the
intrude ?
                                    standard fourth-order Runge-Kutta scheme is used.
Open issues


                         Introduction to Stochastic Spectral Methods              25 Novembre 2008       78 / 133
                                   Example 1 : 1st order stochastic ODE with random
Introduction
                                   variables decay rate and initial condition (7)
RVs and RPs

Several UQ methods
                                          ¯                          ¯
                            Results with (C , σC ) = (1/6, 1/3) and (Θ0 , σΘ0 ) = (1/3, 1/3).
Spectral                                                                                                                                                                                                          !0
                                                   0.5                                                                                   0.4
representation                                                                                                                                                                                                    !1
                                                  0.45                                                                                                                                                            !
                                                                                                                                         0.3                                                                           2
           e
Karhunen-Lo`ve                                     0.4                                                                                                                                                            !
                                                                                                                                                                                                                       3

                                                                                                                                         0.2                                                                      !4
Polynomial Chaos                                  0.35
                                                                                                                                                                                                                  !5
                                                   0.3
generalized Polynomial                                                                                                                   0.1




                                                                                                             Modes
                                                  0.25
                                              !




Chaos
                                                                                                                                                0
                                                   0.2
Post-processing
                                                  0.15                                                                       !0.1
                                                   0.1
                                                                                                                             !0.2
                                                  0.05
Resolution for a
                                                    0
general SPDE                                         0   0.5   1        1.5   2   2.5   3   3.5   4
                                                                                                                             !0.3
                                                                                                                                 0                           0.5   1   1.5   2       2.5        3   3.5       4
                                                                              t                                                                                              t
Stochastic Galerkin
Method (SGM)                           Time evolution of random realizations                          Time evolution of gPC modes of the solution
Stochastic Collocation                             0.5                                                                                          10
                                                                                                                                                    0

                                                                                                                                                                                                          t=1.0
Method (SCM)                                      0.45                                                                                                                                                    t=2.0
                                                                                                                                                                                                          t=3.0
                                                   0.4                                                                                                                                                    t=4.0




                                                                                                                L2!norm Error of the variance
                                                  0.35                                                                                              !5
Multivariate                                                                                                                                    10
                                                   0.3
quadratures
                                                  0.25
                                              !




Full                                               0.2                                                                                              !10
                                                                                                                                                10
Sparse                                            0.15

                                                   0.1

                                                  0.05   MEAN
Intrude or not                                      0
                                                         MEAN +/! STD                                                                           10
                                                                                                                                                    !15

                                                                                                                                                         0             5                   10                     15
                                                     0   0.5   1        1.5   2   2.5   3   3.5   4
intrude ?                                                                     t
                                                                                                                                                                                 P


                                     Time evolution of mean and mean ± std                                Convergence rate of the variance vs.
Open issues                                         solutions                                                    polynomial order P

                         Introduction to Stochastic Spectral Methods                                     25 Novembre 2008                                                                                                  79 / 133
                                   Example 2 : 1st order stochastic ODE with random
Introduction
                                   process decay rate (1)
RVs and RPs

Several UQ methods



Spectral
                                                     dΘ
representation
                                                        = −C Θ; Θ (t = 0) = Θ0 , and t ∈ [0, T ] ,
Karhunen-Lo`ve
           e                                         dt
Polynomial Chaos

generalized Polynomial
Chaos
                                    The decay rate is a stationary second-order Gaussian RP with
Post-processing
                                                                  ¯
                                    exponential kernel and mean C and std σC .
Resolution for a
general SPDE
Stochastic Galerkin
                                    N -terms KL representation of the decay rate :
Method (SGM)

Stochastic Collocation                                                              N
                                                                                    X√
Method (SCM)
                                                              C N (t, ω) = C + σC
                                                                           ¯          λi φi (x )Xi (ω),
Multivariate                                                                        i=1
quadratures
Full
                                    We choose for the following results: C = 1.0 and σC = 1/3.
Sparse



Intrude or not
intrude ?
                                    Different Cl are investigated and convergence rates are evaluated.

Open issues


                         Introduction to Stochastic Spectral Methods                      25 Novembre 2008   80 / 133
                                   Example 2 : 1st order stochastic ODE with random
Introduction
                                   process decay rate (2)
RVs and RPs

Several UQ methods                                2                                                                            3

                                                 1.5
                                                                                                                               2


                                          C(t)




                                                                                                                        C(t)
Spectral                                          1
representation                                                                                                                 1
                                                 0.5

           e
Karhunen-Lo`ve                                    0                                                                            0
                                                   0   0.2    0.4       0.6            0.8    1                                 0    0.2     0.4       0.6   0.8   1
                                                                    t                                                                              t
Polynomial Chaos
                                                  1                                                                            1
generalized Polynomial
Chaos
                                          !(t)




                                                                                                                     !(t)
                                                 0.5                                                                        0.5
Post-processing


                                                  0                                                                            0
Resolution for a                                   0   0.2    0.4       0.6            0.8    1                                 0    0.2     0.4       0.6   0.8   1
                                                                    t                                                                              t
general SPDE
Stochastic Galerkin
                                                             Cl = 1                                                                        Cl = 0.5
Method (SGM)
                                                                                      3
Stochastic Collocation
                                                                                      2
Method (SCM)
                                                                              C(t)



                                                                                      1

                                                                                      0
Multivariate
quadratures                                                                          !1
                                                                                       0     0.2     0.4       0.6             0.8    1
                                                                                                           t
Full
                                                                                      1
Sparse
                                                                              !(t)




                                                                                     0.5

Intrude or not
intrude ?
                                                                                      0
                                                                                       0     0.2     0.4       0.6             0.8    1
                                                                                                           t

Open issues                                                                                        Cl = 1/3

                         Introduction to Stochastic Spectral Methods                                             25 Novembre 2008                                  81 / 133
                                   Example 2 : 1st order stochastic ODE with random
Introduction
                                   process decay rate (3)
RVs and RPs

Several UQ methods                                         T=1, Cl=0.5, N=5, P=5, E[C]=1, Var[C]=0.33333                                                                      1
                                                    1.2                                                                                                                                                               Mean
                                                                                                                                          m=0                                                                         Mean ± Std
                                                                                                                                          m=1
                                                                                                                                                                             0.9
                                                     1
Spectral                                                                                                                                  m=2
                                                                                                                                          m=3                                0.8
representation                                      0.8
                                                                                                                                          m=4
                                                                                                                                          m=5
                                                                                                                                                                             0.7
           e
Karhunen-Lo`ve
                                                    0.6




                                                                                                                                                                      !(t)
                                           !m(t)


                                                                                                                                                                             0.6
Polynomial Chaos
                                                    0.4
generalized Polynomial                                                                                                                                                       0.5
Chaos                                               0.2
                                                                                                                                                                             0.4
Post-processing
                                                     0                                                                                                                       0.3

                                                   !0.2                                                                                                                      0.2
Resolution for a                                       0      0.2        0.4         0.6                                        0.8             1                               0              0.2   0.4       0.6   0.8           1
                                                                                 t                                                                                                                         t
general SPDE
                                         Time evolution of a few gPC modes                                                                                  Time evolution of the mean and mean ±std
Stochastic Galerkin
Method (SGM)                                                                                                                                                            values for Cl = 0.5
                                                                                                                                 0
Stochastic Collocation                                                                                                          10
                                                                                                                                                                                    Cl = 1
Method (SCM)                                                                                                                                                                        Cl = 1/2
                                                                                                                                                                                    Cl = 1/3
                                                                                             Relative L2!norm error of kernel




                                                                                                                                                                                    Cl = 1/4

Multivariate                                                                                                                     !1
                                                                                                                                10
quadratures
Full

Sparse                                                                                                                           !2
                                                                                                                                10



Intrude or not
intrude ?                                                                                                                        !3
                                                                                                                                10
                                                                                                                                      0             2   4       6             8            10
                                                                                                                                                            N

Open issues                                                                          Convergence of the KL exponential kernel

                         Introduction to Stochastic Spectral Methods                                                                                                25 Novembre 2008                                               82 / 133
                                   Example 2 : 1st order stochastic ODE with random
Introduction
                                   process decay rate (4)
RVs and RPs

Several UQ methods
                                                                       !1                                                                           N=1                                                        10
                                                                                                                                                                                                                   !1                                                  N=1
                                                                      10                                                                            N=2                                                                                                                N=2
                                                                                                                                                    N=3                                                                                                                N=3
Spectral                                                                                                                                            N=4                                                                                                                N=4

                                          L2!norm Error of Variance




                                                                                                                                                                                   L2!norm Error of Variance
                                                                       !2                                                                           N=5                                                                                                                N=5
representation                                                        10                                                                                                                                       10
                                                                                                                                                                                                                   !2



           e
Karhunen-Lo`ve
                                                                       !3
                                                                      10                                                                                                                                           !3
Polynomial Chaos                                                                                                                                                                                               10

generalized Polynomial                                                 !4
Chaos                                                                 10                                                                                                                                           !4
                                                                                                                                                                                                               10
Post-processing
                                                                       !5
                                                                      10
                                                                                                                                                                                                                   !5
                                                                                                                                                                                                               10

Resolution for a                                                           1   1.5   2   2.5   3   3.5                               4        4.5         5                                                         1         1.5       2    2.5   3   3.5   4   4.5         5
                                                                                               P                                                                                                                                                   P
general SPDE
Stochastic Galerkin
                                                                                         Cl = 1                                                                                                                                             Cl = 0.5
Method (SGM)
                                                                                                                                         !1                                                                                   N=1
                                                                                                                                     10
Stochastic Collocation                                                                                                                                                                                                        N=2
Method (SCM)                                                                                                                                                                                                                  N=3
                                                                                                                                                                                                                              N=4
                                                                                                         L2!norm Error of Variance




                                                                                                                                                                                                                              N=5
                                                                                                                                         !2
                                                                                                                                     10
Multivariate
quadratures
                                                                                                                                         !3
Full                                                                                                                                 10

Sparse

                                                                                                                                         !4
                                                                                                                                     10
Intrude or not
intrude ?                                                                                                                                 1         1.5       2    2.5   3   3.5                               4        4.5         5
                                                                                                                                                                         P


Open issues                                                                                                                                                       Cl = 1/3

                         Introduction to Stochastic Spectral Methods                                                                                                         25 Novembre 2008                                                                                83 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        84 / 133
                                   Probabilistic Collocation Method (PCM)                       [Tatang 2004, Babuska
Introduction
                                   2004, Xiu 2005]
RVs and RPs

Several UQ methods

                                    The goal is the prediction of statistical moments of the solution (mean
Spectral
representation
                                    value, variance, covariance, etc.) or statistics of some given response.
           e
Karhunen-Lo`ve

Polynomial Chaos
                                    The solution may have a very regular dependence on the input RVs
generalized Polynomial
Chaos
                                    (SGM or SCM based on orthogonal tensor product polynomials feature
Post-processing                     a very fast convergence rate)

Resolution for a                    When the number of RVs is small, SCM is a very effective tool.
general SPDE
Stochastic Galerkin
Method (SGM)
                                    But, approximations based on full tensor product grids suffer from the
Stochastic Collocation              curse of dimensionality.
Method (SCM)



Multivariate
                                    Remedy : Use sparse grids in order to reduce the number of collocation
quadratures                         points while maintaining a high level of accuracy.
Full

Sparse
                                    Smolyak-type sparse grid : for some cases, algebraic convergence wrt
Intrude or not
                                    total number of collocation points. The exponent is connected to both
intrude ?                           the regularity of the solution and the number of input RVs, N , and
Open issues
                                    essentially deteriorates with N by a 1/ log N factor.

                         Introduction to Stochastic Spectral Methods         25 Novembre 2008                    85 / 133
                                   PCM - Framework with Lagrange interpolation (1)
Introduction
                            Complete probability space: Ω, A, P , where Ω is the event space, A ⊂ 2Ω
                                                       `         ´
RVs and RPs

Several UQ methods          the σ-algebra and P : A → [0, 1] the probability measure.
Spectral                                          ¯
                                    Find u : Ω × D → R with u(x , t, ω) and t ∈ [0, T ], ω ∈ Ω, D ⊂ Rd
representation
Karhunen-Lo`ve
           e                        such that P -almost everywhere in Ω:
Polynomial Chaos

generalized Polynomial
Chaos
                                                          L(x , t, ω; u)   =   f (x , t, ω)   with      x ∈ D,
Post-processing
                                                          B(x , t, ω; u)   =   g(x , t, ω)    with      x ∈ ∂D.
Resolution for a
general SPDE
                                    Random inputs ← L (linear or non-linear operator), B (boundary
Stochastic Galerkin
Method (SGM)                        operator), f , g, D ⊂ Rd bounded domain, random parameter R, ...
Stochastic Collocation
Method (SCM)
                                    We assume that the physical boundary δD and the forcing terms (f , g)
Multivariate                        are sufficiently regular and smooth such that the stochastic problem
quadratures
Full
                                    aforementioned is well posed.
Sparse
                                    Finite dimensional noise assumption:
Intrude or not                      R(ω) = R(X1 (ω), X2 (ω), . . . , XN (ω)) : Ω → RN .
intrude ?
                                    Each random variable is a function Xi : ω ∈ Ω → R
Open issues                         u(x , t, ω) ≈ uN (x , t, X1 (ω), X2 (ω), . . . , XN (ω)) (Doob-Dynkin lemma)

                         Introduction to Stochastic Spectral Methods                     25 Novembre 2008         86 / 133
                                   PCM - Framework with Lagrange interpolation (2)
Introduction
RVs and RPs

Several UQ methods                  X (ω) = (X1 (ω), X2 (ω), . . . , XN (ω)): set of i.i.d continuous random
                                    variables with PDF:
Spectral
                                    ρ : Γ → R+ with ρ(X ) = ρ1 (X1 )ρ2 (X2 ) · · · ρN (XN ) = N ρi (Xi )
                                                                                                Q
representation
                                                      QN              QN                          i=1
           e
Karhunen-Lo`ve                                                                          N
Polynomial Chaos
                                    with support: Γ ≡ i=1 Γi = i=1 Xi (Ω) ⊂ R .
generalized Polynomial
Chaos

Post-processing
                                    Find uN = uN (x , t, X ) from the (N + d )-dimensional differential
                                    system, such that:
Resolution for a
general SPDE
                                              L(x , t, X ; uN )        =   fN (x , t, X )    with      (x , X ) ∈ D × Γ
Stochastic Galerkin
Method (SGM)
                                              B(x , t, X ; uN )        =   gN (x , t, X )    with       (x , X ) ∈ ∂D × Γ
Stochastic Collocation
Method (SCM)

                                    We introduce the following space: Pp (Γ) ⊂ L2 (Γ) is the span of tensor
                                                                                ρ
Multivariate
quadratures                         product polynomials with degree at most p = (p1 , . . . , pN ), i.e.
                                    Pp (Γ) = N=1 Ppk (Γk ), with:
                                             N
Full
                                               k
Sparse

                                                               n
Intrude or not
                                            Ppk (Γk ) = span(Xk , n = 0, . . . , pk ), k = 1, . . . , N .
intrude ?
                                    Hence the dimension of Pp (Γ) is Np = N=1 (pk + 1).
                                                                            Q
                                                                                k
Open issues


                         Introduction to Stochastic Spectral Methods                        25 Novembre 2008                87 / 133
                                   PCM - Framework with Lagrange interpolation (3)
Introduction
RVs and RPs                 Collocation methods are based on polynomial interpolations of the solution.
Several UQ methods
                                    A nodal set of (Nc + 1) collocation points is defined in the
Spectral                            N -dimensional random space.
representation
Karhunen-Lo`ve
           e                        After collocation projections, the resulting set of deterministic equations
Polynomial Chaos
                                    is always uncoupled and each solution corresponding to each collocation
generalized Polynomial
Chaos                               point of the set is obtained with a deterministic solver.
Post-processing

                                    The continuous solution can then be approximated by interpolation on
Resolution for a
general SPDE
                                    the data points using for instance multi-dimensional tensor product
Stochastic Galerkin                 Lagrange basis.
Method (SGM)

Stochastic Collocation
Method (SCM)
                                    The polynomial interpolation uN of the solution is:
                                                                                      Nc
Multivariate                                                  Y                       X
quadratures                          uN (x , t, X ) =              (u(x , t, X )) =          u(x , t, z k )Lp (X )
                                                                                                            k        and   z k ∈ Γ,
Full
                                                                                      k =0
Sparse



Intrude or not
                                    where the z k are the collocation points and the Lp are the
                                                                                      k
intrude ?                           multi-dimensional Lagrange polynomials in the interpolation space
                                    satisfy Lp (z j ) = δij for 0 ≤ i, j ≤ Nc .
                                             i
Open issues


                         Introduction to Stochastic Spectral Methods                            25 Novembre 2008               88 / 133
                                   PCM - Framework with Lagrange interpolation (4)
Introduction
RVs and RPs

Several UQ methods                  Evaluation of the solution moments requires integrating those Lagrange
                                    basis:
Spectral
representation                                                   Nc               Z
                                                                 X
                                                                    u(x , t, z k ) Lp (X )ρ(X )d X ,
           e
Karhunen-Lo`ve

Polynomial Chaos
                                              E uN (x , t, X ) =                    k
                                                                            k =0                    Γ
generalized Polynomial
Chaos

Post-processing                     This requires the explicit knowledge of their expressions, which can be
Resolution for a
                                    quite cumbersome (especially for N > 1), unless we choose the nodal
general SPDE                        set of collocation points to be a cubature set points.
Stochastic Galerkin
Method (SGM)
                                    By choosing the cubature weight function to coincide with the joint
Stochastic Collocation
Method (SCM)                        density of the random input, the computation of the moments becomes
                                    straightforward. In this case, we have:
Multivariate
quadratures
                                                                                        Nc
Full                                                                                    X
Sparse                                                             E uN (x , t, X ) =          u(x , t, X k )wk ,
                                                                                        k =0
Intrude or not
intrude ?
                                    where the {wk } are the cubature weights.
Open issues


                         Introduction to Stochastic Spectral Methods                            25 Novembre 2008    89 / 133
                                   PCM - Framework with Lagrange interpolation (5)
Introduction
RVs and RPs

Several UQ methods
                            The drawbacks of the Lagrange interpolation are many.
Spectral
representation                      The interpolation error is hard to control.
           e
Karhunen-Lo`ve

Polynomial Chaos
                                    Indeed the interpolation error is uniformly bounded by a quantity that
generalized Polynomial
Chaos                               depends on the choice of the nodal set (through the Lebesgue constant)
Post-processing
                                    and is very difficult to estimate (especially for N > 1).
Resolution for a
general SPDE                        The problem of the interpolation of a Runge function on equally spaced
Stochastic Galerkin
Method (SGM)
                                    grid nodes is very well known.
Stochastic Collocation
Method (SCM)                        In our case we have seen that the interpolation can not be constructed
                                    for any given nodes.
Multivariate
quadratures
Full                                If no particular care is taken to construct the space of approximations,
Sparse                              the computational cost increase exponentially with N due to the way
Intrude or not
                                    the collocation grid is constructed based on full tensor products.
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods         25 Novembre 2008             90 / 133
                                   PCM - Error analysis
Introduction
RVs and RPs

Several UQ methods
                                    If error associated with the representation of the random inputs
                                    (equation parameters and/or forcings) are under control.
Spectral
representation
Karhunen-Lo`ve
           e                        The solution depends analytically on each RV (e.g. elliptic PDEs).
Polynomial Chaos

generalized Polynomial
Chaos                               Smolyak-type sparse grid stochastic collocation method, can provide
Post-processing                     algebraic convergence wrt the total # of collocation points Nc .
Resolution for a
general SPDE                        The Smolyak representation based on Clenshaw-Curtis 1D rule satisfies
Stochastic Galerkin
Method (SGM)
                                    [Nobile 2007] :
Stochastic Collocation
Method (SCM)                                      C1 (σ)                        1                             σ
                                           ≤                e σ max{1, C1 (σ)}N µ      with        µ=
Multivariate
                                               |1 − C1 (σ)|                    Nc                       1 + log(2N )
quadratures
Full

Sparse
                                    The Smolyak representation based on Gauss-type 1D rules satisfies
                                    [Nobile 2007] :
Intrude or not
intrude ?                                         C1 (σ)                           1                        a2 σ
                                         ≤                  e a1 σ max{1, C1 (σ)}N ν     with      ν=
Open issues
                                               |1 − C1 (σ)|                       Nc                    a3 + log(N )

                         Introduction to Stochastic Spectral Methods            25 Novembre 2008                  91 / 133
                                   Pseudo-spectral gPC-based method
Introduction
RVs and RPs
                                    gPC-based collocation method which is a pseudo-spectral method that
Several UQ methods                  uses the gPC polynomial basis.
                                    {φj (ξ(ω))} are mutually orthogonal polynomials in terms of the
Spectral
representation                      zero-mean random vector ξ, satisfying the orthogonality relation :
Karhunen-Lo`ve
           e
                                                                φi , φj = φ2 δij ,
                                                                           i
Polynomial Chaos
                                                                                        XM
generalized Polynomial
Chaos                               The finite-term expansion takes the form :u(x , ξ) =     ˆ
                                                                                            uj (x )φj (ξ).
Post-processing                                                                                j =0
                                    Non-intrusive method : we project the stochastic solution directly onto
Resolution for a
general SPDE                        each member of the orthogonal basis chosen to span the random space
Stochastic Galerkin
Method (SGM)
                                    and has the advantage not to require modifications to the existing
Stochastic Collocation              deterministic solver. `
Method (SCM)                                                 u(x, ξ), φj (ξ)
                                                  ˆ
                                                  uj (x ) =                     for j = 0, ..., M
Multivariate
                                                                 φ2 (ξ)
                                                                  j
quadratures
                                    The evaluation is equivalent to computing multidimensional integrals
Full

Sparse
                                    over the domain Γ.
                                    The global error of the final representation can be seen as a
Intrude or not
intrude ?                           superposition of an aliasing error, a finite-term projection error and a
                                    numerical error due to the intrinsic numerical approximation of the
Open issues
                                    deterministic solver.
                         Introduction to Stochastic Spectral Methods        25 Novembre 2008            92 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        93 / 133
                                   Multivariate quadratures
Introduction
RVs and RPs

Several UQ methods          In general, we need to evaluate N -dimensional integrals of the form:
                                                             Z
Spectral
representation                    Q N [f ] := E (f (X )) =         f (ω)dPX (ω)
Karhunen-Lo`ve
           e                                                   ΩN
                                                Z       Z
Polynomial Chaos

generalized Polynomial                     =        ...     f (ω1 , . . . , ωN ) dPX1 (ω1 ) . . . dPXN (ωN )
Chaos
                                                                Ω1     ΩN
Post-processing



Resolution for a
general SPDE
                                                     N
                                    Approximation QNe [f ] of Q N [f ] by evaluating the integrand at some Ne
Stochastic Galerkin                 sets of quadrature points {Z 1 ∈ ΩN , . . . , Z Ne ∈ ΩN }
Method (SGM)

Stochastic Collocation
Method (SCM)
                                    Combine the results with some appropriate weights :
Multivariate                                                                        Ne
quadratures                                                                         X
                                                                        N
Full                                                                   QNe [f ] =         w i f (Z i )
Sparse                                                                              i=1

Intrude or not              Different ways of dealing with high-dimensional integrations can be
intrude ?
                            considered depending on the prevalence of accuracy versus efficiency.
Open issues


                         Introduction to Stochastic Spectral Methods                        25 Novembre 2008   94 / 133
                                   Monte-Carlo method
Introduction
RVs and RPs

Several UQ methods          The Monte Carlo (MC) method uses Ne independent realizations of RVs X :
                                                                                        Ne
Spectral
                                                                        N           1 X
representation                                                         QNe [f ] =          f (Z i ),
Karhunen-Lo`ve
           e                                                                        Ne i=1
Polynomial Chaos

generalized Polynomial
Chaos                               This estimate converges almost surely to Q N [f ] (law of large numbers).
Post-processing
                                                                     N           −1/2
                                    For large Ne : MC := |E (f ) − QNe (f )| ≈ σNe    N (0, 1), where
Resolution for a
general SPDE                        N (0, 1) is the standard Gaussian RV and where σ is the std of f .
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
                                    MC methods may be speed up by various techniques for variance
Method (SCM)
                                    reduction (Antithetic Variables; Control Variates; Matching Moment
Multivariate
                                    Methods; Stratification; Importance Sampling).
quadratures
Full                                MC simulations require reliable pseudo-random number generators :
Sparse                                  1. Inadequate random number generators produce biased results (e.g. due
                                           to artificial correlations between the generated numbers).
Intrude or not
intrude ?                               2. A random number generator must produce independent tuples and have
                                           a large cycle length.
Open issues


                         Introduction to Stochastic Spectral Methods                          25 Novembre 2008   95 / 133
                                   Quasi Monte-Carlo method
Introduction
RVs and RPs

Several UQ methods          The Quasi Monte-Carlo (QMC) method evaluates the integrand at
                            correlated quadrature points (not randomly chosen) that are generated from
Spectral
representation              “low discrepancy series” DNe .
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial              A sequence {Z 1 , . . . , Z Ne } is called quasi random if its discrepancy
Chaos

Post-processing
                                    obeys:
                                                                                       −1
                                                                 DNe ≤ c(log Ne )n Ne ,
Resolution for a
general SPDE                        where c, n are constants.
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)
                                    Typical QCM error is of the order of:
                                                                                  −1
Multivariate                                                           QMC   ≈ O(Ne · (log Ne )N ).
quadratures
Full

Sparse
                                    The O(log Ne )N dominates for large number of dimensions N but
                                                          −1
Intrude or not                      otherwise we get a O(Ne ) convergence rate.
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                     25 Novembre 2008        96 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        97 / 133
                                   Standard Gauss quadrature (1)
Introduction
RVs and RPs                         The integral of a function φj : Ωj → R can be approximated by:
Several UQ methods
                                                                                               Nq
                                                                               (j )
                                                                                               X       (j )          (j )
Spectral                                                                  QNq [φ] =                   wi      φ(Zi ),
representation                                                                                i=1
           e
Karhunen-Lo`ve

Polynomial Chaos
                                    w (j ) are the weights and the Z                       (j )
                                                                                                 are the Nq quadrature points.
generalized Polynomial
Chaos                               Multi-dimensional quadrature formula for f : ΩN → R is constructed as
Post-processing
                                    tensor product of one-dimensional quadrature formulas.
                                                                           N                   (1)                   (N )
Resolution for a
general SPDE
                                                                          QNq := QNq ⊗ . . . ⊗ QNq .
Stochastic Galerkin
Method (SGM)
                                    The grid points are constructed based on tensor-products of the
Stochastic Collocation              one-dimensional grid points of quadrature formulas (Gaussian,
Method (SCM)
                                    Clenshaw-Curtis, or Fejer quadrature formulas)
Multivariate
quadratures                                               Z N := Z (1) ⊗ . . . ⊗ Z (N ) .
Full                                The multi-dimensional integral becomes:
Sparse
                                                                       Nq             Nq
                                                    N
                                                                       X              X         (1)           (N )          (1)   (N )
Intrude or not                                     QNq [f ] :=                 ...            wi1 . . . wiN f (Zi1 , . . . , ZiN ).
intrude ?
                                                                       i1 =1          iN =1
Open issues                         Exact for N -multivariate polynomial of order at most P .
                         Introduction to Stochastic Spectral Methods                                       25 Novembre 2008              98 / 133
                                   Standard Gauss quadrature (2)
Introduction
RVs and RPs

Several UQ methods

                                    When a 1D Gauss-type quadrature rule with Nq points is used as the
Spectral
representation                      foundation of the multi-dimensional grid, the integration is exact for
Karhunen-Lo`ve
           e                        multivariate polynomials of order at most P ≤ 2Nq − c.
Polynomial Chaos

generalized Polynomial
Chaos

Post-processing
                                    c is determined by the type of Gauss-quadrature used which can be
                                    either classical Gauss (c = 1), Gauss-Radau (c = 2) or Gauss-Lobatto
Resolution for a
general SPDE
                                    (c = 3).
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)
                                    Wonder how many points are needed to reach a polynomial accuracy of
                                    at most P . The integral requires a total number of evaluations of the
                                                  N
Multivariate
quadratures
                                    integrand of Nq , where
Full                                                                  ‰        ı
                                                                        3P + c
Sparse
                                                               Nq =              .
                                                                          2
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods        25 Novembre 2008             99 / 133
                                   Standard Gauss quadrature : curse of dimensionality
Introduction
RVs and RPs                 Minimum number of Gauss quadrature points required to integrate exactly a
Several UQ methods
                            N -dimensional polynomial fonction of degree p ≤ P .
Spectral
representation
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial                             10
Chaos

Post-processing
                                                    8

Resolution for a
general SPDE                                        6
                                            Nq




Stochastic Galerkin
                                              10




Method (SGM)
                                            log




                                                    4
Stochastic Collocation
Method (SCM)
                                                    2
Multivariate
quadratures
                                                   0
Full                                               8
                                                        7
Sparse                                                                                                                10
                                                             6
                                                                                                                  8
                                                                       5
Intrude or not                                                             4                            6
                                                                 P
intrude ?                                                                      3           4
                                                                                                            N
                                                                                   2   2
Open issues


                         Introduction to Stochastic Spectral Methods                           25 Novembre 2008            100 / 133
                                   Standard Gauss quadrature - Importance of grid
Introduction
                                   nestedness
RVs and RPs

Several UQ methods
                                    Some multi-dimensional grids inherit the poor properties from their
Spectral
representation
                                    one-dimensional grid counterparts.
           e
Karhunen-Lo`ve

Polynomial Chaos                    In the case of Gauss-type quadrature, the high accuracy of the method
generalized Polynomial
Chaos
                                    is balanced by the fact that the successive grids are not embedded.
Post-processing

                                    Next figures show two-dimensional Gauss-Legendre collocation grids for
Resolution for a
general SPDE                        three different levels of refinement.
Stochastic Galerkin
Method (SGM)

Stochastic Collocation              The quadrature points are not redundant.
Method (SCM)



Multivariate                        Refinement the approximation by switching from a coarse to a more
quadratures
Full
                                    refined grid means evaluating the functional on the new grid without
Sparse                              re-using any of the previous evaluations.
Intrude or not
intrude ?                           The method is therefore computationally costly (particularly when the
                                    multi-dimensional grids are based on non-nested one-dimensional grids).
Open issues


                         Introduction to Stochastic Spectral Methods        25 Novembre 2008              101 / 133
                                   Gauss quadrature rule are not nested (1)
Introduction
RVs and RPs                 2D Legendre collocation grids for different levels of resolution.
Several UQ methods
                                                1                                                                        1


                                               0.8                                                                      0.8
Spectral
                                               0.6                                                                      0.6
representation
                                               0.4                                                                      0.4
           e
Karhunen-Lo`ve
                                               0.2                                                                      0.2
Polynomial Chaos
                                                0                                                                        0
generalized Polynomial
                                              !0.2                                                                     !0.2
Chaos
                                              !0.4                                                                     !0.4
Post-processing
                                              !0.6                                                                     !0.6


                                              !0.8                                                                     !0.8
Resolution for a
general SPDE                                   !1
                                                !1   !0.8   !0.6   !0.4   !0.2   0   0.2   0.4   0.6   0.8   1
                                                                                                                        !1
                                                                                                                         !1   !0.8   !0.6   !0.4   !0.2   0   0.2   0.4   0.6   0.8   1


Stochastic Galerkin                                           52 grid points                                                           62 grid points
Method (SGM)
                                                1                                                                        1

Stochastic Collocation
                                               0.8                                                                      0.8
Method (SCM)
                                               0.6                                                                      0.6


                                               0.4                                                                      0.4
Multivariate
quadratures                                    0.2                                                                      0.2


                                                0                                                                        0
Full
                                              !0.2                                                                     !0.2
Sparse
                                              !0.4                                                                     !0.4


                                              !0.6                                                                     !0.6
Intrude or not
intrude ?                                     !0.8                                                                     !0.8


                                               !1                                                                       !1
                                                !1   !0.8   !0.6   !0.4   !0.2   0   0.2   0.4   0.6   0.8   1           !1   !0.8   !0.6   !0.4   !0.2   0   0.2   0.4   0.6   0.8   1

                                                                    2
Open issues                                                 10 grid points                                                                     All grids

                         Introduction to Stochastic Spectral Methods                                             25 Novembre 2008                                                         102 / 133
                                   Gauss quadrature rule are not nested (2)
Introduction
RVs and RPs                 2D Hermite collocation grids for different levels of resolution
Several UQ methods
                                                                     L=5                                            L=6
                                                      5                                                5

Spectral
representation
                                                     2.5                                              2.5
           e
Karhunen-Lo`ve

Polynomial Chaos
                                               x2




                                                                                                x2
                                                      0                                                0
generalized Polynomial
Chaos

Post-processing                                     !2.5                                             !2.5



Resolution for a                                     !5                                               !5
                                                      !5      !2.5    0     2.5       5                !5    !2.5   0     2.5   5
general SPDE                                                          x1                                            x1

                                                             2
Stochastic Galerkin
Method (SGM)
                                                            5 grid points                                   62 grid points
                                                                     L=10                              5
Stochastic Collocation                                 5
Method (SCM)

                                                                                                      2.5
                                                      2.5
Multivariate
quadratures




                                                                                                x2
                                                                                                       0
                                                x2




Full                                                   0


Sparse
                                                     !2.5                                            !2.5


Intrude or not
intrude ?                                             !5                                              !5
                                                       !5     !2.5    0     2.5   5                    !5    !2.5   0     2.5   5
                                                                      x1                                            x1

                                                              2
Open issues                                                 10 grid points                                    All grids

                         Introduction to Stochastic Spectral Methods                      25 Novembre 2008                          103 / 133
                                   Gauss quadrature rule are not nested (3)
Introduction
RVs and RPs                 3D Hermite collocation grids for different levels of resolutio
Several UQ methods
                                                                                         L=5                                                                             L=6



Spectral
                                                      5                                                                                    5
representation
           e
Karhunen-Lo`ve                                       2.5                                                                                  2.5

Polynomial Chaos                                      0                                                                                    0
                                               x3




                                                                                                                                    x3
generalized Polynomial
                                                    !2.5                                                                                 !2.5
Chaos

Post-processing                                      !5                                                                                   !5
                                                      5                                                                                    5
                                                               2.5                                                        5                     2.5                                                 5
                                                                           0                                    2.5                                        0                                  2.5
Resolution for a                                                                                      0                                                                               0
                                                                               !2.5            !2.5                                                            !2.5            !2.5
general SPDE                                                          x2              !5 !5           x                                               x2              !5 !5           x
                                                                                                           1                                                                          1

Stochastic Galerkin
Method (SGM)
                                                                     53 grid points                                                                   63 grid points
                                                                                        L=10
Stochastic Collocation
Method (SCM)
                                                                                                                                           5
                                                           5

Multivariate                                                                                                                              2.5
                                                      2.5
quadratures
                                                                                                                                           0




                                                                                                                                    x3
                                                           0
                                                x3




Full
                                                                                                                                         !2.5
Sparse                                               !2.5

                                                                                                                                          !5
                                                       !5                                                                                  5
                                                        5
                                                                                                                                                2.5                                                 5
Intrude or not                                                  2.5                                                   5
                                                                                                                                                                                              2.5
                                                                           0                                   2.5                                         0
intrude ?                                                                                             0                                                                               0
                                                                               !2.5                                                                            !2.5            !2.5
                                                                                               !2.5
                                                                      x               !5 !5           x                                               x2              !5 !5           x
                                                                       2                               1                                                                                  1


Open issues                                                     103 grid points                                                                                All grids

                         Introduction to Stochastic Spectral Methods                                                          25 Novembre 2008                                                          104 / 133
                                   Growth of one-dimensional Clenshaw-Curtis (Chebyshev
Introduction
                                   extrema) grid
RVs and RPs

Several UQ methods

                                                                  7
Spectral
representation
           e
Karhunen-Lo`ve

Polynomial Chaos                                                  6

generalized Polynomial
Chaos

Post-processing
                                                                  5
                                               Quadrature level




Resolution for a
general SPDE
                                                                  4
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)
                                                                  3

Multivariate
quadratures
Full                                                              2
Sparse



Intrude or not                                                    1
intrude ?                                                         −1   −0.8   −0.6   −0.4   −0.2       0         0.2   0.4     0.6   0.8   1
                                                                                             Quadrature values in 1D

Open issues


                         Introduction to Stochastic Spectral Methods                                              25 Novembre 2008             105 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        106 / 133
                                   Sparse quadrature - Smolyak algorithm (1)
Introduction
RVs and RPs
                            If the tensor product of quadrature formulas combine high-order formulas in
Several UQ methods          only a few random dimensions with low-order formulas in the other
                            dimensions −→ the resulting quadrature may be feasible in high dimensions.
Spectral
representation
                                    Introduction of different quadrature level l in different dimensions.
           e
Karhunen-Lo`ve

Polynomial Chaos                    We assume that the quadrature Qlj (in dimension Ωj ) integrates exactly
generalized Polynomial
Chaos
                                    polynomials of degree not exceeding Pl .
                                                                                                       (1)                      (N )
Post-processing
                                    For a l = (l , . . . , lN ) ∈ NN :QlN := Ql1 ⊗ . . . ⊗ QlN .
Resolution for a
general SPDE
                                    Let us assume also that each Qlj has the same number of nodes Nqlj
Stochastic Galerkin                 and that the lowest level has a single node (Nq1j = 1, ∀j = 1 . . . N ).
Method (SGM)
                                                               (j )            (j )            (j )
Stochastic Collocation
Method (SCM)                        Let the nodes Γl                   = {Zl,1 , . . . , Zl,Nq } and the weights
                                                                                                      lj
                                        (j )            (j )
Multivariate                        wl,1 , . . . , wl,Nq
quadratures                                              lj
Full
                                    used by Qlj .
Sparse
                                    Apply the quadrature to a function φ : Ω(N ) → R :
                                                                 Nql           Nql
Intrude or not                                                   X1            XN
                                                                                         (1)               (N )          (1)           (N )
                                                QlN [φ] :=
intrude ?
                                                                         ...           wl1 ,i1 . . . wlN ,iN φ(Zl1 ,i1 , . . . , ZlN ,iN ).
Open issues                                                      i1 =1         iN =1
                                    Goal : build a quadrature for which only a few li are large.
                         Introduction to Stochastic Spectral Methods                                         25 Novembre 2008                 107 / 133
                                   Sparse quadrature - Smolyak algorithm (2)
Introduction
RVs and RPs

Several UQ methods

                                    Combine such quadratures by introducing quadrature differences :
Spectral
representation                                                          (j )        (j )
           e
Karhunen-Lo`ve
                                                              ∆j := Ql
                                                               l               − Ql−1 , l ∈ N, j = 1, . . . , N ,
Polynomial Chaos
                                               (j )
generalized Polynomial
Chaos
                                    and Q0            := 0, ∀j = 1 . . . N .
Post-processing



Resolution for a
general SPDE
                                    Construct linear combinations of such quantities only for indices that
Stochastic Galerkin                 fall within the unit simplex.
Method (SGM)

Stochastic Collocation
Method (SCM)                        The level l Smolyak quadrature formula in N dimensions is :
                                                               X          (1)         (N )
Multivariate
quadratures
                                                   QN =
                                                     l                 ∆l1 ⊗ . . . ⊗ ∆lN ,
Full                                                                   l ∈NN ,|l |≤N +l−1
Sparse

                                    where |l | = l1 + . . . + lN .
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                          25 Novembre 2008      108 / 133
                                   Sparse quadrature - Smolyak grid (1)
Introduction
RVs and RPs
                                    Every QlN quadrature evaluates the integrand on the grid:
Several UQ methods
                                                                        (N )       (1)            (N )
Spectral
                                                                       Γl      = Γl1 × . . . × ΓlN .
representation
Karhunen-Lo`ve
           e                        QN evaluates the integrand on the union of these grids:
                                     l
Polynomial Chaos                                       [                [
                                                              (N )              (1)          (N )
generalized Polynomial
Chaos
                                            ΓN =
                                              l              Γl =             Γl1 × . . . × ΓlN .
Post-processing                                                 |l|≤N +l−1           |l|≤N +l−1

Resolution for a
                                    As the sparse quadrature is enriched by increasing the quadrature level
general SPDE                        from l to (l + 1), the grid points used in the earlier levels are retained.
Stochastic Galerkin
Method (SGM)
                                    The sparse quadrature is always embedded regardless of the nature of
Stochastic Collocation
Method (SCM)                        the 1D basis quadrature rule used.
                                                                                                              (j )   (j )
Multivariate                        If the one-dimensional quadrature rule are nested (i.e. Γl+1 ⊆ Γl
quadratures
                                              (N )   (N )
Full                                then Γl ⊂ Γl            when lj ≤ lj , j = 1, . . . , N ), this results in a
Sparse
                                    much smaller number of collocation points compared to the non-nested
Intrude or not
                                    formulas.
intrude ?
                                    The Smolyak formula is actually interpolatory whenever nested points
Open issues                         are used.
                         Introduction to Stochastic Spectral Methods                       25 Novembre 2008                 109 / 133
                                   Sparse quadrature - Smolyak grid (2)
Introduction
RVs and RPs

Several UQ methods

                                    It is important to notice that the Smolyak algorithm will perform his
Spectral
representation                      task of building a 1D grid for any family of one-dimensional quadrature
Karhunen-Lo`ve
           e                        rules.
Polynomial Chaos

generalized Polynomial
Chaos
                                    The latter will affect the Nc as well as the overall accuracy of the
Post-processing                     integration.
Resolution for a
general SPDE
                                    At high dimensions, the sparse grid allows for a significant reduction in
Stochastic Galerkin
Method (SGM)                        the numerical cost of the quadrature.
Stochastic Collocation
Method (SCM)



Multivariate
                                               observed that the sparse grid is optimal only when the
                                    [Nobile 2008]
quadratures                         integrand depends roughly uniformly on all dimensions.
Full

Sparse
                                    If the integrand is highly anisotropic, the convergence of the sparse grid
Intrude or not                      integral will deteriorate.
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods         25 Novembre 2008             110 / 133
                              Sparse quadrature - Smolyak grid (3)
Introduction                                                   Higher Order Quadrature on Sparse Grids                395
RVs and RPs
                          Visual representation of a sparse grid based on the midpoint rule
Several UQ methods



Spectral                                       W 11         W 21                                   l1
representation
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial
Chaos

Post-processing                                W 12


Resolution for a
general SPDE
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
Method (SCM)



Multivariate
quadratures
Full

Sparse



Intrude or not
intrude ?

                                               l2
Open issues


                       Introduction to Stochastic subspaces                    represents
                      Fig. 1. Scheme ofSpectral Methods for d = 2: Each square25 Novembre 2008one hierarchical subspace
                                                                                                                 111 / 133
                                   Sparse quadrature - Smolyak grid (4)
Introduction
RVs and RPs                 Three-dimensional Gauss-Legendre Sparse Quadrature with L = 6.
Several UQ methods



Spectral                                            1                                                                    1
representation
                                                   0.5                                                                  0.5
           e
Karhunen-Lo`ve
                                                    0                                                                    0
                                             ξ3




                                                                                                                  ξ3
Polynomial Chaos
                                                  −0.5                                                                 −0.5
generalized Polynomial
Chaos
                                                   −1                                                                   −1
                                                    1                                                                    1
Post-processing
                                                         0.5                                             1                    0.5                                             1
                                                                    0                              0.5                                   0                              0.5
                                                                                              0                                                                    0
                                                                        −0.5           −0.5                                                  −0.5           −0.5
Resolution for a                                               ξ2              −1 −1          ξ1                                    ξ2              −1 −1          ξ1
general SPDE
Stochastic Galerkin                          Reduced hypercube in [0, 1]N                                                     After one reflection
Method (SGM)

Stochastic Collocation
Method (SCM)
                                                    1                                                                    1

                                                   0.5                                                                  0.5
Multivariate
quadratures                                         0                                                                    0
                                             ξ3




                                                                                                                  ξ3
Full
                                                  −0.5                                                                 −0.5
Sparse
                                                   −1                                                                   −1
                                                    1                                                                    1
                                                         0.5                                             1                    0.5                                             1
Intrude or not                                                      0                              0.5                                   0                              0.5
                                                                                              0                                                                    0
                                                                        −0.5                                                                 −0.5
intrude ?                                                                              −0.5                                                                 −0.5
                                                               ξ2              −1 −1          ξ1                                    ξ2              −1 −1          ξ1


Open issues                                          After two reflections                                      Complete set of quadrature points

                         Introduction to Stochastic Spectral Methods                                         25 Novembre 2008                                                     112 / 133
                                   Bounded support
Introduction
RVs and RPs
                            The Smolyak sparse grid can be generated using many different bounded 1D
Several UQ methods          quadrature rules.
Spectral                    Most used : Fej´r and Clenshaw-Curtis (nested).
                                           e
representation
Karhunen-Lo`ve
           e
                            Gauss-Legendre (not nested) can also be used as the basis of the tensor
Polynomial Chaos            product.
generalized Polynomial                                     
Chaos                                                         1             if l = 1
                                Clenshaw-Curtis 1D : ml =                            .
Post-processing
                                                              2l−1 + 1      if l > 1
Resolution for a
general SPDE
                                    Fej´r 1D : ml = 2l − 1
                                       e
Stochastic Galerkin
Method (SGM)
                                                            e
                                    Although nested, the Fej´r 1D quadrature rule grows faster than the
Stochastic Collocation              Clenshaw-Curtis 1D rule due to the exclusion of the end points on the
Method (SCM)
                                    support [−1, 1].
Multivariate
quadratures                         Gauss-type 1D : we have the choice on how 1D grid grows for         levels
Full                                l.
Sparse
                                    Usually, an exponential growth for a grid with odd number of
Intrude or not                      collocation points (starting with one point at the center at the lowest
intrude ?
                                    level) is the most common. But one can also try different growths such
Open issues                         as: linear, linear odd, linear even, and exponential even.
                         Introduction to Stochastic Spectral Methods         25 Novembre 2008              113 / 133
                                   Summary of the 1D quadrature rules for different weights
Introduction
RVs and RPs                                               ρ(x )           Ω         P            Nq          Analytic Solutions    Nestedness
Several UQ methods            Gauss-Legendre                   1         (-1,1)   2Nq − 1         L                 No                 No

                               Gauss-Jacobi       (1 − x )a (1 + x )b    (-1,1)   2Nq − 1         L                 No                 No
Spectral
representation                                        a, b > −1
                                                                                                                      π(2n−1)
                                                        p
           e
Karhunen-Lo`ve
                             Gauss-Chebyshev          1/ 1 − x 2         (-1,1)   2Nq − 1         L         zn = cos( 2L
                                                                                                                                  ),   No
Polynomial Chaos                 (Type I)                                                                        wn = π
                                                                                                                      L
generalized Polynomial                                 p
Chaos                        Gauss-Chebyshev            1 − x2           (-1,1)   2Nq − 1         L                     πn
                                                                                                             zn = cos( L+1 ),          No
Post-processing                   (Type II)                                                              wn = L+1 sin2 ( L+1 )
                                                                                                               π          πn
                                                        p
Resolution for a              Clenshaw-Curtis         1/ 1 − x 2         [-1,1]   Nq − 1     2L−1 + 1             zn only              Yes
general SPDE                                            p
                                 e
                              Fej´r (Type II)         1/ 1 − x 2         (-1,1)   Nq − 1       2L − 1             zn only              Yes
Stochastic Galerkin
Method (SGM)
                              Gauss-Laguerre               e −x         [0, ∞)     2Nq − 1        L                 No                 No
Stochastic Collocation
Method (SCM)
                                                              −x 2
                              Gauss-Hermite               e             (−∞,∞)    2Nq − 1         L                 No                 No
                                                             2
Multivariate                  Gauss-Hermite           √1 e −x /2        (−∞,∞) 2Nq − 1            L                 No                 No
quadratures                                            2π
Full                             (Normal)
Sparse                                    e
                               Mapped Fej´r                             (−∞,∞)                 2L − 1             zn only              Yes
                                  (Type II)
Intrude or not                                               2
                                  Hermite             √1 e −x /2        (−∞,∞) 2m + n − 1     1-3-9-19-35           No                 Yes
intrude ?                                              2π
                             Kronrod-Patterson                                                or 1-4-18-30
Open issues


                         Introduction to Stochastic Spectral Methods                         25 Novembre 2008                           114 / 133
                                        Examples for bounded support (1)
Introduction
RVs and RPs

Several UQ methods



Spectral
representation
Karhunen-Lo`ve
           e
                            2D Legendre collocation grids for different levels of resolution
Polynomial Chaos

generalized Polynomial
                                                L=4                                         L=5                                          L=6
Chaos                              1                                          1                                            1

Post-processing

                                  0.5                                        0.5                                          0.5
Resolution for a
general SPDE




                                                                                                                    x2
                            x2




                                                                       x2
                                   0                                          0                                            0
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
                                 !0.5                                       !0.5                                         !0.5
Method (SCM)



Multivariate                      !1
                                   !1    !0.5   0        0.5    1
                                                                             !1
                                                                              !1     !0.5   0     0.5       1
                                                                                                                          !1
                                                                                                                           !1     !0.5   0     0.5      1
quadratures                                     x1                                          x1                                           x1


Full                                    49 grid points                             129 grid points                              321 grid points
Sparse



Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                                    25 Novembre 2008                             115 / 133
                                        Examples for bounded support (2)
Introduction
RVs and RPs

Several UQ methods



Spectral
representation
Karhunen-Lo`ve
           e
                            2D Chebyshev collocation grids for different levels of resolution
Polynomial Chaos

generalized Polynomial
                                                L=4                                         L=5                                          L=6
Chaos                              1                                          1                                            1

Post-processing

                                  0.5                                        0.5                                          0.5
Resolution for a
general SPDE
                            x2




                                                                       x2




                                                                                                                    x2
                                   0                                          0                                            0
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
                                 !0.5                                       !0.5                                         !0.5
Method (SCM)



Multivariate                      !1
                                   !1    !0.5   0       0.5     1
                                                                             !1
                                                                              !1     !0.5   0     0.5       1
                                                                                                                          !1
                                                                                                                           !1     !0.5   0     0.5      1
quadratures                                     x1                                          x                                            x1
                                                                                             1


Full                                    33 grid points                             169 grid points                              221 grid points
Sparse



Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                                    25 Novembre 2008                             116 / 133
                                        Examples for bounded support (3)
Introduction
RVs and RPs

Several UQ methods



Spectral
representation
Karhunen-Lo`ve
           e
                            2D Clenshaw-Curtis collocation grids for different levels of resolution
Polynomial Chaos

generalized Polynomial
                                                L=4                                        L=5                                          L=6
Chaos                              1                                          1                                           1

Post-processing

                                  0.5                                        0.5                                         0.5
Resolution for a
general SPDE
                            x2




                                                                       x2




                                                                                                                   x2
                                   0                                          0                                           0
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
                                 !0.5                                       !0.5                                        !0.5
Method (SCM)



Multivariate                      !1
                                   !1    !0.5   0       0.5     1
                                                                             !1
                                                                              !1    !0.5   0     0.5       1
                                                                                                                         !1
                                                                                                                          !1     !0.5   0     0.5      1
quadratures                                     x1                                         x1                                           x1


Full                                    29 grid points                             65 grid points                              145 grid points
Sparse



Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                                   25 Novembre 2008                             117 / 133
                                        Examples for bounded support (4)
Introduction
RVs and RPs

Several UQ methods



Spectral
representation
Karhunen-Lo`ve
           e
                            2D Fejer-II collocation grids for different levels of resolution
Polynomial Chaos

generalized Polynomial
                                                L=4                                         L=5                                          L=6
Chaos                              1                                          1                                            1

Post-processing

                                  0.5                                        0.5                                          0.5
Resolution for a
general SPDE




                                                                                                                    x2
                                                                       x2
                                                                                                                           0
                            x2




                                   0                                          0
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
                                 !0.5                                       !0.5                                         !0.5
Method (SCM)



Multivariate                      !1
                                   !1    !0.5   0        0.5    1
                                                                             !1
                                                                              !1     !0.5   0     0.5       1
                                                                                                                          !1
                                                                                                                           !1     !0.5   0     0.5      1
quadratures                                     x1                                          x1                                           x1


Full                                    49 grid points                             129 grid points                              321 grid points
Sparse



Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                                    25 Novembre 2008                             118 / 133
                                   Polynomial exactness (1)
Introduction
RVs and RPs

Several UQ methods
                            Polynomial exactness P (l , N ) of Smolyak sparse grid for different 1D
Spectral
representation
                            quadrature choices :
           e
Karhunen-Lo`ve

Polynomial Chaos
                               1. Gauss-type : P (l , N ) = 2L − 1
generalized Polynomial
Chaos

Post-processing
                               2. Clenshaw-Curtis :
Resolution for a                                    
general SPDE                                          2l − 1                                             if l < 3N + 1;
                                       P (l , N ) =
Stochastic Galerkin
Method (SGM)
                                                      2σ−1 (N + τ + 1) + 2N − 1                          otherwise
Stochastic Collocation
Method (SCM)
                                     e
                               3. Fej´r :
Multivariate                                                           
quadratures
                                                                           2l − 1                    if l < 2N + 1,
                                                   P (l , N ) =
                                                                           2σ−1 (N + τ + 1) − 1
Full

Sparse
                                                                                                     otherwise

Intrude or not                      where σ = floor((l + N − 1)/N ) and τ = (l + N − 1) mod N .
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods                       25 Novembre 2008               119 / 133
                                   Polynomial exactness (2)
Introduction
RVs and RPs                 Accuracy and computational cost comparison between full quadratures
Several UQ methods
                            (Gauss-based and Clenshaw-Curtis-based) and sparse quadratures
Spectral                    (Clenshaw-Curtis-based).
representation
                                                                                                                                                                                                                 N=2
                                                                    30                                                                                                          40
           e
Karhunen-Lo`ve                                                              Sparse CC: N = 2
                                                                                                                                                                                        Full GQ
                                                                                                                                                                                        Full CC
                                                                            Sparse CC: N = 5                                                                                    35
                                                                                                                                                                                        Sparse CC
                                                                    25
Polynomial Chaos                                                            Sparse CC: N = 10

                                                                                                                                                                                30

generalized Polynomial                                              20
                                              Polynomial order, p




                                                                                                                                                          Polynomial order, p
                                                                                                                                                                                25
Chaos
                                                                    15                                                                                                          20
Post-processing
                                                                                                                                                                                15
                                                                    10

                                                                                                                                                                                10

Resolution for a                                                     5
                                                                                                                                                                                 5
general SPDE
                                                                     0                                                                                                           0
                                                                       0     1           2              3          4         5         6        7                                  0                1                     2             3
                                                                     10    10           10             10         10        10        10   10                                    10             10                   10            10
Stochastic Galerkin                                                                             Number of function calls                                                                                Number of function calls
Method (SGM)

Stochastic Collocation                                                          N = {2, 5, 10}                                                                                                          N =2
Method (SCM)                                                                                                N=5                                                                                                  N = 10
                                                                    40                                                                                                          25
                                                                            Full GQ                                                                                                    Full GQ
                                                                            Full CC                                                                                                    Full CC
                                                                    35                                                                                                                 Sparse CC
                                                                            Sparse CC

Multivariate                                                        30
                                                                                                                                                                                20



quadratures
                                              Polynomial order, p




                                                                                                                                                          Polynomial order, p
                                                                    25
                                                                                                                                                                                15

Full
                                                                    20


Sparse                                                              15
                                                                                                                                                                                10



                                                                    10
                                                                                                                                                                                 5

Intrude or not                                                       5


intrude ?                                                            0                                                                                                           0
                                                                       0          2                 4                   6         8         10                                     0        2               4                  6    8        10
                                                                     10          10               10                   10        10        10                                    10       10              10                  10   10       10
                                                                                                Number of function calls                                                                                Number of function calls



Open issues                                                                                  N =5                                                                                                       N = 10

                         Introduction to Stochastic Spectral Methods                                                                                25 Novembre 2008                                                                              120 / 133
                                   Integration of non-periodic, non-linear, discontinuous
Introduction
                                   multi-dimensional functions : the Genz test
RVs and RPs

Several UQ methods

                               1. Oscillarory : f1 (x ) = cos 2πw1 + d ci xi .
                                                              `         P            ´
Spectral
                                                                           i=1
representation                                                                             ´−1
                               2. Product Peak : f2 (x ) = d
                                                                    ` −2
                                                                          + (xi − wi )2
                                                              Q
           e
Karhunen-Lo`ve                                                   i=1 ci                        .
Polynomial Chaos                                            `      Pd        ´−(d+1)
generalized Polynomial         3. Corner Peak : f3 (x ) = 1 + i=1 ci xi                .
Chaos

Post-processing
                                                             ` Pd        2            2
                                                                                        ´
                               4. Gaussian : f4 (x ) = exp − i=1 ci (xi − wi ) .
Resolution for a
                               5. Continuous : f5 (x ) = exp − d ci |xi − wi | .
                                                                ` P                      ´
general SPDE                                                          i=1
Stochastic Galerkin                                           
Method (SGM)
                                                                  0 `               ´ if x1 > w1 or x2 > w2
Stochastic Collocation         6. Discontinuous : f6 (x ) =             Pd
Method (SCM)                                                      exp     i=1 ci xi      otherwise
Multivariate
quadratures
                            The sparse Clenshaw-Curtis quadrature method is more efficient and more
Full
                            accurate than the full Gauss-quadrature method for the integration of
Sparse                      multi-dimensional, non-periodic smooth functions when the number of
                            dimensions N > 5.
Intrude or not
intrude ?
                            Major convergence problems remain for both methods for the integration of
Open issues                 discontinuous and/or irregular and/or anisotropic functions.

                         Introduction to Stochastic Spectral Methods         25 Novembre 2008                 121 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        122 / 133
                                   Intrude or not intrude ? (1)
Introduction
RVs and RPs

Several UQ methods          Stochastic Galerkin method (SGM) :
Spectral
representation                      One deals with the weak form of the solution.
           e
Karhunen-Lo`ve

Polynomial Chaos
                                    Existing numerical solvers need to be extensively adapted (e.g. include
generalized Polynomial
Chaos                               the trial functions).
Post-processing



Resolution for a                    Difficulty with non-linear & non-polynomial functionals representations.
general SPDE
Stochastic Galerkin
Method (SGM)                        The resulting set of deterministic equations are strongly coupled.
Stochastic Collocation
Method (SCM)                        Numerical solvers need to be very robust.

Multivariate
quadratures                         Stability issue KL/PC/gPC converge in L2 , not in L∞ (uniformly).
Full

Sparse
                                    Curse of dimensionality.
Intrude or not
intrude ?                           Better (spectral) theory background (error analysis, basis reduction, ...)
Open issues


                         Introduction to Stochastic Spectral Methods         25 Novembre 2008              123 / 133
                                   Intrude or not intrude ? (2)
Introduction
RVs and RPs

Several UQ methods
                            Stochastic collocation method (SCM) :
Spectral
representation
                                    One deals with the strong form of the solution.
           e
Karhunen-Lo`ve

Polynomial Chaos

generalized Polynomial              No modification of the numerical solver is needed (black box).
Chaos

Post-processing

                                    Curse of dimensionality.
Resolution for a
general SPDE
Stochastic Galerkin                 The solutions are influenced by the deterministic black box solver error,
Method (SGM)

Stochastic Collocation
                                    the quadrature approximation error, in addition to the gPC truncation
Method (SCM)
                                    error of inputs & solution.
Multivariate
quadratures
                                    Overcomes the problem of nonlinearity.
Full

Sparse
                                    For a specific realization, the problem may not be well-posed or the
Intrude or not
intrude ?
                                    solver may not converge.

Open issues


                         Introduction to Stochastic Spectral Methods         25 Novembre 2008             124 / 133
                                   Intrude or not intrude ? (3)
Introduction
RVs and RPs

Several UQ methods          Let’s play with some very crude assumptions here. Let’s assume that we use
                            direct solvers (not adaptive). We assume moreover:
Spectral
representation                      We own a deterministic PDE solver and that it takes α time units to
Karhunen-Lo`ve
           e
                                    run a single simulation for a single set of input parameters.
Polynomial Chaos

generalized Polynomial              We own a SGM solver that is not affected by the additional cost coming
Chaos

Post-processing
                                    from the coupling of the gPC modes in the system of equations.
                                                                                                      N
                                    Our stochastic solution is well represented by a polynomial form PP of
Resolution for a
general SPDE                        highest degree P in N iid RVs with known distributions.
Stochastic Galerkin
Method (SGM)                                =⇒ Need M ≡ M (N , P ) gPC modes to represent the solution.
Stochastic Collocation
Method (SCM)
                            Then:
Multivariate                        For full-tensor or sparse-based SCM, the total computational effort is
quadratures
Full
                                    α × Nc (N , P ).
Sparse                              For SGM, the total computational effort is α × M (N , P ).
Intrude or not                      Next slide compares those estimates for full Gauss quadrature and
intrude ?
                                    Smolyak Clenshaw-Curtis quadrature and for different N dimensions.
Open issues


                         Introduction to Stochastic Spectral Methods         25 Novembre 2008             125 / 133
                                   Intrude or not intrude ? (4)
Introduction
RVs and RPs                 Comparison of computational effort for the SGM and the SCM with
Several UQ methods
                            increasing degree of the gPC expansion, P , at N = 3 to N = 20.
Spectral                                                                    6
                                                                                                         N=3
                                                                                                                                                                      8
                                                                                                                                                                                                    N=5
                                                                       10                                                                                        10
representation                                                                      Galerkin Method                                                                           Galerkin Method
                                                                                    Collation with Full Quadrature                                                            Collation with Full Quadrature
           e
Karhunen-Lo`ve                                                         10
                                                                            5
                                                                                    Collation with Sparse Quadrature                                                          Collation with Sparse Quadrature
                                       Normalized Degrees of Freedom




                                                                                                                                 Normalized Degrees of Freedom
                                                                                                                                                                      6
Polynomial Chaos                                                                                                                                                 10
                                                                            4
                                                                       10
generalized Polynomial
Chaos                                                                       3                                                                                         4
                                                                       10                                                                                        10
Post-processing
                                                                            2
                                                                       10
                                                                                                                                                                      2
                                                                                                                                                                 10
Resolution for a                                                            1
                                                                       10
general SPDE
Stochastic Galerkin                                                    10 0
                                                                            0
                                                                                                                                                                 10 0
                                                                                                                                                                      0
                                                                                                                        1                                                                                             1
Method (SGM)                                                             10                                            10                                          10                                                10
                                                                                                          P                                                                                          P
Stochastic Collocation                                                                                  N=10                                                                                        N=20
                                                                        14                                                                                        25
Method (SCM)                                                           10                                                                                        10
                                                                                    Galerkin Method                                                                           Galerkin Method
                                                                        12
                                                                       10           Collation with Full Quadrature                                                            Collation with Full Quadrature
                                                                                    Collation with Sparse Quadrature                                              20          Collation with Sparse Quadrature
                                       Normalized Degrees of Freedom




                                                                                                                                 Normalized Degrees of Freedom
                                                                                                                                                                 10
Multivariate                                                            10
                                                                       10
quadratures
                                                                                                                                                                  15
                                                                        8
                                                                       10                                                                                        10
Full

Sparse                                                                  6
                                                                       10                                                                                         10
                                                                                                                                                                 10
                                                                        4
                                                                       10
Intrude or not                                                                                                                                                    5
                                                                                                                                                                 10
                                                                        2
intrude ?                                                              10

                                                                        0                                                                                         0
                                                                       10       0                                            1                                   10
                                                                            10                                              10                                            1                     2                3    4   5
Open issues                                                                                               P                                                                                          P




                         Introduction to Stochastic Spectral Methods                                                                                             25 Novembre 2008                                             126 / 133
Introduction
                                1   Introduction
RVs and RPs                            RVs and RPs
Several UQ methods
                                       Several UQ methods
Spectral
                                2   Discretization and spectral representation of random fields
representation
Karhunen-Lo`ve
           e
                                                    e
                                       Karhunen-Lo`ve representation
Polynomial Chaos                       Homogeneous Chaos / Polynomial Chaos representation
generalized Polynomial
Chaos
                                       generalized Polynomial Chaos representation
Post-processing                        Post-processing
Resolution for a
                                3   Resolution for a general SPDE
general SPDE
                                       Stochastic Galerkin Method (SGM)
Stochastic Galerkin
Method (SGM)                           Stochastic Collocation Method (SCM)
Stochastic Collocation
Method (SCM)                    4   Multivariate quadratures
Multivariate
                                       Full tensor-based quadrature
quadratures                            Sparse Smolyak-based quadrature
Full

Sparse
                                5   Intrude or not intrude ?
                                6   Open issues
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods     25 Novembre 2008        127 / 133
                                   Open issues
Introduction
RVs and RPs
                            Main area of current research :
Several UQ methods
                                    Well-posedness             Gaussian measure; sign properties for BVP; error of truncation;
Spectral                            importance of the tails; truncated Jacobi-expansion.
representation
           e
Karhunen-Lo`ve

Polynomial Chaos
                                    “Curse of dimensionality”.
generalized Polynomial
Chaos                               Low stochastic regularity.
Post-processing

                                    Long-term integration :              problem of random frequency; oscillators; gPC loses
Resolution for a
general SPDE                        p-convergence after a finite time; polynomial order must              with time.
Stochastic Galerkin
Method (SGM)
                                    Space/Basis optimality :              arbitrary measures, reduced basis, low-dimensional
Stochastic Collocation
Method (SCM)                        modeling, connection to POD, dependent RVs.

Multivariate
quadratures
                                    Computational complexity :                relates to dimensionality; development of framework
Full                                for stochastic coupling and parallelization; pre-conditioning; multi-grid approach;
Sparse                              hierarchical parallelization.

Intrude or not
intrude ?
                                    Optimization/Design under uncertainty :                    robust design; surrogate-based
                                    optimization; orthogonal polynomials numerically generated; PCM & SC-gPC; anisotropic
Open issues
                                    cubature.

                         Introduction to Stochastic Spectral Methods                         25 Novembre 2008                    128 / 133
                                  “Curse of dimensionality”
Introduction
RVs and RPs

Several UQ methods



Spectral
representation
Karhunen-Lo`ve
           e
                                    The number N required for accurate representation of input processes
Polynomial Chaos                    with low Cl can be extremely large.
generalized Polynomial
Chaos

Post-processing
                                                e
                                    Karhunen-Lo`ve inputs with moderately high dimensionality (N > 10)
Resolution for a                    pose major computational challenges for stochastic spectral methods.
general SPDE
Stochastic Galerkin
Method (SGM)

Stochastic Collocation
                                    Problem even worse in case of low stochastic regularity.
Method (SCM)



Multivariate                        Complexity of sparse grids still depends heavily on N and on the
quadratures
Full
                                    regularity of the integrand.
Sparse



Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods        25 Novembre 2008           129 / 133
                                  “Curse of dimensionality” - Answers (among others)
Introduction
RVs and RPs

Several UQ methods
                                    “coarsening” or “sparsification” of the tensor product gPC subspaces
Spectral
representation
                                    [Schwab]; Reduced stochastic basis [Nair 2002, Nouy 2007]; variable
Karhunen-Lo`ve
           e                        stochastic polynomial degrees. Sparse Finite Elements for stochastic
Polynomial Chaos                    problems [Schwab 2003]
generalized Polynomial
Chaos

Post-processing
                                    Dimension-adaptative tensor-product sparse quadrature. Adaptation of
Resolution for a                    the collocation grid (iteratively) to the anisotropy of the solution;
general SPDE
Stochastic Galerkin
                                    development of error analysis to select the best criteria [Nobile 2008,
Method (SGM)                        Crestaux 2008].
Stochastic Collocation
Method (SCM)



Multivariate                        ANOVA-type decomposition coupled with ME-PCM: MEPCM-A [Foo
quadratures
                                    2008]. “Analysis-of-Variance” involves splitting a multidimensional
Full

Sparse
                                    function into its contributions from different groups of subdimensions.
                                    Promising results on discontinuous functions from the Genz test.
Intrude or not
intrude ?
                                    MEPCM-A is favorable to MC up to 600 dimensions.

Open issues


                         Introduction to Stochastic Spectral Methods        25 Novembre 2008               130 / 133
                                   Low stochastic regularity - PC/gPC failures
Introduction
RVs and RPs

Several UQ methods
                                    Non-linearities in the random space (especially for the stochastic
Spectral
representation
                                    Galerkin approach) can cause problems as they require high polynomials
Karhunen-Lo`ve
           e
                                    order.
Polynomial Chaos

generalized Polynomial
                                    This problem is amplified by the way the standard PC/gPC is
Chaos
                                    constructed as a global (p-type) approximation over the uncertainty
Post-processing
                                    range.
Resolution for a
general SPDE
                                    Discontinuous solution, e.g. hyperbolic problems, or non-smooth
Stochastic Galerkin
Method (SGM)                        stochastic solutions - also called parametric or stochastic bifurcation -
Stochastic Collocation
Method (SCM)                        wrt to the uncertain parameters are not well represented by smooth
                                    polynomials.
Multivariate
quadratures
                                    → Gibbs phenomenon → strong oscillations (can induce loss of
Full

Sparse
                                    well-posedness, e.g. negative value of a positive physical quantity!).

Intrude or not
intrude ?
                                    This problem affects both SGM and SCM/PCM approaches.

Open issues


                         Introduction to Stochastic Spectral Methods          25 Novembre 2008               131 / 133
                                   Low stochastic regularity - Answers among others (1)
Introduction
RVs and RPs

Several UQ methods

                                    Use different types of chaos basis or change the nature of the expansion
Spectral
representation                      to handle the steep dependency or oscillating character of the solution :
           e
Karhunen-Lo`ve

Polynomial Chaos                                                  ıtre
                                             Haar-wavelets [Le Maˆ 2004], Pade-Legendre, piecewise polynomials
generalized Polynomial                       (B-Splines [Millman 2004]).
Chaos

Post-processing



Resolution for a
                                    Make the spectral decomposition more local by decomposing the
general SPDE                        random space (the probability measure) into different regions or
Stochastic Galerkin                 different scales :
Method (SGM)

Stochastic Collocation
Method (SCM)                                 Piecewise polynomials (h/k-type), stochastic finite elements [Deb 2001,
                                             Babuska 2002].
Multivariate
quadratures                                  Multi-elements (h/p-type) gPC (ME-GPC) [Wan 2005], Multi-elements
Full                                         PCM (ME-PCM) [Foo 2008]
Sparse

                                                                                   ıtre
                                             Multi-wavelets/Multi-resolution [Le Maˆ 2004].
Intrude or not
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods              25 Novembre 2008              132 / 133
                                   Low stochastic regularity - Answers among others (2)
Introduction
RVs and RPs

Several UQ methods
                                    Constrain the system by using the gPC representation for the entropic
Spectral                            variable :
representation
           e
Karhunen-Lo`ve
                                             bound the oscillations of the solution in the random space by working out
Polynomial Chaos
                                             the SGM onto the entropic variable (linked to the main variable through
generalized Polynomial
Chaos                                        the entropy of the system) instead, e.g. systems of conservation laws
Post-processing                                 e
                                             [Po¨tte 2008].

Resolution for a
general SPDE
                                    Adapt the quadrature grid to the strong solution gradients :
Stochastic Galerkin
Method (SGM)

Stochastic Collocation                       Reduce the aliasing error of the SCM methods by sampling more where
Method (SCM)
                                             necessary. Adapt the quadrature method to the strong solution
Multivariate
                                             gradients.
quadratures
Full
                                             Dimension-adaptative tensor-product sparse quadrature. Adaptation of
Sparse                                       the collocation grid (iteratively) to the anisotropy of the solution;
                                             development of error analysis to select the best criteria [Nobile 2008,
Intrude or not                               Crestaux 2008].
intrude ?


Open issues


                         Introduction to Stochastic Spectral Methods               25 Novembre 2008               133 / 133

								
To top