VIEWS: 45 PAGES: 133 POSTED ON: 7/13/2011
Introduction RVs and RPs Several UQ methods Spectral Introduction to Stochastic Spectral Methods representation e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial Chaos Didier LUCOR Post-processing Resolution for a Institut Jean Le Rond d’Alembert general SPDE Stochastic Galerkin e Equipe Fluides R´actifs et Turbulence Method (SGM) Stochastic Collocation e Universit´ Pierre et Marie Curie (Paris VI) Method (SCM) Multivariate quadratures Full ECODOQUI Sparse 25 Novembre 2008 Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 1 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 2 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 3 / 133 Introduction Introduction RVs and RPs Several UQ methods Modeling uncertainties, numerical errors and data uncertainties can Spectral representation interact. e Karhunen-Lo`ve Polynomial Chaos Aleatoric/stochastic (not reducible) or epistemic (incomplete generalized Polynomial knowledge) uncertainty. Chaos Post-processing Need to quantitatively access the impact of uncertain data on Resolution for a simulation outputs. general SPDE Stochastic Galerkin Method (SGM) The validity of the model can be established only if uncertainty in Stochastic Collocation numerical predictions due to uncertain input parameters can be Method (SCM) quantiﬁed. Multivariate quadratures Diﬃculty: not looking for the unique solution. Full Sparse Possible sources: simulation constants/parameters, transport Intrude or not coeﬃcients, physical properties, boundary/initial conditions, geometry, intrude ? models, numerical schemes, ... Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 4 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 5 / 133 Random variables (RVs) Introduction Complete probability space: Ω, A, P , where Ω is the event space, A ⊂ 2Ω RVs and RPs ` ´ Several UQ methods the σ-algebra and P the probability measure. Spectral ` ´ representation X : Ω, A, P → R, with probability density function (pdf) fX and e Karhunen-Lo`ve Polynomial Chaos cumulative density function (cdf) FX . generalized Polynomial Chaos Random vector : X = {Xi (ω)}N , N ∈ N. i=1 Post-processing ˜ ˜ uncorrelated if : cov(X1 , X2 ) = E(X1 ⊗ X2 ) = 0 Resolution for a general SPDE Stochastic Galerkin independent if : E[φ1 (X1 ) φ2 (X2 )] ≡ E[φ1 (X1 )] E[φ2 (X2 )] Method (SGM) Stochastic Collocation Method (SCM) Expectation operator : if u ∈ L1 (Ω), P Z Z Multivariate quadratures E[u] =< u >= u(ω)dP (ω) = u(x )fX (x )dx Ω R Full Sparse ¯ u = E[u] Intrude or not intrude ? varu = E[˜ 2 ] u where u =u −u ˜ ¯ P (u ≤ u0 ) = P ({ω ∈ Ω : u(ω) ≤ u0 }) = E[1{u≤u0 } ] Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 6 / 133 Random processes (RPs) Introduction Random process (RP) u(x´ ω) indexed by a bounded domain D ⊂ Rd on the RVs and RPs Several UQ methods ` , probability space Ω, A, P . Spectral representation 1. It is a set of RVs indexed by x ∈ D. For each x = x0 , u(x = x0 , ω) is a e Karhunen-Lo`ve Polynomial Chaos RV on Ω. It is deﬁned by all its ﬁnite-dimensional distributions generalized Polynomial functions. Fx1 ,...,xN = P {u(x1 ) ≤ x1 . . . u(xN ) ≤ xN } ˆ ˆ Chaos Post-processing 2. It is a function-valued RV. u is a function of D × Ω with value u(x , ω). Resolution for a for given x ∈ D and ω ∈ Ω. For each ﬁxed ω ∈ Ω, u(x , ω) is a function general SPDE - a realization - of x in D. Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) A random ﬁeld (RF) is a measurable function : u : D × Ω → R. Multivariate Expectation operator : Z Z quadratures Full E[u] = u(x , ω) = u(x , ω)dP (ω) = u(x , x )fX (x )dx Sparse Ω R Intrude or not u (x ) = E[u(x )], varu (x ) = E[˜ (x )2 ] where u (x ) = u(x ) − u (x ) ¯ u ˜ ¯ intrude ? P (u(x ) ≤ u0 ) = P ({ω ∈ Ω : u(x , ω) ≤ u0 }) = E[1{u(x )≤u0 } ] Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 7 / 133 Special case of Gaussian random ﬁelds Introduction RVs and RPs Several UQ methods A random ﬁeld, for which all ﬁnite-dimensional distributions are jointly Spectral Gaussian. it is suﬃcient to specify its second order statistics, i.e. its mean representation and its covariance function. e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial Convenient properties: Chaos Post-processing Uncorrelated Gaussian RVs → independent Resolution for a general SPDE Linear combinations of Gaussian RVs are also Gaussian Stochastic Galerkin Method (SGM) Gaussian ﬁelds occur naturally due to the CLT Stochastic Collocation Method (SCM) Second order statistical information is often the only one available in Multivariate applications quadratures Full Gaussian random ﬁelds are the maximum entropy model Sparse Intrude or not However, they may hence be inappropriate for describing positive or strictly intrude ? positive RPs. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 8 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 9 / 133 Statistical methods Introduction RVs and RPs Several UQ methods √ Spectral representation Brute-force Monte Carlo method : Converges as 1/ N ; Convergence Karhunen-Lo`ve e rate independent of number of RVs. Robust. Parallelizable. Polynomial Chaos generalized Polynomial Chaos Post-processing Monte Carlo based methods : Resolution for a QMC (Quasi-MC) general SPDE Stochastic Galerkin MCMC (Markov chain MC) Method (SGM) Stochastic Collocation Method (SCM) Variance reduction technique : limitation with large number of RVs (importance sampling, correlated sampling, conditional sampling, ...) Multivariate quadratures Full Response Surface Method (RSM) : realizations reduced by interpolation Sparse in state space; same limitation with large number of RVs. Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 10 / 133 Non-statistical methods Introduction RVs and RPs Several UQ methods Indirect methods : Spectral Fokker-Planck equation : Solves for distribution function; Challenging in representation high dimensions (computational cost), BCs e Karhunen-Lo`ve Polynomial Chaos Moments equations : Closure of equations is key. Good for linear generalized Polynomial Chaos problems with Gaussian RVs Post-processing Resolution for a Direct methods (e.g. SFEM, stochastic ﬁnite element method) : general SPDE Stochastic Galerkin Method (SGM) Interval analysis : maximum output bounds Stochastic Collocation Method (SCM) Perturbation-based methods : Taylor expansion around means. Diﬀer at the local representation of randomness: mid-point, local average, Multivariate piecewise polynomial, etc quadratures Full Operator-based methods : Weighted integral method; Neumann Sparse expansion Intrude or not e Stochastic spectral methods : Polynomial chaos, Karhunen-Lo`ve, intrude ? Wiener-Askey/gPC chaos & representation [Wiener 1938, Lo`ve 1977, e Open issues Cameron & Martin 1947, Ghanem & Spanos 1991, Xiu & Karniadakis 2002]. Introduction to Stochastic Spectral Methods 25 Novembre 2008 11 / 133 Stochastic spectral methods for UQ Introduction RVs and RPs Several UQ methods Spectral Means of representing 2nd order RPs parametrically through a ﬁnite set representation Karhunen-Lo`ve e of RVs (ﬁnite dimensional noise assumption). Polynomial Chaos generalized Polynomial Chaos Projection of the solution onto the space spanned by those RVs. Post-processing Pros: not limited to small uncertainties with Gaussian distributions, Resolution for a general SPDE convergence rate, richness of the information (not only solution stats), Stochastic Galerkin Method (SGM) does not require high skills in prob. or stat. Stochastic Collocation Method (SCM) Cons: dimensionality, robustness (stability issues for non-linear and Multivariate instationary problems), solvers modiﬁcation. quadratures Full Sparse Future eﬀorts: computational eﬃciency, error estimates, adaptivity, sparsity, reduced basis. Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 12 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 13 / 133 Finite dimensional noise assumption Introduction RVs and RPs Several UQ methods Need to reduce the inﬁnite-dimensional probability space to a Spectral ﬁnite-dimensional space. representation e Karhunen-Lo`ve Polynomial Chaos Characterizing the probability space by a countable number N of generalized Polynomial Chaos mutually independent RVs. Post-processing Resolution for a Approximation of the target random process (decomposition). general SPDE Stochastic Galerkin Method (SGM) The random ﬁeld has been represented in a ﬁnite number of Stochastic Collocation Method (SCM) independent random variables −→ function on a high dimensional space. Multivariate quadratures Full The independence of the underlying RVs allows to see each of them as Sparse the axis of a coordinate system (Doob-Dynkin lemma). Intrude or not intrude ? e One possible choice : the Karhunen Lo´ve type expansion. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 14 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 15 / 133 KL representation - Derivation of the expansion (1) Introduction RVs and RPs Several UQ methods The Karhunen-Lœve (KL) expansion of a RP is a Fourier-type series based on the spectral expansion of its covariance function of a RP. Spectral representation Karhunen-Lo`ve e We consider a second-order RP : u(x , ω) ∈ L2 (D × Ω) ⇒ E[u(x )2 ] < +∞, Polynomial Chaos ∀x ∈ D ⊂ Rd , and its covariance Ru (x 1 , x 2 ) ∈ (D × D) : generalized Polynomial Chaos Post-processing Ru (x 1 , x 2 ) = E(u(x 1 , ω) ⊗ u(x 2 , ω)) 1 1 Resolution for a general SPDE 0.9 0.9 0.8 0.8 Stochastic Galerkin Method (SGM) 0.7 0.7 Covariance Covariance 0.6 0.6 Stochastic Collocation Method (SCM) 0.5 0.5 0.4 0.4 0.3 Exponential 0.3 Exponential Multivariate Gaussian Gaussian 0.2 0.2 quadratures Sine Sine 0.1 Triangular 0.1 Triangular Full Linear Exponential Linear Exponential 0 0 −1 −0.5 0 0.5 1 !1 !0.5 0 0.5 1 Sparse Lag Lag 1D covariance kernels : L = 2 and correlation length of Cl = L/2 and Cl = L/8. Intrude or not intrude ? They demonstrate how fast the correlation drop between two distant points. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 16 / 133 KL representation - Derivation and expansion (2) Introduction RVs and RPs Several UQ methods We can deﬁne a covariance operator such that: Z Spectral representation Ru : L2 (D) → L2 (D), (Ru u)(x ) := Ru (x 1 , x 2 )u(x 2 )d x 2 , ∀u ∈ L2 (D) Karhunen-Lo`ve e D Polynomial Chaos generalized Polynomial The covariance operator is real, symmetric and positive-deﬁnite. Chaos Post-processing It has a countable sequence of eigenpairs with: eigenvalues {λi } ⊂ R+ and Resolution for a orthogonal eigenfunctions φi (x ) (complete basis). general SPDE Stochastic Galerkin Method (SGM) Spectral representation of the kernel : Stochastic Collocation ∞ X Method (SCM) Ru (x 1 , x 2 ) = λi φi (x 1 )φi (x 2 ), Multivariate i=1 quadratures Full Second-order Fredholm equation (for the eigenvalues and eigenfunctions): Sparse Z Z Intrude or not Ru (x 1 , x 2 )φi (x 2 )d x 2 = λi φi (x 1 ) with φi (x )φj (x )d x = δij intrude ? D D Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 17 / 133 KL representation - Derivation and expansion (3) Introduction RVs and RPs Several UQ methods The KL representation of u(x , ω) is: Spectral ∞ representation X√ e Karhunen-Lo`ve ¯ u(x , ω) = u (x ) + σu λi φi (x )Xi (ω), Polynomial Chaos i=1 generalized Polynomial Chaos with Xi : centred, normalized, uncorrelated RVs (but not necessarily Post-processing independent); EXi = 0, E(Xi Xj ) = δij . Resolution for a general SPDE Xi (ω) are evaluated from: Stochastic Galerkin Z Method (SGM) 1 ` ´ Stochastic Collocation Xi (ω) = u(x , ω) − u (x ) φi (x )d x ¯ Method (SCM) λi D Multivariate quadratures In practice : diﬀerent ways of dealing with those RVs. Due to the fact that Full there exists an inﬁnity of second-order RPs sharing the same given Sparse covariance kernel (they all hold the same decomposition. But, they do not Intrude or not have necessarily the same RVs). intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 18 / 133 KL representation - Derivation and expansion (4) Introduction RVs and RPs Practically, we can only deal with ﬁnite representation and the KL expansion Several UQ methods has to be truncated. We have: Spectral representation N X√ Karhunen-Lo`ve e u N (x , ω) = u (x ) + σu ¯ λi φi (x )Xi (ω), Polynomial Chaos i=1 generalized Polynomial Chaos Post-processing The more modes N we add into the KL expansion, the more small-scale details we capture with the representation. Resolution for a general SPDE Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Multivariate quadratures Full Sparse Intrude or not intrude ? KL representation of groundwater ﬂow stochastic conductivity for diﬀerent level of resolution (Courtesy of Hermann Matthies, TU Braunschweig) Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 19 / 133 KL representation - Eigenvalues distributions Introduction RVs and RPs Eigenvalues of the exponential kernel and Gaussian kernel for diﬀerent Several UQ methods correlation lengths. (a): Cl = 1; (b): Cl = 0.1 Spectral (a) (b) representation 10 0 10 0 Exponential Kernel Exponential Kernel Karhunen-Lo`ve e Gaussian Kernel Gaussian Kernel −1 10 Polynomial Chaos −5 10 generalized Polynomial −2 Eigenvalues Eigenvalues 10 Chaos Post-processing −3 −10 10 10 Resolution for a 10 −4 general SPDE −15 10 −5 Stochastic Galerkin 0 5 10 15 20 10 0 5 10 15 20 Method (SGM) n n Stochastic Collocation Method (SCM) More modes are needed as the correlation decreases (i.e. the noise level Multivariate quadratures increases). Full Sparse In the asymptotic limit of white noise, an inﬁnity number of modes will be needed. Intrude or not intrude ? For a given correlation length, the smoothness of the covariance kernel Open issues impacts the convergence rate of the eigenvalues. Introduction to Stochastic Spectral Methods 25 Novembre 2008 20 / 133 KL representation - Example of KL eigenfunctions Introduction RVs and RPs Distributions of the eigenfunctions of the exponential kernel (top row) and Several UQ methods Gaussian kernel (bottom row) for diﬀerent correlation lengths Cl . Spectral Cl = 1 Cl = 0.1 1.5 1.5 representation e Karhunen-Lo`ve 1 1 Polynomial Chaos 0.5 0.5 generalized Polynomial Chaos fn(y) fn(y) 0 0 Post-processing −0.5 N=1 −0.5 N=1 N=2 N=2 Resolution for a N=3 N=3 −1 N=4 −1 N=4 general SPDE N=5 N=5 N=6 N=6 Stochastic Galerkin −1.5 −1.5 −0.5 0 0.5 −0.5 0 0.5 Method (SGM) y y Stochastic Collocation Cl = 1 Cl = 0.1 4 Method (SCM) 1.5 3 1 Multivariate 2 quadratures 0.5 1 Full f (y) fn(y) 0 0 n Sparse −1 N=1 −0.5 N=1 −2 N=2 N=2 Intrude or not N=3 N=3 intrude ? N=4 −1 N=4 −3 N=5 N=5 N=6 N=6 −4 −1.5 −0.5 0 0.5 −0.5 0 0.5 Open issues y y Introduction to Stochastic Spectral Methods 25 Novembre 2008 21 / 133 Error minimizing property Introduction RVs and RPs Several UQ methods If we truncate after the N largest eigenvalues, we have an optimal - in Spectral representation variance - expansion in N RVs. e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial There is no other linear representation with N terms that has a smaller Chaos mean square quadratic error 2 .N Post-processing Resolution for a general SPDE u N converges to u in variance uniformly : Stochastic Galerkin Method (SGM) ∞ X sup E (u(x ) − u N (x ))2 = sup λi ui2 (x ) → 0, as m → ∞ Stochastic Collocation ` ´ Method (SCM) x ∈D x ∈D i=N +1 Multivariate quadratures Full The KL representation based on the eigenfunctions of the covariance Sparse kernel is the only representation that will involved non-correlated Intrude or not (orthogonal) RVs. intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 22 / 133 Convergence rate of the spectrum (1) Introduction RVs and RPs Several UQ methods The truncation error decreases monotonically with the number of terms Spectral representation in the expansion. e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial The convergence is inversely proportional to the correlation length and Chaos Post-processing depends on the regularity of the covariance kernel. Resolution for a general SPDE if Ru is piecewise analytic on D × D, with D ⊂ Rd [Frauenfelder 2005] : Stochastic Galerkin Method (SGM) Stochastic Collocation λn ≤ c1 exp(−c2 n 1/d ), ∀m ≥ 1, c1 , c2 > 0 ind. of n Method (SCM) Multivariate where c1 , c2 > 0 are constants independent of n; quadratures Full Sparse if Ru is piecewise H k ,0 ≡ H k ⊗ L2 with k > 0, we have : Intrude or not λn ≤ c3 n −k /d , ∀n ≥ 1, c3 > 0 intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 23 / 133 Convergence rate of the spectrum (2) Introduction RVs and RPs Several UQ methods Decay of eigenvalues for the Gaussian kernel function. Spectral representation Left : Eigenvalues in 3D domain and theoretical estimate. e Karhunen-Lo`ve Polynomial Chaos Right : Eigenvalue decays in 1D, 2D, and 3D domains. generalized Polynomial Chaos Exponential decay of eigenvalues in all three cases with smaller decay Post-processing rate at higher dimensions. Resolution for a general SPDE 1 Largest 2000 Eigenvalues of Factorizable Kernel exp(!10|x!x!|2), x " (!1,1)3 2 10 10 computed eigenvalues # m Stochastic Galerkin estimated eigenvalues 1/ $(m1/3/2) 0 Method (SGM) 0 10 10 Stochastic Collocation !2 10 Method (SCM) !1 10 !4 10 !2 10 Eigenvalue Multivariate !6 10 m quadratures # !3 10 !8 Full 10 !4 Sparse 10 !10 10 !5 10 !12 10 Intrude or not 1D Gaussian kernel 2D Gaussian kernel intrude ? !6 !14 3D Gaussian kernel 10 10 0 200 400 600 800 1000 1200 1400 1600 1800 2000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 m Index Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 24 / 133 Convergence rate of the spectrum (3) Introduction RVs and RPs 14 Largest Eigenvalues of exp(!|x!x!|1+"), x# (!1,1) Several UQ methods 1 10 "=0.00, observed "=0.00, predicted "=0.25, observed Spectral Decay of eigenvalues for the covariance 0 10 "=0.25, predicted "=0.50, observed "=0.50, predicted representation kernel function Ru = exp(−|x1 − x2 |1+δ ). "=0.75, observed "=0.75, predicted e Karhunen-Lo`ve !1 10 Polynomial Chaos generalized Polynomial For smaller values of δ, the eigenvalues !2 10 Chaos decay slower, as predicted by the estimate. Post-processing !3 10 Resolution for a general SPDE !4 10 0 2 4 6 8 10 12 14 Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) In another study [Lucor 2004], the convergence rate for a RP with an exponential covariance function over a temporal domain T : Multivariate quadratures 2 h 2 (x ; ω) − hN (x ; ω) T 1 Full ε≡ 2 (x ; ω) ∼ 0.4053 . Sparse h Cl N Intrude or not Numerical simulations for random oscillators established that this indeed is a intrude ? very sharp error estimate. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 25 / 133 Special case of Gaussian processes Introduction RVs and RPs Several UQ methods A very convenient special case is the one where the RP to represent u(x , ω) is a Gaussian RP. Spectral representation Karhunen-Lo`ve e In this case, it has a KL representation with RVs ξi (ω) : Gaussian vector. Polynomial Chaos generalized Polynomial These Gaussian RVs are uncorrelated ⇒ independent. Chaos Post-processing N X√ ¯ u(x , ω) = u (x ) + σu λi φi (x )ξi (ω), Resolution for a i=1 general SPDE Stochastic Galerkin Method (SGM) The task of numerically representing continuous non-Gaussian random ﬁelds Stochastic Collocation via a KL decomposition that the random variables involved are independent, Method (SCM) falls into the topic of numerical representation of non-Gaussian processes is Multivariate not a trivial task. quadratures Full That it is possible to construct multidimensional functional spaces based on Sparse ﬁnite number of dependent random variables [Ghanem & Soize]. Intrude or not intrude ? However, such a construction does not, in its current form, allow straightforward numerical implementations. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 26 / 133 Choices of covariance kernel functions (1) Introduction RVs and RPs A covariance function can be generated through ﬁrst-order Several UQ methods autoregression model or ﬁrst-order Markov process. Spectral representation ui = αui−1 + βXi , α = e −1/γ , α2 + β 2 = 1, e Karhunen-Lo`ve Polynomial Chaos Correspondence of random processes and their covariance functions : generalized Polynomial [Su 2006] Chaos Post-processing Dimension d Random process u(x ; ω) Covariance function Ru “ ” d =1 ui = αui−1 + βXi exp − C r Resolution for a l general SPDE d =1 b ui = 2 (ui−1 + ui+1 ) + Xi (1 + r /Cl ) exp(−r /Cl ) “ ” Stochastic Galerkin d =2 b ui,j = 4 (ui±1,j + ui,j ±1 ) + Xi,j r K r Method (SGM) Cl “ 1 C l” Stochastic Collocation d =3 b ui,j ,k = 6 (ui±1,j ,k + ui,j ±1,k + ui,j ,k ±1 ) + Xi,j ,k exp − C r Method (SCM) l d = 1, 2, 3 N/A exp(−r 2 ) Multivariate quadratures Full The corresponding continuous process satisﬁes approximation the Sparse Helmholtz equation : ∂2u u β − 2 =2 X. Intrude or not ∂x 2 Cl ∆x 2 intrude ? Solving the equation with periodic boundary conditions produces the Open issues solution u(x , X ) from which the covariance can then be computed. Introduction to Stochastic Spectral Methods 25 Novembre 2008 27 / 133 Choices of covariance kernel functions (2) Introduction RVs and RPs Several UQ methods Spectral representation e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial Chaos Post-processing Resolution for a general SPDE Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Multivariate quadratures Full Sparse Intrude or not intrude ? KL realizations of RPs in two spatial dimensions for diﬀerent covariance kernels (rows) and Open issues diﬀerent correlation lengths (columns). (Courtesy of Andreas Keese, TU Braunschweig). Introduction to Stochastic Spectral Methods 25 Novembre 2008 28 / 133 Rational spectra Introduction RVs and RPs Several UQ methods If we consider a certain class of RPs, that are stationary output of linear Spectral representation ﬁlters to white noise excitation, they have a spectral density that takes the Karhunen-Lo`ve e general form: Polynomial Chaos N (ω 2 ) S (ω) = , generalized Polynomial Chaos D(ω 2 ) Post-processing where N and D are polynomials of the angular frequency ω of order n and d Resolution for a respectively. general SPDE Stochastic Galerkin Method (SGM) The nice property is that the Fredholm equation can be transformed in this Stochastic Collocation Method (SCM) case into a second order homogeneous diﬀerential equation that yields: Multivariate ˆ d2 ˜ ˆ d2 ˜ quadratures λD φ(x 1 ) = N φ(x 1 ) Full dx21 dx21 Sparse This equation can be solved in terms of λ. There exists in fact some analytic Intrude or not or semi-analytic solutions for a few spectrums [Ghanem & Spanos 1991]. intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 29 / 133 Rational spectra - exponential kernel (1) Introduction RVs and RPs Several UQ methods Let us consider the ﬁrst-order, stationary, Markovian process in space Spectral u(x , ω) (or in time) with exponential correlation function deﬁned as representation Karhunen-Lo`ve e Ru (x1 , x2 ) = σu e −|x2 −x1 |/Cl 2 ∀(x1 , x2 ) ⊂ D × D, Polynomial Chaos 2 generalized Polynomial where is the variance, Cl is the correlation length and σu Chaos Post-processing D = [−L/2, L/2] ∈ R. Resolution for a The u(x , ω) process can be shown to be the stationary solution of the general SPDE Stochastic Galerkin diﬀerential equation: Method (SGM) r 1 2 Stochastic Collocation u(x ) = − u(x ) + σu ˙ W (x ), Method (SCM) Cl Cl Multivariate in which W (x ) is the zero-mean stationary white noise with covariance quadratures Full function δ(x ). Sparse It has been shown that higher order Markovian kernels may be expressed as Intrude or not intrude ? linear combinations of ﬁrst order ones. This model is widely used in the literature. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 30 / 133 Rational spectra - exponential kernel (2) Introduction RVs and RPs Several UQ methods The integral Fredholm equation can be diﬀerentiated twice which gives a Spectral diﬀerential equation of the form: representation 1 e Karhunen-Lo`ve (2 − Cl λ) Polynomial Chaos φ (x ) + γ 2 φ(x ) = 0 ∀x ∈ [−L/2, L/2], with γ2 = , generalized Polynomial Cl λ Chaos Post-processing where γ can be numerically solved by evaluating the diﬀerential equation at Resolution for a the boundaries of the domain. general SPDE ∗ Stochastic Galerkin The variables γn and γn are the ﬁrst n roots of those non-linear equations Method (SGM) Stochastic Collocation 1 L Method (SCM) − γ tan(γ ) = 0 Cl 2 Multivariate quadratures ∗ 1 ∗L γ − tan(γ ) = 0, Full Cl 2 Sparse where L is the reference length of the spatial domain. Intrude or not intrude ? Typically, one can use a Newton-Raphson algorithm to evaluate the roots. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 31 / 133 Rational spectra - exponential kernel (3) Introduction RVs and RPs Several UQ methods Once this is done, the eigenvalues are: Spectral representation 2 2 e Karhunen-Lo`ve λn = and λ∗ = n . Polynomial Chaos Cl (γn + Cl2 ) 2 Cl (γn + Cl2 ) ∗2 generalized Polynomial Chaos The corresponding eigenfunctions are thus Post-processing Resolution for a cos(γn x ) general SPDE φn (x ) = r , Stochastic Galerkin L sin(2γn L ) 2 Method (SGM) 2 + 2γn Stochastic Collocation Method (SCM) ∗ sin(γn y) φ∗ (x ) n = r , Multivariate sin(2γn L ) ∗ L quadratures 2 − 2γn∗ 2 Full Sparse and they form the symmetric and the anti-symmetric components in the KL Intrude or not intrude ? expansion. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 32 / 133 Rational spectra - exponential kernel (4) Introduction RVs and RPs The 4-term and 10-term KL approximation of the exponential covariance Several UQ methods surface versus x1 and x2 with L = 1 and Cl = L/2 (left column), and the Spectral corresponding relative error surfaces (right column). representation e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial Chaos Post-processing Resolution for a general SPDE Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Multivariate quadratures Full Sparse Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 33 / 133 Rational spectra - exponential kernel (5) Introduction RVs and RPs Several UQ methods Variance computed from the KL representation of the RP with a total of N Spectral representation modes. Left : Cl = 1.0 and right : Cl = 0.1. e Karhunen-Lo`ve Polynomial Chaos Cl=1.0 Cl=0.1 generalized Polynomial 1 1 Chaos 0.95 Post-processing 0.99 0.9 Resolution for a 0.85 0.98 general SPDE 0.8 Var(CN) Var(CN) Stochastic Galerkin Method (SGM) 0.97 0.75 Stochastic Collocation 0.7 Method (SCM) 0.96 N=8 N=8 0.65 N=16 N=16 N=24 0.6 N=24 Multivariate 0.95 N=32 N=32 quadratures N=40 0.55 N=40 Full 0.94 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5 −0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5 Sparse lag lag Intrude or not intrude ? Maximum error is at the boundaries of the domain. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 34 / 133 Non-rational spectra Introduction RVs and RPs Much more diﬃcult to derive the solution of non-rational spectra. Several UQ methods Possible to derive an ODE from the Fredholm integral equation, but we Spectral representation only have analytical solutions for a few RPs. Karhunen-Lo`ve e The Fredholm integral equation can be solved by standard techniques Polynomial Chaos generalized Polynomial (means of a variational formulation, Galerkin-type or collocation-type Chaos methods). Post-processing In projection methods, approximation of the eigenfunctions : Resolution for a κn general SPDE X Stochastic Galerkin φn (x ) = m am,k vk (x ), x ∈ D, m = 1, 2, · · · . Method (SGM) k =1 Stochastic Collocation Method (SCM) The Galerkin approximation : ﬁnd λm = 0 and φn ∈ Vn such that m Z Z Rh (x1 , x2 )φn (x2 )v (x )dxdx2 = λm φn (x )v (x )dx , ∀v ∈ Vn . Multivariate quadratures m m Full D×D D Sparse Matrix eigenvalue problem : Av = λMv, where matrices A and M are Intrude or not symmetric and positive deﬁnite. intrude ? Fast Multipole Method with O(κn log(κn )) instead of O(κ2 ) n Open issues operations. Introduction to Stochastic Spectral Methods 25 Novembre 2008 35 / 133 Non-rational spectra - Gaussian kernel (1) Introduction RVs and RPs A smooth kernel, the Gaussian kernel (analytic fct. mean-square Several UQ methods diﬀerentiable of any order) : Spectral −(x2 −x1 )2 representation 2 L2 C 2 Karhunen-Lo`ve e Ru (x1 , x2 ) = σu e l ∀(x1 , x2 ) ⊂ D × D, Polynomial Chaos 2 generalized Polynomial where L is the diameter of the domain D and σu and Cl are the standard Chaos deviation and the correlation length, respectively. Post-processing Resolution for a general SPDE Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Multivariate quadratures Full Sparse Intrude or not intrude ? The 4-term KL approximation of the Gaussian covariance surface versus x1 and x2 with L = 1 Open issues and Cl = L/2, and the corresponding relative error surfaces of these approximations. Introduction to Stochastic Spectral Methods 25 Novembre 2008 36 / 133 Non-rational spectra - Gaussian kernel (2) Introduction RVs and RPs Several UQ methods Spectral The convergence in the relative L2 -norm error of the Gaussian kernel and its representation variance with increasing number of expansion, N. e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial 0 0 10 10 Chaos Cl=0.01 Cl=0.01 Post-processing Cl=0.1 Cl=0.1 Relative L2−norm error of variance Relative L −norm error of kernel −4 Cl=1 −4 Cl=1 10 Cl=10 10 Cl=10 Resolution for a Cl=100 Cl=100 general SPDE Stochastic Galerkin −8 −8 Method (SGM) 10 10 Stochastic Collocation 2 Method (SCM) −12 −12 10 10 Multivariate quadratures Full −16 −16 10 10 Sparse 0 5 10 15 20 25 30 35 40 0 5 10 15 20 25 30 35 40 N N Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 37 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 38 / 133 Deﬁnitions and convergence properties (1) Introduction RVs and RPs Several UQ methods The Polynomial Chaos (PC) is an orthogonal development for non-linear functionals with Gaussian measure. Spectral representation Let Θ ⊆ L2 (Ω, B, P ) be a separable Gaussian Hilbert space. The space Θ is e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial a subspace of L2 (Ω, B, P ) that only contains centered Gaussian RVs Chaos Post-processing ξ := {ξj }N and that is complete (the covariance as scalar product). j =1 Resolution for a general SPDE The space PlN of all N -variate polynomials of degree l is denoted by: Stochastic Galerkin Method (SGM) Stochastic Collocation PlN (Θ) := {p(ξ) | p is polynomial of degree l ; ξ ∈ Θ, N < ∞} Method (SCM) Multivariate The space of all polynomials is called P(Θ) := ∞ PlN (Θ) and we denote S quadratures l=0 Full ¯ by PlN the closure with respect to L2 and deﬁne Sparse N H=0 := ¯N P0 , Intrude or not intrude ? N H=l := ¯ ¯N PlN Pl−1 for l ∈N Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 39 / 133 Deﬁnitions and convergence properties (2) Introduction RVs and RPs Several UQ methods N H=l is called the homogeneous chaos of order l Spectral representation Sl N N Karhunen-Lo`ve e H≤l := H=l is called the polynomial chaos of order l i=0 Polynomial Chaos ∞ M N generalized Polynomial Chaos L2 (Ω, Σ(Θ), P ) = H=l is called the polynomial chaos Post-processing l=0 decomposition Resolution for a general SPDE Stochastic Galerkin An orthogonal basis of the PC space are the multivariate Hermite N Method (SGM) polynomials H≤P of degree P < ∞ in N independent Gaussian RVs. Stochastic Collocation Method (SCM) Multivariate Those polynomials can be constructed as tensor-products of univariate 1 quadratures Hermite polynomials HP ≡ HP . Full Sparse The entire family of polynomials can be written in a compact fashion by Intrude or not intrude ? means of multi-indices. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 40 / 133 Deﬁnitions and convergence properties (3) Introduction RVs and RPs Several UQ methods Based on the previous PC decomposition, any Σ(Θ)-measurable RV on Ω (not necessarily Gaussian) with ﬁnite variance has a L2 convergent Spectral representation approximation in the multivariate Hermite polynomial space [Cameron & Martin Karhunen-Lo`ve e 1947]. This convergence is in the mean square sense. Polynomial Chaos generalized Polynomial Chaos The RV u(ω) can be represented in terms of orthogonal Hermite polynomials Post-processing of independent normalized Gaussian RVs, ξ = {ξi (ω)}∞ : i=1 ∞ X Resolution for a general SPDE u(ω) = ˆ uk Hk (ξ(ω)), Stochastic Galerkin k =0 Method (SGM) Stochastic Collocation ˆ where the deterministic coeﬃcients uk are deﬁned as: Method (SCM) Z Multivariate u, Hk quadratures ˆ uk = where u, Hk = E[u(ω)Hk (ξ(ω))] = u(ξ)Hk (ξ(ω))ρ(ξ)d ξ, Hk , Hk Γ Full Sparse 2 e −ξi /2 QN with ρ(ξ) = i=1 √ 2π . Intrude or not intrude ? Orthogonality condition : < Hi , Hj >= E[Hi , Hj ] = E[Hi2 ] δij . Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 41 / 133 Truncated PC representation : Curse of dimensionality Introduction RVs and RPs In practice, stochastic ﬁelds can be approximated by a PC of a certain ﬁnite Several UQ methods order, corresponding to a ﬁnite set of N random variables and a maximum polynomial order P . Spectral representation Karhunen-Lo`ve e Due to the tensor form construction of the PC multi-dimensional basis, an Polynomial Chaos expansion in N independent stochastic dimensions, and of highest order P generalized Polynomial Chaos has a total number of terms M : Post-processing „ « M P +N dim(HP ) = M = Resolution for a N general SPDE Stochastic Galerkin Method (SGM) Vector space dimensions of the polynomial chaos of degree P in N Stochastic Collocation independent RVs : Method (SCM) P=2 P=3 P=4 P=5 P=6 P=7 P=8 P=9 P = 10 Multivariate N = 2 6 10 15 21 28 36 45 55 66 quadratures N = 3 10 20 35 56 84 120 165 220 286 Full N = 4 15 35 70 126 210 330 495 715 1 001 Sparse N = 5 21 56 126 252 462 792 1 287 2 002 3 003 N = 6 28 84 210 462 924 1 716 3 003 5 005 8 008 Intrude or not N = 7 36 120 330 792 1 716 3 432 6 435 11 440 19 448 intrude ? N = 8 45 165 495 1 287 3 003 6 435 12 870 24 310 43 758 N = 9 55 220 715 2 002 5 005 11 440 24 310 48 620 92 378 N = 10 66 286 1 001 3 003 8 008 19 448 43 758 92 378 184 756 Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 42 / 133 One-dimensional Hermite polynomials Introduction RVs and RPs We consider the univariate Hermite polynomials H (ξ) that will serve as the Several UQ methods foundation for the construction of the multi-dimensional Hermite Spectral polynomials (orthogonal with respect to the Gaussian measure). representation Karhunen-Lo`ve e 1 ξ2 Polynomial Chaos The pdf of a Gaussian RV ξ is deﬁned as :ρ(ξ) = √ e − 2 . generalized Polynomial 2π Chaos Post-processing 30 Resolution for a 20 general SPDE Stochastic Galerkin Method (SGM) 10 Stochastic Collocation Method (SCM) 0 Multivariate quadratures Full −10 Sparse −20 Intrude or not intrude ? −30 −3 −2 −1 0 1 2 3 ξ Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 43 / 133 Multi-dimensional Hermite polynomials construction (1) Introduction RVs and RPs Several UQ methods For (N > 1), one can build the multi-dimensional PC basis as Spectral tensor-products of univariate polynomials. representation Karhunen-Lo`ve e For Hermite polynomials, a simple way to construct the k th polynomial N 1 Polynomial Chaos Hk (ξ) is to tensorize one-dimensional Hermite polynomials Hki (ξi ). generalized Polynomial Chaos Post-processing We deﬁne the multi-index k = {k1 . . . ki , . . . kN }, such that we have: Resolution for a N Y general SPDE N 1 Hk (ξ) = Hki (ξi ), Stochastic Galerkin Method (SGM) i=1 Stochastic Collocation Method (SCM) and k := {ki }N is a sequence whose each component refers to the i=1 Multivariate 1 polynomial degree p of the i th one-dimensional polynomial Hp≤P (ξi ) quadratures N Full contributing to Hk (ξ). Sparse If we deﬁne the modulus of k by |k | = N ki , the set of all Hk (ξ) with N P i=1 N N Intrude or not intrude ? |k | = p is an orthogonal basis of Hp and the set of all Hk (ξ) with |k | ≤ P N is a polynomial chaos of order P : H≤P . Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 44 / 133 Multi-dimensional Hermite polynomials construction (2) Introduction RVs and RPs Examples: We consider ﬁrst a multi-dimensional Hermite polynomial of order Several UQ methods at most P = 3 in N = 2 Gaussian independent RVs {ξ1 , ξ2 }. Spectral representation We list the M = 10 multi-indices k by order of appearance for this case: e Karhunen-Lo`ve » – Polynomial Chaos 0 1 0 2 1 0 3 2 1 0 generalized Polynomial k (N = 2; P = 3) = . Chaos 0 0 1 0 1 2 0 1 2 3 Post-processing The ﬁrst row corresponds to the k1 values and the second one to the k2 Resolution for a general SPDE values. Stochastic Galerkin Method (SGM) Vertical lines correspond to the limitations between diﬀerent polynomial Stochastic Collocation order representations. Method (SCM) 2 The ﬁrst polynomial is a constant: H0 (ξ1 , ξ2 ) = 1. Multivariate quadratures We have, for instance for the 8-th polynomial of the basis: Full Sparse N =2 Y 2 1 1 1 H7 (ξ1 , ξ2 ) = Hki (ξi ) = Hk1 (ξ1 ) ⊗ Hk2 (ξ2 ) Intrude or not intrude ? i=1 1 1 2 2 = H1 (ξ1 ) × H2 (ξ2 ) = (ξ1 ) × (ξ2 − 1) = ξ1 ξ2 − ξ1 . Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 45 / 133 Multi-dimensional Hermite polynomials construction (3) Introduction RVs and RPs 2 3 3 Several UQ methods 1.5 2 2 1 1 1 0 0 !1 !1 0.5 !2 !2 Spectral 0 3 !3 3 !3 3 representation 2 1 0 0 1 2 3 2 1 0 0 1 2 3 2 1 0 0 1 2 3 !1 !1 !1 !2 !1 !2 !1 !2 !1 !2 !2 !2 Karhunen-Lo`ve e x2 !3 !3 x1 (a) x2 !3 !3 x1 (b1 ) x2 !3 !3 x1 (b2 ) Polynomial Chaos 8 8 10 6 6 generalized Polynomial 4 5 4 Chaos 2 0 2 0 !5 0 Post-processing !2 !10 !2 3 3 3 2 3 2 2 3 1 1 3 1 2 2 2 0 1 0 1 0 1 !1 0 !1 0 !1 0 !2 !1 !2 !1 !2 !1 !2 !2 !2 Resolution for a x2 !3 !3 x1 (c1 ) x2 !3 !3 x1 (c2 ) x2 !3 !3 x1 (c3 ) general SPDE 30 100 150 20 100 Stochastic Galerkin 10 50 50 Method (SGM) 0 0 0 !10 !50 !50 !20 !100 Stochastic Collocation !30 !100 !150 3 Method (SCM) 2 1 2 3 3 2 1 3 3 2 1 3 2 2 0 1 0 1 0 1 !1 0 !1 0 !1 0 !2 !1 !2 !1 !2 !1 !2 !2 !2 x2 !3 !3 x1 (d1 ) x2 !3 !3 x 1 (d2 ) x2 !3 !3 x 1 (d3 ) Multivariate quadratures 150 100 100 30 20 50 50 10 Full 0 0 0 !50 !10 !50 Sparse !100 !20 !150 !100 !30 3 3 3 2 2 2 3 1 3 1 3 1 2 2 2 0 1 0 1 0 1 !1 0 !1 0 !1 0 !2 !1 !2 !1 !2 !1 !2 !2 !2 Intrude or not x2 !3 !3 x 1 (d4 ) x2 !3 !3 x 1 (d5 ) x2 !3 !3 x1 (d6 ) intrude ? Hermite polynomials in a truncated domain. (a): constant; (b1−2 ): 1st -order; (c1−3 ): Open issues 2nd -order; (d1−6 ): 5th -order. Introduction to Stochastic Spectral Methods 25 Novembre 2008 46 / 133 Multi-dimensional Hermite polynomials construction (4) Introduction For a multi-dimensional Hermite polynomial of order at most P = 3 in RVs and RPs Several UQ methods N = 3 Gaussian independent RVs, we have: Spectral k » (N = 3; P = 3) = – 0 1 0 0 2 1 1 0 0 0 3 2 2 1 1 1 0 0 0 0 representation 0 0 1 0 0 1 0 2 1 0 0 1 0 2 1 0 3 2 1 0 e Karhunen-Lo`ve 0 0 0 1 0 0 1 0 1 2 0 0 1 0 1 2 0 1 2 3 Polynomial Chaos k p Polynomial decomposition generalized Polynomial Chaos 0 0 H0 (ξ1 ) H0 (ξ2 ) H0 (ξ3 ) Post-processing 1 1 H1 (ξ1 ) H0 (ξ2 ) H0 (ξ3 ) 2 H0 (ξ1 ) H1 (ξ2 ) H0 (ξ3 ) Resolution for a Multi-indices values and ordering 3 H0 (ξ1 ) H0 (ξ2 ) H1 (ξ3 ) general SPDE 4 2 H2 (ξ1 ) H0 (ξ2 ) H0 (ξ3 ) described in this section are totally 5 H1 (ξ1 ) H1 (ξ2 ) H0 (ξ3 ) Stochastic Galerkin Method (SGM) independent from the type of random 6 H1 (ξ1 ) H0 (ξ2 ) H1 (ξ3 ) Stochastic Collocation 7 H0 (ξ1 ) H2 (ξ2 ) H0 (ξ3 ) Method (SCM) distributions and polynomial basis that 8 H0 (ξ1 ) H1 (ξ2 ) H1 (ξ3 ) 9 H0 (ξ1 ) H0 (ξ2 ) H2 (ξ3 ) are considered. 10 3 H3 (ξ1 ) H0 (ξ2 ) H0 (ξ3 ) Multivariate quadratures 11 H2 (ξ1 ) H1 (ξ2 ) H0 (ξ3 ) Full They merely reﬂect the adopted ranking 12 H2 (ξ1 ) H0 (ξ2 ) H1 (ξ3 ) 13 H1 (ξ1 ) H2 (ξ2 ) H0 (ξ3 ) Sparse logic together with the tensor-product 14 H1 (ξ1 ) H1 (ξ2 ) H1 (ξ3 ) construction. 15 H1 (ξ1 ) H0 (ξ2 ) H2 (ξ3 ) Intrude or not 16 H0 (ξ1 ) H3 (ξ2 ) H0 (ξ3 ) intrude ? 17 H0 (ξ1 ) H2 (ξ2 ) H1 (ξ3 ) 18 H0 (ξ1 ) H1 (ξ2 ) H2 (ξ3 ) Open issues 19 H0 (ξ1 ) H0 (ξ2 ) H3 (ξ3 ) Introduction to Stochastic Spectral Methods 25 Novembre 2008 47 / 133 Examples of PC representation of non-gaussian RFs Introduction RVs and RPs Several UQ methods Convergence of any functional in L2 (Ω, A, P ) does not mean necessarily a Spectral fast convergence for non-Gaussian random ﬁelds. representation e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial Chaos Post-processing Resolution for a general SPDE Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Multivariate quadratures Full Sparse PDF from Hermite PC approximations of Beta(α, β) distributions. (left): α = β = 0; (right): Intrude or not α = 2, β = 0 (Courtesy of Donbin Xiu, Purdue university). intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 48 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 49 / 133 Deﬁnitions (1) Introduction RVs and RPs Second-order random ﬁeld u : Ω → V over the Hilbert space V : Several UQ methods 2 E u = E(u, u) < ∞. Spectral representation The RV u(ω) can be represented by gPC : Karhunen-Lo`ve e ∞ X Polynomial Chaos u(x , t, ω) = ˆ uk (x , t)Φk (X (ω)) generalized Polynomial Chaos k =0 Post-processing ˆ The knowledge of uk fully determines the RP. Resolution for a general SPDE {Φj (X (ω))} are orthogonal polynomials in terms of a zero-mean Stochastic Galerkin Method (SGM) random vector X := {Xj (ω)}N , satisfying the orthogonality relation j =1 Stochastic Collocation Method (SCM) Φi Φj = Φ2 δij , i Multivariate Numerically, we have to truncate the representation : quadratures Full M X Sparse u(x , t, ω) = ˆ uk (x , t)Φk (X (ω)) k =0 Intrude or not intrude ? where M depends of the number of random dimensions N and the Open issues highest polynomial order P of the polynomial basis. Introduction to Stochastic Spectral Methods 25 Novembre 2008 50 / 133 Deﬁnitions (2) Introduction RVs and RPs Several UQ methods The inner product is in the Hilbert space determined by the measure of Spectral the random variables : representation Z Z e Karhunen-Lo`ve Polynomial Chaos f (X )g(X ) = f (X )g(X )dP (ω) = f (X )g(X )w (X )d X ω∈Ω generalized Polynomial Chaos Post-processing with w (X ) denoting the density of the law dP (ω) with respect to the Lebesgue measure d X and with integration taken over a suitable Resolution for a general SPDE domain, determined by the range of the random vector X . Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) In the discrete case, the above orthogonal relation takes the form : X Multivariate quadratures f (X )g(X ) = f (X )g(X )w (X ). Full X Sparse Intrude or not Correspondence between the type of the orthogonal polynomials {Φ} intrude ? and the law of the random variables X . Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 51 / 133 The Askey scheme (1) Introduction RVs and RPs The Askey scheme of hypergeometric polynomials Several UQ methods 4 F3(4) Wilson Racah Spectral representation e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial Chaos F2(3) Continuous Continuous 3 dual Hahn Hahn Hahn Dual Hahn Post-processing Resolution for a general SPDE Stochastic Galerkin Method (SGM) Meixner 2 F1(2) - Jacobi Meixner Krawtchouk Stochastic Collocation Pollaczek Method (SCM) Multivariate quadratures Full 1 F1(1) Laguerre Charlier 2 F0(1) Sparse Intrude or not intrude ? 2 F0(0) Hermite Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 52 / 133 The Askey scheme (2) Introduction RVs and RPs Several UQ methods Spectral representation The correspondence between the polynomial of the Wiener-Askey type and Karhunen-Lo`ve e the associated probability distribution (N ≥ 0 is a ﬁnite number). Polynomial Chaos generalized Polynomial Chaos Post-processing Random variable ξ Wiener-Askey PC {φ(ξ)} Support Continuous Gaussian Hermite-chaos (−∞, ∞) Resolution for a distribution gamma Laguerre-chaos [0, ∞) general SPDE beta Jacobi-chaos [a, b] Stochastic Galerkin Method (SGM) uniform Legendre-chaos [a, b] Stochastic Collocation Discrete Poisson Charlier-chaos {0, 1, 2, . . . } Method (SCM) distribution binomial Krawtchouk-chaos {0, 1, . . . , N } negative binomial Meixner-chaos {0, 1, 2, . . . } Multivariate hypergeometric Hahn-chaos {0, 1, . . . , N } quadratures Full Sparse Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 53 / 133 gPC basis choice Introduction RVs and RPs Several UQ methods Spectral representation Optimal choice of the gPC basis for non-Gaussian RP with irregular Karhunen-Lo`ve e and/or unknown measures is an open question. Polynomial Chaos generalized Polynomial Chaos Post-processing For non-linear problems, optimal gPC representation (from Askey scheme) of the inputs optimality for the solution. Resolution for a general SPDE Stochastic Galerkin Method (SGM) Possibility of handing RPs whose underlying RVs have diﬀerents Stochastic Collocation distributions =⇒ concept of gPC mixed basis. Method (SCM) Multivariate quadratures Possibility of tailoring a basis =⇒ Concept of gPC reprsentation for Full arbitrary measures. Sparse Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 54 / 133 Mixed basis (1) Introduction RVs and RPs Several UQ methods Example: Spectral representation Let us consider a multi-dimensional Hermite-Legendre polynomial Φk of e Karhunen-Lo`ve Polynomial Chaos order at most P = 3 in N = 2 independent RVs, where ξ is Gaussian RV generalized Polynomial and ζ is a uniform RV. Chaos Post-processing Resolution for a We tensorize the 1D Hermite polynomial with the 1D Legendre polynomial general SPDE corresponding to the Gaussian and the uniform RVs, respectively. Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) We have, for instance for the 8-th polynomial of the basis: Multivariate quadratures Full Φ2 (ξ, ζ) 7 = 1 Hk1 (ξ) ⊗ L12 (ζ) k 1 3 1 1 H1 (ξ) × L1 (ζ) = (ξ) × (3ζ 2 − 1) = ξζ 2 − ξ Sparse = 2 2 2 2 Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 55 / 133 Mixed basis (2) Introduction RVs and RPs Another example : the simple ﬁrst-order linear ODE: Several UQ methods dΘ Spectral = −C Θ with Θ (t = 0) = Θ0 and t ∈ [0, T ] . representation dt e Karhunen-Lo`ve Polynomial Chaos The deterministic and stochastic analytic solutions are, respectively: ¯ ¯ generalized Polynomial Chaos Θ(t) = Θ0 e −C t and Θ(t, ω) = (Θ0 + σΘ0 ξ)e (C −σC ζ)t . ¯ ¯ Post-processing The exact mean solution in this case is: Resolution for a Z ∞ Z 1 2 general SPDE e (−ξ /2) 1 Stochastic Galerkin Θ(t) = (Θ0 + σΘ0 ξ) e −(C +σC ζ)t × √ × d ζd ξ Method (SGM) −∞ −1 2π 2 Stochastic Collocation ∗ ∗ Method (SCM) (e 2t − 1) e −t (C /σC +1) = Θ0 /2 . Multivariate t∗ quadratures Full The exact variance solution is: Θ2 − Θ 2 Sparse var [Θ] (t) = 2 ∗ ∗ Intrude or not ! 2 Θ0 + σΘ0 (e 4t − 1) e −2t (C /σC +1) , t ∗ = σC t. intrude ? = × 4 t∗ Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 56 / 133 Mixed basis (3) Introduction RVs and RPs ¯ ¯ Results with (C , σC ) = (1/6, 1/3) and (Θ0 , σΘ0 ) = (1/3, 1/4) : Several UQ methods 0.7 0.35 !0 0.6 0.3 ! Spectral 1 !2 representation 0.5 0.25 !3 0.4 0.2 !4 e Karhunen-Lo`ve 0.3 0.15 ! Modes 5 Polynomial Chaos ! 0.2 0.1 generalized Polynomial Chaos 0.1 0.05 Post-processing 0 0 !0.1 !0.05 !0.2 !0.1 Resolution for a 0 0.5 1 1.5 2 2.5 3 3.5 4 0 0.5 1 1.5 2 2.5 3 3.5 4 t t general SPDE Stochastic Galerkin Time evolution of random realizations Convergence rate of the variance vs. Method (SGM) polynomial order P Stochastic Collocation 10 0 Method (SCM) t=1.0 t=2.0 t=3.0 t=4.0 L !norm error of the variance Multivariate !5 10 quadratures Full Sparse !10 10 2 Intrude or not intrude ? !15 10 1 2 3 4 5 6 7 8 9 10 P Open issues Convergence rate of the variance vs. polynomial order P Introduction to Stochastic Spectral Methods 25 Novembre 2008 57 / 133 gPC for arbitrary distributions (1) Introduction RVs and RPs Several UQ methods The idea is to build a space of solutions with functions orthogonals with Spectral representation respect to an arbitrary measure. e Karhunen-Lo`ve Polynomial Chaos Orthogonal polynomials satisfy a three-term recurrence relation, generalized Polynomial Chaos Post-processing πi+1 (X ) = (X − αi )πi (X ) − βi πi−1 (X ), i = 0, 1, · · · , Np Resolution for a π0 (X ) = 1, π−1 (X ) = 0, general SPDE Stochastic Galerkin where {πi (X )} is a set of (monic) orthogonal polynomials, Method (SGM) πi (X ) = X i + O(X i−1 ) Stochastic Collocation Method (SCM) i = 0, 1, · · · Np Multivariate quadratures and the coeﬃcients αi and βi are uniquely determined by a positive Full measure, which corresponds to our probability measure. Sparse Intrude or not For continuous measure : the Stieltjes procedure and the modiﬁed intrude ? Chebyshev algorithm (less stable). Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 58 / 133 gPC for arbitrary distributions (2) Introduction RVs and RPs Several UQ methods The Stieltjes procedure uses the Darboux’s formulae to compute the Spectral coeﬃcients αi and βi : representation Karhunen-Lo`ve e (X πi , πi ) Polynomial Chaos αi = , i = 0, 1, · · · Np generalized Polynomial (πi , πi ) Chaos Post-processing and (πi , πi ) Resolution for a β0 = (π0 , π0 ), βi = , i = 1, 2, · · · , general SPDE (πi−1 , πi−1 ) Stochastic Galerkin Method (SGM) where (·, ·) denotes the inner product in terms of the measure ρ(X ). Stochastic Collocation Method (SCM) Multivariate The inner product can be evaluated by Gauss-type quadrature rule. quadratures Full Sparse Such a quadrature rule can be regarded as a discrete measure that yields the corresponding discrete versious of these procedures. Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 59 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 60 / 133 Post-processing (1) Introduction RVs and RPs Several UQ methods Once the modal coeﬃcients are computed, it is then possible to perform a number of analytical operations on the stochastic solution. Spectral representation Moments, sensitivity analysis, conﬁdence intervals and pdf of the Karhunen-Lo`ve e solution can be easily evaluated due to the orthogonality of the modes. Polynomial Chaos generalized Polynomial Chaos Covariance between two ﬁelds u(x , t, ω) and v (x , t, ω) : Post-processing Resolution for a Ruv (x1 , x2 , t) = < u(x1 , t, ω)− < u(x1 , t, ω) >, v (x2 , t, ω)− < v (x2 , t, ω) >> general SPDE M X Stochastic Galerkin Method (SGM) = uk (x1 , t)ˆk (x2 , t) < Φ2 > ˆ v k Stochastic Collocation k =0 Method (SCM) Multivariate Auto-correlation Ru : quadratures Full Ru (x1 , x2 , t) = < u(x1 , t, ω), u(x2 , t, ω) > Sparse M X Intrude or not = uk (x1 , t)ˆk (x2 , t) < Φ2 > ˆ u k intrude ? k =0 Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 61 / 133 Post-processing (2) Introduction RVs and RPs Several UQ methods Expected values: Spectral representation 1. Mean : Karhunen-Lo`ve e ˆ µu = E[u(x , t, X )] = u0 Polynomial Chaos generalized Polynomial Chaos 2. Variance : M X Post-processing 2 σu = E[u(x , t, X )2 ] = ˆ2 uk E[Φ2 ] k Resolution for a k =1 general SPDE Stochastic Galerkin Method (SGM) 3. Skewness : Stochastic Collocation M M M 1 1 XX X E[u(x , t, X )3 ] = 3 Method (SCM) δu = 3 ˆ ˆ ˆ ui uj uk E[Φi Φj Φk ] σu σu i=1 j =1 k =1 Multivariate quadratures Full 4. Kurtosis : Sparse M M M M 1 1 XX XX Intrude or not κu = 4 E[u(x , t, X )4 ] = 4 ˆ ˆ ˆ ˆ ui uj uk ul E[Φi Φj Φk Φl ] intrude ? σu σu i=1 j =1 k =1 l=1 Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 62 / 133 Post-processing (3) Introduction RVs and RPs Several UQ methods Sensitivity analysis : Spectral 2 Variance-based : Sobol’ sensitivity indices Si = var(E[u|Xi ])/σu representation Karhunen-Lo`ve e (analytically computed from gPC coeﬃcients) Polynomial Chaos generalized Polynomial Chaos Example : 5 X Post-processing u(X1 , X2 ) = ˆ uk φk (X1 , X2 ) k =0 Resolution for a 2 2−1 general SPDE = u0 + u1 X1 + u2 X2 + u3 (X1 − 1) + u4 X1 X2 + u5 (X2 ˆ ˆ ˆ ˆ ˆ ˆ ) Stochastic Galerkin Method (SGM) M 2 X 2 2 Stochastic Collocation =⇒ σu = ˆ uk E[Φk ] Method (SCM) k =1 2 2 2−1 2 2 Multivariate =⇒ = E[(ˆ1 X1 + u3 (X1 − 1)) ] + E[(ˆ2 X2 + u5 (X2 u ˆ u ˆ u )) ] + E[(ˆ4 X1 X2 ) quadratures Full =⇒ 1 = S1 + S2 + S12 Sparse Intrude or not intrude ? Hierarchical nature of gPC is an advantage here. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 63 / 133 Post-processing (4) Introduction RVs and RPs Several UQ methods Distributions and conditional densities : Spectral 1. Histogram representation Karhunen-Lo`ve e 2. Kernel-smoothing density estimate Polynomial Chaos X fX (Xn ) generalized Polynomial 3. fu (x , t, x ) = ˛ ∂u(x ,t,X ) ˛ with Xn roots of Chaos n ˛ ∂X |X =Xn ˛ Post-processing M X Resolution for a u(x , t, X ) = ˆ uk Φk = x . general SPDE i=0 Stochastic Galerkin Method (SGM) Stochastic Collocation Reliability analysis : Method (SCM) 1. Probability failure Pf of u: Multivariate Z quadratures Full Pf = fX (X )dX = E[1D (u)] with D = {G(X ) = R−u(x , t, X ) < 0} D Sparse Intrude or not 2. α-Quantile uα : intrude ? ` ´ P(u(x , t, X ) ≤ uα (x , t) = α i.e. uα = inf {u(x , t), F u(x , t) > α} Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 64 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 65 / 133 Computational approaches Introduction RVs and RPs Several UQ methods Discretize spatial (and temporal) domains (e.g. ﬁnite diﬀerences, ﬁnite Spectral elements, spectral methods) representation Karhunen-Lo`ve e Numerically represent with a ﬁnite number of RVs the random ﬁelds Polynomial Chaos which are input to the model (e.g. KL expansion, PC/gPC expansion) generalized Polynomial Chaos Post-processing Compute solution/statistics/functionals of the solution: Resolution for a via Perturbation method : stochasticity considered as small perturbation general SPDE around the mean value, then Taylor expand and truncate. Stochastic Galerkin Method (SGM) ’directly’ via sampling-based high-dimensional numerical integration (e.g. Stochastic Collocation Method (SCM) Monte-Carlo & variants, FORM, Probabilistic Collocation Method (PCM), Stochastic Collocation Method (SCM)). Needs realizations at Multivariate collocation points. quadratures Full build a surrogate model of the solution (response surface) and then Sparse integrate it numerically (cheap). Needs realizations at collocation points. PC/gPC representations are one possibility. Intrude or not intrude ? Stochastic Galerkin projection on PC/gPC space. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 66 / 133 Stochastic diﬀerential equation Introduction RVs and RPs Several UQ methods ` ´ We consider a complete probability space: Ω, A, P , where Ω is the event Spectral representation space, A ⊂ 2Ω the σ-algebra and P the probability measure. e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial Find u(x , t, ω) with t ∈ [0, T ], ω ∈ Ω, such that: Chaos Post-processing L(x , t, ω; u) = f (x , t, ω) with x ∈ D, Resolution for a general SPDE B(x , t, ω; u) = g(x , t, ω) with x ∈ ∂D. Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Random inputs ← L (linear or non-linear operator), B (boundary operator), f , g, D ⊂ Rd bounded domain, random parameter R, ... Multivariate quadratures Full We assume that the physical boundary δD and the forcing terms (f , g) Sparse are suﬃciently regular and smooth such that the stochastic problem aforementioned is well posed. Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 67 / 133 Strong & weak form of the stochastic ODE/PDE (1) Introduction RVs and RPs Several UQ methods Finite dimensional noise assumption: Spectral R(ω) = R(X1 (ω), X2 (ω), . . . , XN (ω)) representation Karhunen-Lo`ve e Each random variable is a function Xi : ω ∈ Ω → R Polynomial Chaos u(x , t, ω) ≈ u(x , t, X1 (ω), X2 (ω), . . . , XN (ω)) (Doob-Dynkin lemma) generalized Polynomial Chaos Post-processing X (ω) = (X1 (ω), X2 (ω), . . . , XN (ω)): set of i.i.d continuous random Resolution for a variables with PDF ρ and support Γ : general SPDE Stochastic Galerkin N Y N Y Xi (Ω) ⊂ RN . Method (SGM) Stochastic Collocation ρ(X ) = ρ1 (X1 )ρ2 (X2 ) · · · ρN (XN ) = ρi (Xi ), Γ ≡ Method (SCM) i=1 i=1 Multivariate quadratures Strong form: ﬁnd u(x , t, X ) from the (N + d )-dimensional diﬀerential Full Sparse system, such that: Intrude or not L(x , t, X ; u) = f (x , t, X ) with (x , X ) ∈ D × Γ intrude ? B(x , t, X ; u) = g(x , t, X ) with (x , X ) ∈ ∂D × Γ Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 68 / 133 Strong & weak form of the stochastic ODE/PDE (2) Introduction RVs and RPs Several UQ methods Spectral Finite dimensional subspace VΓ ⊂ L2 (Γ) of all square integrable ρ representation function in Γ with respect to the measure ρ(X )d X . e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial Chaos Post-processing Weak form: ﬁnd uV (x , t, X ) ∈ VΓ (X ), such that: Resolution for a general SPDE ∀ φ(X ) ∈ VΓ , x ∈ D : Stochastic Galerkin Z Z Method (SGM) Stochastic Collocation L(x , t, X ; uV )φ(X )ρ(X )d X = f (x , t, X )φ(X )ρ(X )d X , Method (SCM) Γ Γ Multivariate quadratures ∀ φ(X ) ∈ VΓ , x ∈ ∂D : Full Z Z Sparse B(x , t, X ; uV )φ(X )ρ(X )d X = g(x , t, X )φ(X )ρ(X )d X . Intrude or not Γ Γ intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 69 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 70 / 133 Stochastic Galerkin projection (1) Introduction RVs and RPs Several UQ methods The Galerkin projection is a standard procedure in spectral methods : Spectral 1. The diﬀerents terms are spectrally expanded to a certain order. representation Karhunen-Lo`ve e 2. The obtained equation are projected onto the spectral basis. Polynomial Chaos generalized Polynomial Chaos gPC decomposition : Post-processing M X (N + P )! Resolution for a general SPDE u(x , t, ω) = ˆ uk (x , t)Φk (X ) with M = −1 k =0 N ! P! Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) We assume the deterministic coeﬃcient can be themselves treated with Multivariate an appropriate representation. quadratures Full Sparse For instance, if the evolution problem is normally space-discretized : X Intrude or not intrude ? ˆ uk (x , t) = ˜l uk (t)Nl (x ) l Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 71 / 133 Stochastic Galerkin projection (2) Introduction RVs and RPs Several UQ methods Substitute in the weak form of the model problem. For i = 0, 2, . . . M : Z M Spectral ` ´ X ´ representation L x , t, X ; ˆ uk (x , t)Φk (X ) Φi (X )ρ(X )d X e Karhunen-Lo`ve Γ k =0 Polynomial Chaos Z generalized Polynomial = f (x , t, X )Φi (X )ρ(X )d X . Chaos Γ Post-processing Resolution for a Orthogonality condition ⇒ system of (M + 1) deterministic equations general SPDE ˆ for ui (x , t). Stochastic Galerkin Method (SGM) System is coupled unless the problem is linear (in random space). Stochastic Collocation ˆ Implicitly accounts for uk coupling Method (SCM) Standard numerical method can be used to solve. Multivariate quadratures Requires the resolution of a large system (integrals of residuals). Full Problems for non-linearities. Sparse Intrude or not Variations of the stochastic Galerkin method when poor convergence intrude ? (discontinuity, stochastic bifurcation, long-time integration). Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 72 / 133 Example 1 : 1st order stochastic ODE with random Introduction variables decay rate and initial condition (1) RVs and RPs Several UQ methods dΘ = −C Θ; Θ (t = 0) = Θ0 , with C > 0 and t ∈ [0, T ] . Spectral representation dt e Karhunen-Lo`ve Polynomial Chaos The decay rate coeﬃcient C and the initial condition Θ0 are considered generalized Polynomial Chaos to be random inputs to the system (uniform RV ζ = (ζ1 , ζ2 )). Post-processing Resolution for a The stochastic counterpart of the equation becomes non-linear due to general SPDE the product between the random quantity C and the solution Θ. Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) ¯ ¯ C (ζ1 ) = C + σC ζ1 and Θ0 (ζ2 ) = Θ0 + σΘ0 ζ2 where (σC , σΘ0 ) are the standard deviations of each random variables respectively. Multivariate quadratures Full Sparse The deterministic and stochastic analytic solutions are, respectively: ¯ ¯ Intrude or not Θ(t) = Θ0 e −C t ¯ and ¯ Θ(t, ω) = (Θ0 + σΘ0 ζ2 ) e (C +σC ζ1 )t . intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 73 / 133 Example 1 : 1st order stochastic ODE with random Introduction variables decay rate and initial condition (2) RVs and RPs Several UQ methods Spectral The exact mean solution in this case is : representation e Karhunen-Lo`ve Z 1 Z 1 1 1 Polynomial Chaos Θ(t) = (Θ0 + σΘ0 ζ2 )e −(C +σC ζ1 )t × × d ζ1 d ζ2 generalized Polynomial Chaos −1 −1 2 2 ∗ Post-processing e −C t /σC sinh (t ∗ ) = Θ0 . Resolution for a t∗ general SPDE Stochastic Galerkin Method (SGM) The exact variance solution is : Stochastic Collocation Method (SCM) var [Θ] (t) = Θ2 − Θ 2 Multivariate 2 ∗ quadratures ! 2 3Θ0 + σΘ0 e −2C t /σC sinh (2t ∗ ) Full = × , Sparse 6 t∗ where t ∗ = σC t. Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 74 / 133 Example 1 : 1st order stochastic ODE with random Introduction variables decay rate and initial condition (3) RVs and RPs Several UQ methods Spectral representation The weak form of the problem consists in: e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial Find ΘV (t, ζ) ∈ VΓ (ζ) ⊂ L2 (Γ), such that: ρ Chaos Post-processing Resolution for a ∀ φ(ζ) ∈ VΓ and t ∈ ]0, T ] : general SPDE Stochastic Galerkin Z Z Method (SGM) ˙ ΘV φ(ζ)ρ(ζ)d ζ = − C (ζ)ΘV φ(ζ)ρ(ζ)d ζ, Stochastic Collocation Method (SCM) Γ Γ Multivariate quadratures ∀ φ(ζ) ∈ VΓ : Full Sparse Z Z ΘV (0, ζ)φ(ζ)ρ(ζ)d ζ = Θ0 (ζ)φ(ζ)ρ(ζ)d ζ. Intrude or not Γ Γ intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 75 / 133 Example 1 : 1st order stochastic ODE with random Introduction variables decay rate and initial condition (4) RVs and RPs Several UQ methods The Legendre polynomial basis is well ﬁtted to uniform distributions. Spectral representation We make the choice to use Legendre polynomials to represent both our e Karhunen-Lo`ve Polynomial Chaos random inputs and stochastic solution. generalized Polynomial Chaos M X Post-processing C (ω) = ˆ ˆ ˆ ¯ Ci Li (ζ1 , ζ2 ) = C0 L0 (ζ1 , ζ2 ) + C1 L1 (ζ1 , ζ2 ) = C + σC ζ1 i=0 Resolution for a general SPDE ˆ ¯ ˆ ˆ =⇒ C0 = C , C1 = σC and Ci = 0 for i > 1. Stochastic Galerkin Method (SGM) Stochastic Collocation M Method (SCM) X Θ0 (ω) = ˆ Θ0l Ll (ζ1 , ζ2 ) Multivariate l=0 quadratures Full = ˆ ˆ ˆ ¯ Θ00 L0 (ζ1 , ζ2 ) + Θ01 L1 (ζ1 , ζ2 ) + Θ02 L2 (ζ1 , ζ2 ) = Θ0 + σΘ0 ζ2 Sparse ˆ ¯ ˆ ˆ ˆ =⇒ Θ00 = Θ0 , Θ01 = 0, Θ02 = σΘ0 and Θ0i = 0 for i > 2. Intrude or not intrude ? M X For the solution, we write : Θ(t, ω) = ˆ Θj (t)Lj (ζ1 , ζ2 ) Open issues j =0 Introduction to Stochastic Spectral Methods 25 Novembre 2008 76 / 133 Example 1 : 1st order stochastic ODE with random Introduction variables decay rate and initial condition (5) RVs and RPs Several UQ methods We inject all expansions into the following weak form : Spectral Z Z Z Z representation ˙ ΘV L(ζ1 , ζ2 )ρ(ζ1 , ζ2 )dζ1 dζ2 = − C (ζ1 , ζ2 )ΘV L(ζ1 , ζ2 )ρ(ζ1 , ζ2 )dζ1 dζ2 , e Karhunen-Lo`ve ζ1 ζ2 ζ1 ζ2 Polynomial Chaos Z Z Z Z generalized Polynomial Chaos ΘV (0, ζ1 , ζ2 )L(ζ1 , ζ2 )ρ(ζ1 , ζ2 )dζ1 dζ2 = Θ0 (ζ1 , ζ2 )L(ζ1 , ζ2 )ρ(ζ1 , ζ2 )dζ1 dζ2 , ζ1 ζ2 ζ1 ζ2 Post-processing for all L(ζ1 , ζ2 ) ∈ VΓ and t ∈ [0, T ]. Resolution for a general SPDE Stochastic Galerkin Method (SGM) After dropping the (ζ1 , ζ2 )-dependency, for the sake of clarity, we get a Stochastic Collocation system of equations : Method (SCM) Z Z X M Z Z X M M ˆ˙ ˆ X ˆ Multivariate Θj (t)Lj Lk ρ d ζ1 d ζ2 = − Ci Li Θj (t)Lj Lk ρ d ζ1 d ζ2 , quadratures ζ1 ζ2 j =0 ζ1 ζ2 i=0 j =0 Full Sparse Z Z M X Z Z M X ˆ Θj (0)Lj Lk ρ d ζ1 d ζ2 = ˆ Θ0l Ll Lk ρ d ζ1 d ζ2 , Intrude or not ζ1 ζ2 j =0 ζ1 ζ2 l=0 intrude ? for k = 0 . . . M and t ∈ [0, T ]. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 77 / 133 Example 1 : 1st order stochastic ODE with random Introduction variables decay rate and initial condition (6) RVs and RPs Several UQ methods Spectral System of (M + 1) coupled ODEs (due to the orthogonality of the basis) : representation Karhunen-Lo`ve e For k = 0 . . . M , Polynomial Chaos M M ˙ ˆ XX ˆ ˆ generalized Polynomial Chaos Θk (t) L2 = − k Ci Θj (t) Li Lj Lk for t ∈ ]0, T ] Post-processing i=0 j =0 Resolution for a ˆ ˆ Θk (0) = Θ0k general SPDE Stochastic Galerkin Method (SGM) Stochastic Collocation The inner product quantities L2 and Li Lj Lk involve k Method (SCM) multi-dimensional integrals of know polynomials over know supports Multivariate and can be numerically computed and tabulated for each type of quadratures distribution prior to the numerical simulations. Full Sparse ˆ The random modes Θk are smooth and thus any standard ordinary Intrude or not diﬀerential equation solver can be employed here. In the following, the intrude ? standard fourth-order Runge-Kutta scheme is used. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 78 / 133 Example 1 : 1st order stochastic ODE with random Introduction variables decay rate and initial condition (7) RVs and RPs Several UQ methods ¯ ¯ Results with (C , σC ) = (1/6, 1/3) and (Θ0 , σΘ0 ) = (1/3, 1/3). Spectral !0 0.5 0.4 representation !1 0.45 ! 0.3 2 e Karhunen-Lo`ve 0.4 ! 3 0.2 !4 Polynomial Chaos 0.35 !5 0.3 generalized Polynomial 0.1 Modes 0.25 ! Chaos 0 0.2 Post-processing 0.15 !0.1 0.1 !0.2 0.05 Resolution for a 0 general SPDE 0 0.5 1 1.5 2 2.5 3 3.5 4 !0.3 0 0.5 1 1.5 2 2.5 3 3.5 4 t t Stochastic Galerkin Method (SGM) Time evolution of random realizations Time evolution of gPC modes of the solution Stochastic Collocation 0.5 10 0 t=1.0 Method (SCM) 0.45 t=2.0 t=3.0 0.4 t=4.0 L2!norm Error of the variance 0.35 !5 Multivariate 10 0.3 quadratures 0.25 ! Full 0.2 !10 10 Sparse 0.15 0.1 0.05 MEAN Intrude or not 0 MEAN +/! STD 10 !15 0 5 10 15 0 0.5 1 1.5 2 2.5 3 3.5 4 intrude ? t P Time evolution of mean and mean ± std Convergence rate of the variance vs. Open issues solutions polynomial order P Introduction to Stochastic Spectral Methods 25 Novembre 2008 79 / 133 Example 2 : 1st order stochastic ODE with random Introduction process decay rate (1) RVs and RPs Several UQ methods Spectral dΘ representation = −C Θ; Θ (t = 0) = Θ0 , and t ∈ [0, T ] , Karhunen-Lo`ve e dt Polynomial Chaos generalized Polynomial Chaos The decay rate is a stationary second-order Gaussian RP with Post-processing ¯ exponential kernel and mean C and std σC . Resolution for a general SPDE Stochastic Galerkin N -terms KL representation of the decay rate : Method (SGM) Stochastic Collocation N X√ Method (SCM) C N (t, ω) = C + σC ¯ λi φi (x )Xi (ω), Multivariate i=1 quadratures Full We choose for the following results: C = 1.0 and σC = 1/3. Sparse Intrude or not intrude ? Diﬀerent Cl are investigated and convergence rates are evaluated. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 80 / 133 Example 2 : 1st order stochastic ODE with random Introduction process decay rate (2) RVs and RPs Several UQ methods 2 3 1.5 2 C(t) C(t) Spectral 1 representation 1 0.5 e Karhunen-Lo`ve 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 t t Polynomial Chaos 1 1 generalized Polynomial Chaos !(t) !(t) 0.5 0.5 Post-processing 0 0 Resolution for a 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 t t general SPDE Stochastic Galerkin Cl = 1 Cl = 0.5 Method (SGM) 3 Stochastic Collocation 2 Method (SCM) C(t) 1 0 Multivariate quadratures !1 0 0.2 0.4 0.6 0.8 1 t Full 1 Sparse !(t) 0.5 Intrude or not intrude ? 0 0 0.2 0.4 0.6 0.8 1 t Open issues Cl = 1/3 Introduction to Stochastic Spectral Methods 25 Novembre 2008 81 / 133 Example 2 : 1st order stochastic ODE with random Introduction process decay rate (3) RVs and RPs Several UQ methods T=1, Cl=0.5, N=5, P=5, E[C]=1, Var[C]=0.33333 1 1.2 Mean m=0 Mean ± Std m=1 0.9 1 Spectral m=2 m=3 0.8 representation 0.8 m=4 m=5 0.7 e Karhunen-Lo`ve 0.6 !(t) !m(t) 0.6 Polynomial Chaos 0.4 generalized Polynomial 0.5 Chaos 0.2 0.4 Post-processing 0 0.3 !0.2 0.2 Resolution for a 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 t t general SPDE Time evolution of a few gPC modes Time evolution of the mean and mean ±std Stochastic Galerkin Method (SGM) values for Cl = 0.5 0 Stochastic Collocation 10 Cl = 1 Method (SCM) Cl = 1/2 Cl = 1/3 Relative L2!norm error of kernel Cl = 1/4 Multivariate !1 10 quadratures Full Sparse !2 10 Intrude or not intrude ? !3 10 0 2 4 6 8 10 N Open issues Convergence of the KL exponential kernel Introduction to Stochastic Spectral Methods 25 Novembre 2008 82 / 133 Example 2 : 1st order stochastic ODE with random Introduction process decay rate (4) RVs and RPs Several UQ methods !1 N=1 10 !1 N=1 10 N=2 N=2 N=3 N=3 Spectral N=4 N=4 L2!norm Error of Variance L2!norm Error of Variance !2 N=5 N=5 representation 10 10 !2 e Karhunen-Lo`ve !3 10 !3 Polynomial Chaos 10 generalized Polynomial !4 Chaos 10 !4 10 Post-processing !5 10 !5 10 Resolution for a 1 1.5 2 2.5 3 3.5 4 4.5 5 1 1.5 2 2.5 3 3.5 4 4.5 5 P P general SPDE Stochastic Galerkin Cl = 1 Cl = 0.5 Method (SGM) !1 N=1 10 Stochastic Collocation N=2 Method (SCM) N=3 N=4 L2!norm Error of Variance N=5 !2 10 Multivariate quadratures !3 Full 10 Sparse !4 10 Intrude or not intrude ? 1 1.5 2 2.5 3 3.5 4 4.5 5 P Open issues Cl = 1/3 Introduction to Stochastic Spectral Methods 25 Novembre 2008 83 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 84 / 133 Probabilistic Collocation Method (PCM) [Tatang 2004, Babuska Introduction 2004, Xiu 2005] RVs and RPs Several UQ methods The goal is the prediction of statistical moments of the solution (mean Spectral representation value, variance, covariance, etc.) or statistics of some given response. e Karhunen-Lo`ve Polynomial Chaos The solution may have a very regular dependence on the input RVs generalized Polynomial Chaos (SGM or SCM based on orthogonal tensor product polynomials feature Post-processing a very fast convergence rate) Resolution for a When the number of RVs is small, SCM is a very eﬀective tool. general SPDE Stochastic Galerkin Method (SGM) But, approximations based on full tensor product grids suﬀer from the Stochastic Collocation curse of dimensionality. Method (SCM) Multivariate Remedy : Use sparse grids in order to reduce the number of collocation quadratures points while maintaining a high level of accuracy. Full Sparse Smolyak-type sparse grid : for some cases, algebraic convergence wrt Intrude or not total number of collocation points. The exponent is connected to both intrude ? the regularity of the solution and the number of input RVs, N , and Open issues essentially deteriorates with N by a 1/ log N factor. Introduction to Stochastic Spectral Methods 25 Novembre 2008 85 / 133 PCM - Framework with Lagrange interpolation (1) Introduction Complete probability space: Ω, A, P , where Ω is the event space, A ⊂ 2Ω ` ´ RVs and RPs Several UQ methods the σ-algebra and P : A → [0, 1] the probability measure. Spectral ¯ Find u : Ω × D → R with u(x , t, ω) and t ∈ [0, T ], ω ∈ Ω, D ⊂ Rd representation Karhunen-Lo`ve e such that P -almost everywhere in Ω: Polynomial Chaos generalized Polynomial Chaos L(x , t, ω; u) = f (x , t, ω) with x ∈ D, Post-processing B(x , t, ω; u) = g(x , t, ω) with x ∈ ∂D. Resolution for a general SPDE Random inputs ← L (linear or non-linear operator), B (boundary Stochastic Galerkin Method (SGM) operator), f , g, D ⊂ Rd bounded domain, random parameter R, ... Stochastic Collocation Method (SCM) We assume that the physical boundary δD and the forcing terms (f , g) Multivariate are suﬃciently regular and smooth such that the stochastic problem quadratures Full aforementioned is well posed. Sparse Finite dimensional noise assumption: Intrude or not R(ω) = R(X1 (ω), X2 (ω), . . . , XN (ω)) : Ω → RN . intrude ? Each random variable is a function Xi : ω ∈ Ω → R Open issues u(x , t, ω) ≈ uN (x , t, X1 (ω), X2 (ω), . . . , XN (ω)) (Doob-Dynkin lemma) Introduction to Stochastic Spectral Methods 25 Novembre 2008 86 / 133 PCM - Framework with Lagrange interpolation (2) Introduction RVs and RPs Several UQ methods X (ω) = (X1 (ω), X2 (ω), . . . , XN (ω)): set of i.i.d continuous random variables with PDF: Spectral ρ : Γ → R+ with ρ(X ) = ρ1 (X1 )ρ2 (X2 ) · · · ρN (XN ) = N ρi (Xi ) Q representation QN QN i=1 e Karhunen-Lo`ve N Polynomial Chaos with support: Γ ≡ i=1 Γi = i=1 Xi (Ω) ⊂ R . generalized Polynomial Chaos Post-processing Find uN = uN (x , t, X ) from the (N + d )-dimensional diﬀerential system, such that: Resolution for a general SPDE L(x , t, X ; uN ) = fN (x , t, X ) with (x , X ) ∈ D × Γ Stochastic Galerkin Method (SGM) B(x , t, X ; uN ) = gN (x , t, X ) with (x , X ) ∈ ∂D × Γ Stochastic Collocation Method (SCM) We introduce the following space: Pp (Γ) ⊂ L2 (Γ) is the span of tensor ρ Multivariate quadratures product polynomials with degree at most p = (p1 , . . . , pN ), i.e. Pp (Γ) = N=1 Ppk (Γk ), with: N Full k Sparse n Intrude or not Ppk (Γk ) = span(Xk , n = 0, . . . , pk ), k = 1, . . . , N . intrude ? Hence the dimension of Pp (Γ) is Np = N=1 (pk + 1). Q k Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 87 / 133 PCM - Framework with Lagrange interpolation (3) Introduction RVs and RPs Collocation methods are based on polynomial interpolations of the solution. Several UQ methods A nodal set of (Nc + 1) collocation points is deﬁned in the Spectral N -dimensional random space. representation Karhunen-Lo`ve e After collocation projections, the resulting set of deterministic equations Polynomial Chaos is always uncoupled and each solution corresponding to each collocation generalized Polynomial Chaos point of the set is obtained with a deterministic solver. Post-processing The continuous solution can then be approximated by interpolation on Resolution for a general SPDE the data points using for instance multi-dimensional tensor product Stochastic Galerkin Lagrange basis. Method (SGM) Stochastic Collocation Method (SCM) The polynomial interpolation uN of the solution is: Nc Multivariate Y X quadratures uN (x , t, X ) = (u(x , t, X )) = u(x , t, z k )Lp (X ) k and z k ∈ Γ, Full k =0 Sparse Intrude or not where the z k are the collocation points and the Lp are the k intrude ? multi-dimensional Lagrange polynomials in the interpolation space satisfy Lp (z j ) = δij for 0 ≤ i, j ≤ Nc . i Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 88 / 133 PCM - Framework with Lagrange interpolation (4) Introduction RVs and RPs Several UQ methods Evaluation of the solution moments requires integrating those Lagrange basis: Spectral representation Nc Z X u(x , t, z k ) Lp (X )ρ(X )d X , e Karhunen-Lo`ve Polynomial Chaos E uN (x , t, X ) = k k =0 Γ generalized Polynomial Chaos Post-processing This requires the explicit knowledge of their expressions, which can be Resolution for a quite cumbersome (especially for N > 1), unless we choose the nodal general SPDE set of collocation points to be a cubature set points. Stochastic Galerkin Method (SGM) By choosing the cubature weight function to coincide with the joint Stochastic Collocation Method (SCM) density of the random input, the computation of the moments becomes straightforward. In this case, we have: Multivariate quadratures Nc Full X Sparse E uN (x , t, X ) = u(x , t, X k )wk , k =0 Intrude or not intrude ? where the {wk } are the cubature weights. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 89 / 133 PCM - Framework with Lagrange interpolation (5) Introduction RVs and RPs Several UQ methods The drawbacks of the Lagrange interpolation are many. Spectral representation The interpolation error is hard to control. e Karhunen-Lo`ve Polynomial Chaos Indeed the interpolation error is uniformly bounded by a quantity that generalized Polynomial Chaos depends on the choice of the nodal set (through the Lebesgue constant) Post-processing and is very diﬃcult to estimate (especially for N > 1). Resolution for a general SPDE The problem of the interpolation of a Runge function on equally spaced Stochastic Galerkin Method (SGM) grid nodes is very well known. Stochastic Collocation Method (SCM) In our case we have seen that the interpolation can not be constructed for any given nodes. Multivariate quadratures Full If no particular care is taken to construct the space of approximations, Sparse the computational cost increase exponentially with N due to the way Intrude or not the collocation grid is constructed based on full tensor products. intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 90 / 133 PCM - Error analysis Introduction RVs and RPs Several UQ methods If error associated with the representation of the random inputs (equation parameters and/or forcings) are under control. Spectral representation Karhunen-Lo`ve e The solution depends analytically on each RV (e.g. elliptic PDEs). Polynomial Chaos generalized Polynomial Chaos Smolyak-type sparse grid stochastic collocation method, can provide Post-processing algebraic convergence wrt the total # of collocation points Nc . Resolution for a general SPDE The Smolyak representation based on Clenshaw-Curtis 1D rule satisﬁes Stochastic Galerkin Method (SGM) [Nobile 2007] : Stochastic Collocation Method (SCM) C1 (σ) 1 σ ≤ e σ max{1, C1 (σ)}N µ with µ= Multivariate |1 − C1 (σ)| Nc 1 + log(2N ) quadratures Full Sparse The Smolyak representation based on Gauss-type 1D rules satisﬁes [Nobile 2007] : Intrude or not intrude ? C1 (σ) 1 a2 σ ≤ e a1 σ max{1, C1 (σ)}N ν with ν= Open issues |1 − C1 (σ)| Nc a3 + log(N ) Introduction to Stochastic Spectral Methods 25 Novembre 2008 91 / 133 Pseudo-spectral gPC-based method Introduction RVs and RPs gPC-based collocation method which is a pseudo-spectral method that Several UQ methods uses the gPC polynomial basis. {φj (ξ(ω))} are mutually orthogonal polynomials in terms of the Spectral representation zero-mean random vector ξ, satisfying the orthogonality relation : Karhunen-Lo`ve e φi , φj = φ2 δij , i Polynomial Chaos XM generalized Polynomial Chaos The ﬁnite-term expansion takes the form :u(x , ξ) = ˆ uj (x )φj (ξ). Post-processing j =0 Non-intrusive method : we project the stochastic solution directly onto Resolution for a general SPDE each member of the orthogonal basis chosen to span the random space Stochastic Galerkin Method (SGM) and has the advantage not to require modiﬁcations to the existing Stochastic Collocation deterministic solver. ` Method (SCM) u(x, ξ), φj (ξ) ˆ uj (x ) = for j = 0, ..., M Multivariate φ2 (ξ) j quadratures The evaluation is equivalent to computing multidimensional integrals Full Sparse over the domain Γ. The global error of the ﬁnal representation can be seen as a Intrude or not intrude ? superposition of an aliasing error, a ﬁnite-term projection error and a numerical error due to the intrinsic numerical approximation of the Open issues deterministic solver. Introduction to Stochastic Spectral Methods 25 Novembre 2008 92 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 93 / 133 Multivariate quadratures Introduction RVs and RPs Several UQ methods In general, we need to evaluate N -dimensional integrals of the form: Z Spectral representation Q N [f ] := E (f (X )) = f (ω)dPX (ω) Karhunen-Lo`ve e ΩN Z Z Polynomial Chaos generalized Polynomial = ... f (ω1 , . . . , ωN ) dPX1 (ω1 ) . . . dPXN (ωN ) Chaos Ω1 ΩN Post-processing Resolution for a general SPDE N Approximation QNe [f ] of Q N [f ] by evaluating the integrand at some Ne Stochastic Galerkin sets of quadrature points {Z 1 ∈ ΩN , . . . , Z Ne ∈ ΩN } Method (SGM) Stochastic Collocation Method (SCM) Combine the results with some appropriate weights : Multivariate Ne quadratures X N Full QNe [f ] = w i f (Z i ) Sparse i=1 Intrude or not Diﬀerent ways of dealing with high-dimensional integrations can be intrude ? considered depending on the prevalence of accuracy versus eﬃciency. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 94 / 133 Monte-Carlo method Introduction RVs and RPs Several UQ methods The Monte Carlo (MC) method uses Ne independent realizations of RVs X : Ne Spectral N 1 X representation QNe [f ] = f (Z i ), Karhunen-Lo`ve e Ne i=1 Polynomial Chaos generalized Polynomial Chaos This estimate converges almost surely to Q N [f ] (law of large numbers). Post-processing N −1/2 For large Ne : MC := |E (f ) − QNe (f )| ≈ σNe N (0, 1), where Resolution for a general SPDE N (0, 1) is the standard Gaussian RV and where σ is the std of f . Stochastic Galerkin Method (SGM) Stochastic Collocation MC methods may be speed up by various techniques for variance Method (SCM) reduction (Antithetic Variables; Control Variates; Matching Moment Multivariate Methods; Stratiﬁcation; Importance Sampling). quadratures Full MC simulations require reliable pseudo-random number generators : Sparse 1. Inadequate random number generators produce biased results (e.g. due to artiﬁcial correlations between the generated numbers). Intrude or not intrude ? 2. A random number generator must produce independent tuples and have a large cycle length. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 95 / 133 Quasi Monte-Carlo method Introduction RVs and RPs Several UQ methods The Quasi Monte-Carlo (QMC) method evaluates the integrand at correlated quadrature points (not randomly chosen) that are generated from Spectral representation “low discrepancy series” DNe . e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial A sequence {Z 1 , . . . , Z Ne } is called quasi random if its discrepancy Chaos Post-processing obeys: −1 DNe ≤ c(log Ne )n Ne , Resolution for a general SPDE where c, n are constants. Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Typical QCM error is of the order of: −1 Multivariate QMC ≈ O(Ne · (log Ne )N ). quadratures Full Sparse The O(log Ne )N dominates for large number of dimensions N but −1 Intrude or not otherwise we get a O(Ne ) convergence rate. intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 96 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 97 / 133 Standard Gauss quadrature (1) Introduction RVs and RPs The integral of a function φj : Ωj → R can be approximated by: Several UQ methods Nq (j ) X (j ) (j ) Spectral QNq [φ] = wi φ(Zi ), representation i=1 e Karhunen-Lo`ve Polynomial Chaos w (j ) are the weights and the Z (j ) are the Nq quadrature points. generalized Polynomial Chaos Multi-dimensional quadrature formula for f : ΩN → R is constructed as Post-processing tensor product of one-dimensional quadrature formulas. N (1) (N ) Resolution for a general SPDE QNq := QNq ⊗ . . . ⊗ QNq . Stochastic Galerkin Method (SGM) The grid points are constructed based on tensor-products of the Stochastic Collocation one-dimensional grid points of quadrature formulas (Gaussian, Method (SCM) Clenshaw-Curtis, or Fejer quadrature formulas) Multivariate quadratures Z N := Z (1) ⊗ . . . ⊗ Z (N ) . Full The multi-dimensional integral becomes: Sparse Nq Nq N X X (1) (N ) (1) (N ) Intrude or not QNq [f ] := ... wi1 . . . wiN f (Zi1 , . . . , ZiN ). intrude ? i1 =1 iN =1 Open issues Exact for N -multivariate polynomial of order at most P . Introduction to Stochastic Spectral Methods 25 Novembre 2008 98 / 133 Standard Gauss quadrature (2) Introduction RVs and RPs Several UQ methods When a 1D Gauss-type quadrature rule with Nq points is used as the Spectral representation foundation of the multi-dimensional grid, the integration is exact for Karhunen-Lo`ve e multivariate polynomials of order at most P ≤ 2Nq − c. Polynomial Chaos generalized Polynomial Chaos Post-processing c is determined by the type of Gauss-quadrature used which can be either classical Gauss (c = 1), Gauss-Radau (c = 2) or Gauss-Lobatto Resolution for a general SPDE (c = 3). Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Wonder how many points are needed to reach a polynomial accuracy of at most P . The integral requires a total number of evaluations of the N Multivariate quadratures integrand of Nq , where Full ‰ ı 3P + c Sparse Nq = . 2 Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 99 / 133 Standard Gauss quadrature : curse of dimensionality Introduction RVs and RPs Minimum number of Gauss quadrature points required to integrate exactly a Several UQ methods N -dimensional polynomial fonction of degree p ≤ P . Spectral representation e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial 10 Chaos Post-processing 8 Resolution for a general SPDE 6 Nq Stochastic Galerkin 10 Method (SGM) log 4 Stochastic Collocation Method (SCM) 2 Multivariate quadratures 0 Full 8 7 Sparse 10 6 8 5 Intrude or not 4 6 P intrude ? 3 4 N 2 2 Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 100 / 133 Standard Gauss quadrature - Importance of grid Introduction nestedness RVs and RPs Several UQ methods Some multi-dimensional grids inherit the poor properties from their Spectral representation one-dimensional grid counterparts. e Karhunen-Lo`ve Polynomial Chaos In the case of Gauss-type quadrature, the high accuracy of the method generalized Polynomial Chaos is balanced by the fact that the successive grids are not embedded. Post-processing Next ﬁgures show two-dimensional Gauss-Legendre collocation grids for Resolution for a general SPDE three diﬀerent levels of reﬁnement. Stochastic Galerkin Method (SGM) Stochastic Collocation The quadrature points are not redundant. Method (SCM) Multivariate Reﬁnement the approximation by switching from a coarse to a more quadratures Full reﬁned grid means evaluating the functional on the new grid without Sparse re-using any of the previous evaluations. Intrude or not intrude ? The method is therefore computationally costly (particularly when the multi-dimensional grids are based on non-nested one-dimensional grids). Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 101 / 133 Gauss quadrature rule are not nested (1) Introduction RVs and RPs 2D Legendre collocation grids for diﬀerent levels of resolution. Several UQ methods 1 1 0.8 0.8 Spectral 0.6 0.6 representation 0.4 0.4 e Karhunen-Lo`ve 0.2 0.2 Polynomial Chaos 0 0 generalized Polynomial !0.2 !0.2 Chaos !0.4 !0.4 Post-processing !0.6 !0.6 !0.8 !0.8 Resolution for a general SPDE !1 !1 !0.8 !0.6 !0.4 !0.2 0 0.2 0.4 0.6 0.8 1 !1 !1 !0.8 !0.6 !0.4 !0.2 0 0.2 0.4 0.6 0.8 1 Stochastic Galerkin 52 grid points 62 grid points Method (SGM) 1 1 Stochastic Collocation 0.8 0.8 Method (SCM) 0.6 0.6 0.4 0.4 Multivariate quadratures 0.2 0.2 0 0 Full !0.2 !0.2 Sparse !0.4 !0.4 !0.6 !0.6 Intrude or not intrude ? !0.8 !0.8 !1 !1 !1 !0.8 !0.6 !0.4 !0.2 0 0.2 0.4 0.6 0.8 1 !1 !0.8 !0.6 !0.4 !0.2 0 0.2 0.4 0.6 0.8 1 2 Open issues 10 grid points All grids Introduction to Stochastic Spectral Methods 25 Novembre 2008 102 / 133 Gauss quadrature rule are not nested (2) Introduction RVs and RPs 2D Hermite collocation grids for diﬀerent levels of resolution Several UQ methods L=5 L=6 5 5 Spectral representation 2.5 2.5 e Karhunen-Lo`ve Polynomial Chaos x2 x2 0 0 generalized Polynomial Chaos Post-processing !2.5 !2.5 Resolution for a !5 !5 !5 !2.5 0 2.5 5 !5 !2.5 0 2.5 5 general SPDE x1 x1 2 Stochastic Galerkin Method (SGM) 5 grid points 62 grid points L=10 5 Stochastic Collocation 5 Method (SCM) 2.5 2.5 Multivariate quadratures x2 0 x2 Full 0 Sparse !2.5 !2.5 Intrude or not intrude ? !5 !5 !5 !2.5 0 2.5 5 !5 !2.5 0 2.5 5 x1 x1 2 Open issues 10 grid points All grids Introduction to Stochastic Spectral Methods 25 Novembre 2008 103 / 133 Gauss quadrature rule are not nested (3) Introduction RVs and RPs 3D Hermite collocation grids for diﬀerent levels of resolutio Several UQ methods L=5 L=6 Spectral 5 5 representation e Karhunen-Lo`ve 2.5 2.5 Polynomial Chaos 0 0 x3 x3 generalized Polynomial !2.5 !2.5 Chaos Post-processing !5 !5 5 5 2.5 5 2.5 5 0 2.5 0 2.5 Resolution for a 0 0 !2.5 !2.5 !2.5 !2.5 general SPDE x2 !5 !5 x x2 !5 !5 x 1 1 Stochastic Galerkin Method (SGM) 53 grid points 63 grid points L=10 Stochastic Collocation Method (SCM) 5 5 Multivariate 2.5 2.5 quadratures 0 x3 0 x3 Full !2.5 Sparse !2.5 !5 !5 5 5 2.5 5 Intrude or not 2.5 5 2.5 0 2.5 0 intrude ? 0 0 !2.5 !2.5 !2.5 !2.5 x !5 !5 x x2 !5 !5 x 2 1 1 Open issues 103 grid points All grids Introduction to Stochastic Spectral Methods 25 Novembre 2008 104 / 133 Growth of one-dimensional Clenshaw-Curtis (Chebyshev Introduction extrema) grid RVs and RPs Several UQ methods 7 Spectral representation e Karhunen-Lo`ve Polynomial Chaos 6 generalized Polynomial Chaos Post-processing 5 Quadrature level Resolution for a general SPDE 4 Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) 3 Multivariate quadratures Full 2 Sparse Intrude or not 1 intrude ? −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 Quadrature values in 1D Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 105 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 106 / 133 Sparse quadrature - Smolyak algorithm (1) Introduction RVs and RPs If the tensor product of quadrature formulas combine high-order formulas in Several UQ methods only a few random dimensions with low-order formulas in the other dimensions −→ the resulting quadrature may be feasible in high dimensions. Spectral representation Introduction of diﬀerent quadrature level l in diﬀerent dimensions. e Karhunen-Lo`ve Polynomial Chaos We assume that the quadrature Qlj (in dimension Ωj ) integrates exactly generalized Polynomial Chaos polynomials of degree not exceeding Pl . (1) (N ) Post-processing For a l = (l , . . . , lN ) ∈ NN :QlN := Ql1 ⊗ . . . ⊗ QlN . Resolution for a general SPDE Let us assume also that each Qlj has the same number of nodes Nqlj Stochastic Galerkin and that the lowest level has a single node (Nq1j = 1, ∀j = 1 . . . N ). Method (SGM) (j ) (j ) (j ) Stochastic Collocation Method (SCM) Let the nodes Γl = {Zl,1 , . . . , Zl,Nq } and the weights lj (j ) (j ) Multivariate wl,1 , . . . , wl,Nq quadratures lj Full used by Qlj . Sparse Apply the quadrature to a function φ : Ω(N ) → R : Nql Nql Intrude or not X1 XN (1) (N ) (1) (N ) QlN [φ] := intrude ? ... wl1 ,i1 . . . wlN ,iN φ(Zl1 ,i1 , . . . , ZlN ,iN ). Open issues i1 =1 iN =1 Goal : build a quadrature for which only a few li are large. Introduction to Stochastic Spectral Methods 25 Novembre 2008 107 / 133 Sparse quadrature - Smolyak algorithm (2) Introduction RVs and RPs Several UQ methods Combine such quadratures by introducing quadrature diﬀerences : Spectral representation (j ) (j ) e Karhunen-Lo`ve ∆j := Ql l − Ql−1 , l ∈ N, j = 1, . . . , N , Polynomial Chaos (j ) generalized Polynomial Chaos and Q0 := 0, ∀j = 1 . . . N . Post-processing Resolution for a general SPDE Construct linear combinations of such quantities only for indices that Stochastic Galerkin fall within the unit simplex. Method (SGM) Stochastic Collocation Method (SCM) The level l Smolyak quadrature formula in N dimensions is : X (1) (N ) Multivariate quadratures QN = l ∆l1 ⊗ . . . ⊗ ∆lN , Full l ∈NN ,|l |≤N +l−1 Sparse where |l | = l1 + . . . + lN . Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 108 / 133 Sparse quadrature - Smolyak grid (1) Introduction RVs and RPs Every QlN quadrature evaluates the integrand on the grid: Several UQ methods (N ) (1) (N ) Spectral Γl = Γl1 × . . . × ΓlN . representation Karhunen-Lo`ve e QN evaluates the integrand on the union of these grids: l Polynomial Chaos [ [ (N ) (1) (N ) generalized Polynomial Chaos ΓN = l Γl = Γl1 × . . . × ΓlN . Post-processing |l|≤N +l−1 |l|≤N +l−1 Resolution for a As the sparse quadrature is enriched by increasing the quadrature level general SPDE from l to (l + 1), the grid points used in the earlier levels are retained. Stochastic Galerkin Method (SGM) The sparse quadrature is always embedded regardless of the nature of Stochastic Collocation Method (SCM) the 1D basis quadrature rule used. (j ) (j ) Multivariate If the one-dimensional quadrature rule are nested (i.e. Γl+1 ⊆ Γl quadratures (N ) (N ) Full then Γl ⊂ Γl when lj ≤ lj , j = 1, . . . , N ), this results in a Sparse much smaller number of collocation points compared to the non-nested Intrude or not formulas. intrude ? The Smolyak formula is actually interpolatory whenever nested points Open issues are used. Introduction to Stochastic Spectral Methods 25 Novembre 2008 109 / 133 Sparse quadrature - Smolyak grid (2) Introduction RVs and RPs Several UQ methods It is important to notice that the Smolyak algorithm will perform his Spectral representation task of building a 1D grid for any family of one-dimensional quadrature Karhunen-Lo`ve e rules. Polynomial Chaos generalized Polynomial Chaos The latter will aﬀect the Nc as well as the overall accuracy of the Post-processing integration. Resolution for a general SPDE At high dimensions, the sparse grid allows for a signiﬁcant reduction in Stochastic Galerkin Method (SGM) the numerical cost of the quadrature. Stochastic Collocation Method (SCM) Multivariate observed that the sparse grid is optimal only when the [Nobile 2008] quadratures integrand depends roughly uniformly on all dimensions. Full Sparse If the integrand is highly anisotropic, the convergence of the sparse grid Intrude or not integral will deteriorate. intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 110 / 133 Sparse quadrature - Smolyak grid (3) Introduction Higher Order Quadrature on Sparse Grids 395 RVs and RPs Visual representation of a sparse grid based on the midpoint rule Several UQ methods Spectral W 11 W 21 l1 representation e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial Chaos Post-processing W 12 Resolution for a general SPDE Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Multivariate quadratures Full Sparse Intrude or not intrude ? l2 Open issues Introduction to Stochastic subspaces represents Fig. 1. Scheme ofSpectral Methods for d = 2: Each square25 Novembre 2008one hierarchical subspace 111 / 133 Sparse quadrature - Smolyak grid (4) Introduction RVs and RPs Three-dimensional Gauss-Legendre Sparse Quadrature with L = 6. Several UQ methods Spectral 1 1 representation 0.5 0.5 e Karhunen-Lo`ve 0 0 ξ3 ξ3 Polynomial Chaos −0.5 −0.5 generalized Polynomial Chaos −1 −1 1 1 Post-processing 0.5 1 0.5 1 0 0.5 0 0.5 0 0 −0.5 −0.5 −0.5 −0.5 Resolution for a ξ2 −1 −1 ξ1 ξ2 −1 −1 ξ1 general SPDE Stochastic Galerkin Reduced hypercube in [0, 1]N After one reﬂection Method (SGM) Stochastic Collocation Method (SCM) 1 1 0.5 0.5 Multivariate quadratures 0 0 ξ3 ξ3 Full −0.5 −0.5 Sparse −1 −1 1 1 0.5 1 0.5 1 Intrude or not 0 0.5 0 0.5 0 0 −0.5 −0.5 intrude ? −0.5 −0.5 ξ2 −1 −1 ξ1 ξ2 −1 −1 ξ1 Open issues After two reﬂections Complete set of quadrature points Introduction to Stochastic Spectral Methods 25 Novembre 2008 112 / 133 Bounded support Introduction RVs and RPs The Smolyak sparse grid can be generated using many diﬀerent bounded 1D Several UQ methods quadrature rules. Spectral Most used : Fej´r and Clenshaw-Curtis (nested). e representation Karhunen-Lo`ve e Gauss-Legendre (not nested) can also be used as the basis of the tensor Polynomial Chaos product. generalized Polynomial Chaos 1 if l = 1 Clenshaw-Curtis 1D : ml = . Post-processing 2l−1 + 1 if l > 1 Resolution for a general SPDE Fej´r 1D : ml = 2l − 1 e Stochastic Galerkin Method (SGM) e Although nested, the Fej´r 1D quadrature rule grows faster than the Stochastic Collocation Clenshaw-Curtis 1D rule due to the exclusion of the end points on the Method (SCM) support [−1, 1]. Multivariate quadratures Gauss-type 1D : we have the choice on how 1D grid grows for levels Full l. Sparse Usually, an exponential growth for a grid with odd number of Intrude or not collocation points (starting with one point at the center at the lowest intrude ? level) is the most common. But one can also try diﬀerent growths such Open issues as: linear, linear odd, linear even, and exponential even. Introduction to Stochastic Spectral Methods 25 Novembre 2008 113 / 133 Summary of the 1D quadrature rules for diﬀerent weights Introduction RVs and RPs ρ(x ) Ω P Nq Analytic Solutions Nestedness Several UQ methods Gauss-Legendre 1 (-1,1) 2Nq − 1 L No No Gauss-Jacobi (1 − x )a (1 + x )b (-1,1) 2Nq − 1 L No No Spectral representation a, b > −1 π(2n−1) p e Karhunen-Lo`ve Gauss-Chebyshev 1/ 1 − x 2 (-1,1) 2Nq − 1 L zn = cos( 2L ), No Polynomial Chaos (Type I) wn = π L generalized Polynomial p Chaos Gauss-Chebyshev 1 − x2 (-1,1) 2Nq − 1 L πn zn = cos( L+1 ), No Post-processing (Type II) wn = L+1 sin2 ( L+1 ) π πn p Resolution for a Clenshaw-Curtis 1/ 1 − x 2 [-1,1] Nq − 1 2L−1 + 1 zn only Yes general SPDE p e Fej´r (Type II) 1/ 1 − x 2 (-1,1) Nq − 1 2L − 1 zn only Yes Stochastic Galerkin Method (SGM) Gauss-Laguerre e −x [0, ∞) 2Nq − 1 L No No Stochastic Collocation Method (SCM) −x 2 Gauss-Hermite e (−∞,∞) 2Nq − 1 L No No 2 Multivariate Gauss-Hermite √1 e −x /2 (−∞,∞) 2Nq − 1 L No No quadratures 2π Full (Normal) Sparse e Mapped Fej´r (−∞,∞) 2L − 1 zn only Yes (Type II) Intrude or not 2 Hermite √1 e −x /2 (−∞,∞) 2m + n − 1 1-3-9-19-35 No Yes intrude ? 2π Kronrod-Patterson or 1-4-18-30 Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 114 / 133 Examples for bounded support (1) Introduction RVs and RPs Several UQ methods Spectral representation Karhunen-Lo`ve e 2D Legendre collocation grids for diﬀerent levels of resolution Polynomial Chaos generalized Polynomial L=4 L=5 L=6 Chaos 1 1 1 Post-processing 0.5 0.5 0.5 Resolution for a general SPDE x2 x2 x2 0 0 0 Stochastic Galerkin Method (SGM) Stochastic Collocation !0.5 !0.5 !0.5 Method (SCM) Multivariate !1 !1 !0.5 0 0.5 1 !1 !1 !0.5 0 0.5 1 !1 !1 !0.5 0 0.5 1 quadratures x1 x1 x1 Full 49 grid points 129 grid points 321 grid points Sparse Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 115 / 133 Examples for bounded support (2) Introduction RVs and RPs Several UQ methods Spectral representation Karhunen-Lo`ve e 2D Chebyshev collocation grids for diﬀerent levels of resolution Polynomial Chaos generalized Polynomial L=4 L=5 L=6 Chaos 1 1 1 Post-processing 0.5 0.5 0.5 Resolution for a general SPDE x2 x2 x2 0 0 0 Stochastic Galerkin Method (SGM) Stochastic Collocation !0.5 !0.5 !0.5 Method (SCM) Multivariate !1 !1 !0.5 0 0.5 1 !1 !1 !0.5 0 0.5 1 !1 !1 !0.5 0 0.5 1 quadratures x1 x x1 1 Full 33 grid points 169 grid points 221 grid points Sparse Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 116 / 133 Examples for bounded support (3) Introduction RVs and RPs Several UQ methods Spectral representation Karhunen-Lo`ve e 2D Clenshaw-Curtis collocation grids for diﬀerent levels of resolution Polynomial Chaos generalized Polynomial L=4 L=5 L=6 Chaos 1 1 1 Post-processing 0.5 0.5 0.5 Resolution for a general SPDE x2 x2 x2 0 0 0 Stochastic Galerkin Method (SGM) Stochastic Collocation !0.5 !0.5 !0.5 Method (SCM) Multivariate !1 !1 !0.5 0 0.5 1 !1 !1 !0.5 0 0.5 1 !1 !1 !0.5 0 0.5 1 quadratures x1 x1 x1 Full 29 grid points 65 grid points 145 grid points Sparse Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 117 / 133 Examples for bounded support (4) Introduction RVs and RPs Several UQ methods Spectral representation Karhunen-Lo`ve e 2D Fejer-II collocation grids for diﬀerent levels of resolution Polynomial Chaos generalized Polynomial L=4 L=5 L=6 Chaos 1 1 1 Post-processing 0.5 0.5 0.5 Resolution for a general SPDE x2 x2 0 x2 0 0 Stochastic Galerkin Method (SGM) Stochastic Collocation !0.5 !0.5 !0.5 Method (SCM) Multivariate !1 !1 !0.5 0 0.5 1 !1 !1 !0.5 0 0.5 1 !1 !1 !0.5 0 0.5 1 quadratures x1 x1 x1 Full 49 grid points 129 grid points 321 grid points Sparse Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 118 / 133 Polynomial exactness (1) Introduction RVs and RPs Several UQ methods Polynomial exactness P (l , N ) of Smolyak sparse grid for diﬀerent 1D Spectral representation quadrature choices : e Karhunen-Lo`ve Polynomial Chaos 1. Gauss-type : P (l , N ) = 2L − 1 generalized Polynomial Chaos Post-processing 2. Clenshaw-Curtis : Resolution for a general SPDE 2l − 1 if l < 3N + 1; P (l , N ) = Stochastic Galerkin Method (SGM) 2σ−1 (N + τ + 1) + 2N − 1 otherwise Stochastic Collocation Method (SCM) e 3. Fej´r : Multivariate quadratures 2l − 1 if l < 2N + 1, P (l , N ) = 2σ−1 (N + τ + 1) − 1 Full Sparse otherwise Intrude or not where σ = ﬂoor((l + N − 1)/N ) and τ = (l + N − 1) mod N . intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 119 / 133 Polynomial exactness (2) Introduction RVs and RPs Accuracy and computational cost comparison between full quadratures Several UQ methods (Gauss-based and Clenshaw-Curtis-based) and sparse quadratures Spectral (Clenshaw-Curtis-based). representation N=2 30 40 e Karhunen-Lo`ve Sparse CC: N = 2 Full GQ Full CC Sparse CC: N = 5 35 Sparse CC 25 Polynomial Chaos Sparse CC: N = 10 30 generalized Polynomial 20 Polynomial order, p Polynomial order, p 25 Chaos 15 20 Post-processing 15 10 10 Resolution for a 5 5 general SPDE 0 0 0 1 2 3 4 5 6 7 0 1 2 3 10 10 10 10 10 10 10 10 10 10 10 10 Stochastic Galerkin Number of function calls Number of function calls Method (SGM) Stochastic Collocation N = {2, 5, 10} N =2 Method (SCM) N=5 N = 10 40 25 Full GQ Full GQ Full CC Full CC 35 Sparse CC Sparse CC Multivariate 30 20 quadratures Polynomial order, p Polynomial order, p 25 15 Full 20 Sparse 15 10 10 5 Intrude or not 5 intrude ? 0 0 0 2 4 6 8 10 0 2 4 6 8 10 10 10 10 10 10 10 10 10 10 10 10 10 Number of function calls Number of function calls Open issues N =5 N = 10 Introduction to Stochastic Spectral Methods 25 Novembre 2008 120 / 133 Integration of non-periodic, non-linear, discontinuous Introduction multi-dimensional functions : the Genz test RVs and RPs Several UQ methods 1. Oscillarory : f1 (x ) = cos 2πw1 + d ci xi . ` P ´ Spectral i=1 representation ´−1 2. Product Peak : f2 (x ) = d ` −2 + (xi − wi )2 Q e Karhunen-Lo`ve i=1 ci . Polynomial Chaos ` Pd ´−(d+1) generalized Polynomial 3. Corner Peak : f3 (x ) = 1 + i=1 ci xi . Chaos Post-processing ` Pd 2 2 ´ 4. Gaussian : f4 (x ) = exp − i=1 ci (xi − wi ) . Resolution for a 5. Continuous : f5 (x ) = exp − d ci |xi − wi | . ` P ´ general SPDE i=1 Stochastic Galerkin Method (SGM) 0 ` ´ if x1 > w1 or x2 > w2 Stochastic Collocation 6. Discontinuous : f6 (x ) = Pd Method (SCM) exp i=1 ci xi otherwise Multivariate quadratures The sparse Clenshaw-Curtis quadrature method is more eﬃcient and more Full accurate than the full Gauss-quadrature method for the integration of Sparse multi-dimensional, non-periodic smooth functions when the number of dimensions N > 5. Intrude or not intrude ? Major convergence problems remain for both methods for the integration of Open issues discontinuous and/or irregular and/or anisotropic functions. Introduction to Stochastic Spectral Methods 25 Novembre 2008 121 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 122 / 133 Intrude or not intrude ? (1) Introduction RVs and RPs Several UQ methods Stochastic Galerkin method (SGM) : Spectral representation One deals with the weak form of the solution. e Karhunen-Lo`ve Polynomial Chaos Existing numerical solvers need to be extensively adapted (e.g. include generalized Polynomial Chaos the trial functions). Post-processing Resolution for a Diﬃculty with non-linear & non-polynomial functionals representations. general SPDE Stochastic Galerkin Method (SGM) The resulting set of deterministic equations are strongly coupled. Stochastic Collocation Method (SCM) Numerical solvers need to be very robust. Multivariate quadratures Stability issue KL/PC/gPC converge in L2 , not in L∞ (uniformly). Full Sparse Curse of dimensionality. Intrude or not intrude ? Better (spectral) theory background (error analysis, basis reduction, ...) Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 123 / 133 Intrude or not intrude ? (2) Introduction RVs and RPs Several UQ methods Stochastic collocation method (SCM) : Spectral representation One deals with the strong form of the solution. e Karhunen-Lo`ve Polynomial Chaos generalized Polynomial No modiﬁcation of the numerical solver is needed (black box). Chaos Post-processing Curse of dimensionality. Resolution for a general SPDE Stochastic Galerkin The solutions are inﬂuenced by the deterministic black box solver error, Method (SGM) Stochastic Collocation the quadrature approximation error, in addition to the gPC truncation Method (SCM) error of inputs & solution. Multivariate quadratures Overcomes the problem of nonlinearity. Full Sparse For a speciﬁc realization, the problem may not be well-posed or the Intrude or not intrude ? solver may not converge. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 124 / 133 Intrude or not intrude ? (3) Introduction RVs and RPs Several UQ methods Let’s play with some very crude assumptions here. Let’s assume that we use direct solvers (not adaptive). We assume moreover: Spectral representation We own a deterministic PDE solver and that it takes α time units to Karhunen-Lo`ve e run a single simulation for a single set of input parameters. Polynomial Chaos generalized Polynomial We own a SGM solver that is not aﬀected by the additional cost coming Chaos Post-processing from the coupling of the gPC modes in the system of equations. N Our stochastic solution is well represented by a polynomial form PP of Resolution for a general SPDE highest degree P in N iid RVs with known distributions. Stochastic Galerkin Method (SGM) =⇒ Need M ≡ M (N , P ) gPC modes to represent the solution. Stochastic Collocation Method (SCM) Then: Multivariate For full-tensor or sparse-based SCM, the total computational eﬀort is quadratures Full α × Nc (N , P ). Sparse For SGM, the total computational eﬀort is α × M (N , P ). Intrude or not Next slide compares those estimates for full Gauss quadrature and intrude ? Smolyak Clenshaw-Curtis quadrature and for diﬀerent N dimensions. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 125 / 133 Intrude or not intrude ? (4) Introduction RVs and RPs Comparison of computational eﬀort for the SGM and the SCM with Several UQ methods increasing degree of the gPC expansion, P , at N = 3 to N = 20. Spectral 6 N=3 8 N=5 10 10 representation Galerkin Method Galerkin Method Collation with Full Quadrature Collation with Full Quadrature e Karhunen-Lo`ve 10 5 Collation with Sparse Quadrature Collation with Sparse Quadrature Normalized Degrees of Freedom Normalized Degrees of Freedom 6 Polynomial Chaos 10 4 10 generalized Polynomial Chaos 3 4 10 10 Post-processing 2 10 2 10 Resolution for a 1 10 general SPDE Stochastic Galerkin 10 0 0 10 0 0 1 1 Method (SGM) 10 10 10 10 P P Stochastic Collocation N=10 N=20 14 25 Method (SCM) 10 10 Galerkin Method Galerkin Method 12 10 Collation with Full Quadrature Collation with Full Quadrature Collation with Sparse Quadrature 20 Collation with Sparse Quadrature Normalized Degrees of Freedom Normalized Degrees of Freedom 10 Multivariate 10 10 quadratures 15 8 10 10 Full Sparse 6 10 10 10 4 10 Intrude or not 5 10 2 intrude ? 10 0 0 10 0 1 10 10 10 1 2 3 4 5 Open issues P P Introduction to Stochastic Spectral Methods 25 Novembre 2008 126 / 133 Introduction 1 Introduction RVs and RPs RVs and RPs Several UQ methods Several UQ methods Spectral 2 Discretization and spectral representation of random ﬁelds representation Karhunen-Lo`ve e e Karhunen-Lo`ve representation Polynomial Chaos Homogeneous Chaos / Polynomial Chaos representation generalized Polynomial Chaos generalized Polynomial Chaos representation Post-processing Post-processing Resolution for a 3 Resolution for a general SPDE general SPDE Stochastic Galerkin Method (SGM) Stochastic Galerkin Method (SGM) Stochastic Collocation Method (SCM) Stochastic Collocation Method (SCM) 4 Multivariate quadratures Multivariate Full tensor-based quadrature quadratures Sparse Smolyak-based quadrature Full Sparse 5 Intrude or not intrude ? 6 Open issues Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 127 / 133 Open issues Introduction RVs and RPs Main area of current research : Several UQ methods Well-posedness Gaussian measure; sign properties for BVP; error of truncation; Spectral importance of the tails; truncated Jacobi-expansion. representation e Karhunen-Lo`ve Polynomial Chaos “Curse of dimensionality”. generalized Polynomial Chaos Low stochastic regularity. Post-processing Long-term integration : problem of random frequency; oscillators; gPC loses Resolution for a general SPDE p-convergence after a ﬁnite time; polynomial order must with time. Stochastic Galerkin Method (SGM) Space/Basis optimality : arbitrary measures, reduced basis, low-dimensional Stochastic Collocation Method (SCM) modeling, connection to POD, dependent RVs. Multivariate quadratures Computational complexity : relates to dimensionality; development of framework Full for stochastic coupling and parallelization; pre-conditioning; multi-grid approach; Sparse hierarchical parallelization. Intrude or not intrude ? Optimization/Design under uncertainty : robust design; surrogate-based optimization; orthogonal polynomials numerically generated; PCM & SC-gPC; anisotropic Open issues cubature. Introduction to Stochastic Spectral Methods 25 Novembre 2008 128 / 133 “Curse of dimensionality” Introduction RVs and RPs Several UQ methods Spectral representation Karhunen-Lo`ve e The number N required for accurate representation of input processes Polynomial Chaos with low Cl can be extremely large. generalized Polynomial Chaos Post-processing e Karhunen-Lo`ve inputs with moderately high dimensionality (N > 10) Resolution for a pose major computational challenges for stochastic spectral methods. general SPDE Stochastic Galerkin Method (SGM) Stochastic Collocation Problem even worse in case of low stochastic regularity. Method (SCM) Multivariate Complexity of sparse grids still depends heavily on N and on the quadratures Full regularity of the integrand. Sparse Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 129 / 133 “Curse of dimensionality” - Answers (among others) Introduction RVs and RPs Several UQ methods “coarsening” or “sparsiﬁcation” of the tensor product gPC subspaces Spectral representation [Schwab]; Reduced stochastic basis [Nair 2002, Nouy 2007]; variable Karhunen-Lo`ve e stochastic polynomial degrees. Sparse Finite Elements for stochastic Polynomial Chaos problems [Schwab 2003] generalized Polynomial Chaos Post-processing Dimension-adaptative tensor-product sparse quadrature. Adaptation of Resolution for a the collocation grid (iteratively) to the anisotropy of the solution; general SPDE Stochastic Galerkin development of error analysis to select the best criteria [Nobile 2008, Method (SGM) Crestaux 2008]. Stochastic Collocation Method (SCM) Multivariate ANOVA-type decomposition coupled with ME-PCM: MEPCM-A [Foo quadratures 2008]. “Analysis-of-Variance” involves splitting a multidimensional Full Sparse function into its contributions from diﬀerent groups of subdimensions. Promising results on discontinuous functions from the Genz test. Intrude or not intrude ? MEPCM-A is favorable to MC up to 600 dimensions. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 130 / 133 Low stochastic regularity - PC/gPC failures Introduction RVs and RPs Several UQ methods Non-linearities in the random space (especially for the stochastic Spectral representation Galerkin approach) can cause problems as they require high polynomials Karhunen-Lo`ve e order. Polynomial Chaos generalized Polynomial This problem is ampliﬁed by the way the standard PC/gPC is Chaos constructed as a global (p-type) approximation over the uncertainty Post-processing range. Resolution for a general SPDE Discontinuous solution, e.g. hyperbolic problems, or non-smooth Stochastic Galerkin Method (SGM) stochastic solutions - also called parametric or stochastic bifurcation - Stochastic Collocation Method (SCM) wrt to the uncertain parameters are not well represented by smooth polynomials. Multivariate quadratures → Gibbs phenomenon → strong oscillations (can induce loss of Full Sparse well-posedness, e.g. negative value of a positive physical quantity!). Intrude or not intrude ? This problem aﬀects both SGM and SCM/PCM approaches. Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 131 / 133 Low stochastic regularity - Answers among others (1) Introduction RVs and RPs Several UQ methods Use diﬀerent types of chaos basis or change the nature of the expansion Spectral representation to handle the steep dependency or oscillating character of the solution : e Karhunen-Lo`ve Polynomial Chaos ıtre Haar-wavelets [Le Maˆ 2004], Pade-Legendre, piecewise polynomials generalized Polynomial (B-Splines [Millman 2004]). Chaos Post-processing Resolution for a Make the spectral decomposition more local by decomposing the general SPDE random space (the probability measure) into diﬀerent regions or Stochastic Galerkin diﬀerent scales : Method (SGM) Stochastic Collocation Method (SCM) Piecewise polynomials (h/k-type), stochastic ﬁnite elements [Deb 2001, Babuska 2002]. Multivariate quadratures Multi-elements (h/p-type) gPC (ME-GPC) [Wan 2005], Multi-elements Full PCM (ME-PCM) [Foo 2008] Sparse ıtre Multi-wavelets/Multi-resolution [Le Maˆ 2004]. Intrude or not intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 132 / 133 Low stochastic regularity - Answers among others (2) Introduction RVs and RPs Several UQ methods Constrain the system by using the gPC representation for the entropic Spectral variable : representation e Karhunen-Lo`ve bound the oscillations of the solution in the random space by working out Polynomial Chaos the SGM onto the entropic variable (linked to the main variable through generalized Polynomial Chaos the entropy of the system) instead, e.g. systems of conservation laws Post-processing e [Po¨tte 2008]. Resolution for a general SPDE Adapt the quadrature grid to the strong solution gradients : Stochastic Galerkin Method (SGM) Stochastic Collocation Reduce the aliasing error of the SCM methods by sampling more where Method (SCM) necessary. Adapt the quadrature method to the strong solution Multivariate gradients. quadratures Full Dimension-adaptative tensor-product sparse quadrature. Adaptation of Sparse the collocation grid (iteratively) to the anisotropy of the solution; development of error analysis to select the best criteria [Nobile 2008, Intrude or not Crestaux 2008]. intrude ? Open issues Introduction to Stochastic Spectral Methods 25 Novembre 2008 133 / 133