VIEWS: 0 PAGES: 37 POSTED ON: 2/9/2012 Public Domain
Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1 A.vanKeulen@tudelft.nl WB1440 Engineering Optimization – Concepts and Applications Contents ● Optimization problem checking and simplification ● Model simplification WB1440 Engineering Optimization – Concepts and Applications Model simplification ● Basic idea: Expensive Cheap model Optimizer ● Motivation: – Replacement of expensive function, evaluated many times – Interaction between different disciplines – Estimation of derivatives – Noise WB1440 Engineering Optimization – Concepts and Applications Model simplification (2) ● Drawback: loss of accuracy ● Different ranges: local, mid-range, global ● Synonyms: – Approximation models Procedure: – Metamodels – Surrogate models – Compact models Extract Construct information approximation – Reduced order models WB1440 Engineering Optimization – Concepts and Applications Model simplification (3) ● Information extraction: linked to techniques from physical experiments: “plan of experiments” / DoE ● Many approaches! Covered here: – Taylor series expansions – Exact fitting – Least squares fitting (response surface techniques) – Kriging – Reduced basis methods – Briefly: neural nets, genetic programming, simplified physical models ● Crucial: purpose, range and level of detail WB1440 Engineering Optimization – Concepts and Applications Taylor series expansions ● Approximation based on local information: 1 1 f ( x h) f ( x ) f ' ( x ) h f ' ' ( x ) h 2 1 1! 2! 1 (n) f ( x)h n n 0 n! Truncation error! N N 1 (n) f ( x ) h o( h ) f ( x ) h 1 (n) n N n n 0 n! n 0 n! ● Use of derivative information! ● Valid in neighbourhood of x WB1440 Engineering Optimization – Concepts and Applications Taylor approximation example 5 x cos(x / 5) f x / 2 1 e 5 3 Function Approximation (x = 20) th20th order 45th order rdorder st order 312nd order order x WB1440 Engineering Optimization – Concepts and Applications Exact fitting (interpolation) ● # datapoints = # fitting parameters ● Every datapoint reproduced exactly ● Example: f a0 a1 x f2 1 x1 a0 f1 1 x a f f1 2 1 2 x1 x2 WB1440 Engineering Optimization – Concepts and Applications Exact fitting (2) ● Easy for intrinsically linear functions: n f a0 f 0 a1 f1 a2 f 2 ai f i i 1 ● Often used: polynomials, generalized polynomials: f a bx1m x2 n log( f a ) log b m log x1 n log x2 ● No smoothing / filtering / noise reduction ● Danger of oscillations with high-order polynomials WB1440 Engineering Optimization – Concepts and Applications Oscillations 9th order polynomial ● Referred to as “Runge phenomenon” 1 5th order 9th order 1 25x 2 ● In practice: use order 6 or less WB1440 Engineering Optimization – Concepts and Applications Least squares fitting ● Less fitting parameters than datapoints ● Smoothing / filtering behaviour ● “Best fit”? Minimize sum of deviations: squared deviations: N ~ min | f ( xii ) f ( xi ) | 2 a i 1 f ~ f x WB1440 Engineering Optimization – Concepts and Applications Least squares fitting (2) ● Choose fitting function linear in parameters ai : ~ f ( x) a0 f0 ( x) a1 f1 ( x) a2 f 2 ( x) am f m ( x) ~ f ( x0 ) f 0 ( x0 ) f1 ( x0 ) f 2 ( x0 ) f m ( x0 ) a0 0 ~ f ( x1 ) f 0 ( x1 ) f1 ( x1 ) f 2 ( x1 ) f m ( x1 ) a1 1 ~ f ( x2 ) f 0 ( x2 ) f1 ( x2 ) f 2 ( x2 ) f m ( x2 ) a2 2 ~ f ( x N ) f 0 ( x N ) f1 ( xN ) f 2 ( xN ) f m ( x N ) am N ~ ● Short notation: f Ma ε WB1440 Engineering Optimization – Concepts and Applications LS fitting (3) ● Minimize sum of squared errors: a ~ min L ε ε f Ma T T ~ f Ma (Optimization problem!) L a T ~ T~ 0 2M f Ma 2M f 2MT Ma M Ma M f T ~ T ~ a M M M f T 1 T WB1440 Engineering Optimization – Concepts and Applications Polynomial LS fitting ● Polynomial of degree m: ~ f ( x) a0 x0 a1x1 a2 x2 am xm ~ ~ N f ( x0 ) xi 1 0 xi 2 x0 2 x0i m a00 0 f i 2 x x a ~ m m 1 ~ xm2 a 1 ~ 2 xi f ( x i 1 1 xi x1 xi1 a11 f i xi 3 2 1) x x x2 ~ ( x 1 x 4 x 2 f 2) i x 3 x2 i 2 xi2 a2 i i x m a 2 f x i 2 x m~ m 1 2m ~ m xx f ( xm )xi 1 mi xm xm am m f i xi xi am m2 2 m i WB1440 Engineering Optimization – Concepts and Applications Polynomial LS example 1.2 samples quadratic 1 6th degree 0.8 0.6 0.4 0.2 0 -0.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 WB1440 Engineering Optimization – Concepts and Applications Multidimensional LS fitting ● Polynomial in multiple dimensions: ~ f ( x, y ) a0 a1 x a2 y a3 x 2 a4 y 2 a5 xy a6 x 3 a7 y 3 a8 x 2 y a9 xy 2 ai f i ● Number of coefficients ai for quadratic polynomial in Rn: 1 m (n 1)(n 2) 2 Curse of dimensionality! WB1440 Engineering Optimization – Concepts and Applications Response surface ● Generate datapoints through sampling: – Generate design points through Design of Experiments – Evaluate responses x3 ● Fit analytical model ● Check accuracy x2 x1 2n full factorial design Fractional factorial design WB1440 Engineering Optimization – Concepts and Applications Latin Hypercube Sampling (LHS) ● Popular method: LHS ● Based on idea of Latin square: ● Properties: 1 – Space-filling 0.8 – Any number of design points 0.6 0.4 – Intended for box-like domains 0.2 – Matlab: lhsdesign 0 0 0.2 0.4 0.6 0.8 1 WB1440 Engineering Optimization – Concepts and Applications (LS) Fit quality indicators ● Accuracy? More / fewer terms? ● Examine the residuals 0 – Small – Random! xi ● Statistical quality indicators: i 2 – R2 correlation measure: R2 1 ~ 2 Okay: >0.6 fi f – F-ratio (signal to noise): F ~ f f i 2 m Okay: >>1 N m 1 2 i WB1440 Engineering Optimization – Concepts and Applications Nonlinear LS ● Linear LS: intrinsically linear functions (linear in ai): f ( x) a0 x 0 a1 x1 a2 x 2 aT p f ( x) a0e x a1 log x a2 x aT p ● Nonlinear LS: more complicated functions of ai: a0 x f ( x) a1 x a2 x 2 1 ● More difficult to fit! (Nonlinear optimization problem) ● Matlab: lsqnonlin WB1440 Engineering Optimization – Concepts and Applications LS pitfalls ● Scattered data: f x ● Wrong choice of basis functions: f x WB1440 Engineering Optimization – Concepts and Applications Kriging ● Named after D.C. Krige, mining engineer, 1951 ● Statistical approach: correlation between neighbouring points – Interpolation by weighted sum: N y ( x) i ( x, xi ) yi i 1 – Weights depend on distance – Certain spatial correlation function is assumed (usually Gaussian) WB1440 Engineering Optimization – Concepts and Applications Kriging properties ● Kriging interpolation is “most likely” in some sense (based on assumptions of the method) ● Interpolation: no smoothing / filtering ● Many variations exist! ● Advantage: no need to assume form of interpolation function ● Fitting process more elaborate than LS procedure WB1440 Engineering Optimization – Concepts and Applications Kriging example ● Results depend strongly on statistical assumptions and method used: Dataset z(x,y) Kriging interpolation WB1440 Engineering Optimization – Concepts and Applications Reduced order model ● Idea: describing system in reduced basis: – Example: structural dynamics ~ ~ ~ Mu Ku f Mw Kw f ● Select small number of “modes” to build basis – Example: eigenmodes WB1440 Engineering Optimization – Concepts and Applications Reduced order model (2) k N ● Reduced basis: u ω w i 1 i i B ω1 ω 2 ω k u Bw N1 Nk k1 ● Reduced system equations: Mu Ku f ~ M BT MB kN NN Nk MB w KBw f ~ K B KB T BT MB w BT KBw BT f ~ f BT f kN N1 u Bw WB1440 Engineering Optimization – Concepts and Applications Reduced order models ● Many approaches! – Selection of type and number of basis vectors – Dealing with nonlinearity / multiple disciplines ● Active research topic ● No interpolation / fitting, but approximate modeling WB1440 Engineering Optimization – Concepts and Applications Example:Aircraft model Structural model Mass model Aerodynamic model WB1440 Engineering Optimization – Concepts and Applications Neural nets WB1440 Engineering Optimization – Concepts and Applications Neural nets x f(x) output S(input) To determine internal neuron parameters, neural nets must be trained on data. WB1440 Engineering Optimization – Concepts and Applications Neural net features ● Versatile, can capture complex behavior ● Filtering, smoothing ● Many variations possible – Network – Number of neurons, layers – Transfer functions ● Many training steps might be required (nonlinear optimization) ● Matlab: see e.g. nndtoc WB1440 Engineering Optimization – Concepts and Applications Genetic programming ● Building mathematical functions using evolution-like approach ● Approach good fit by crossover and mutation of expressions ^2 2 x1 x3 x + 2 / x3 x1 x2 WB1440 Engineering Optimization – Concepts and Applications Genetic programming ● LS fitting with population of analytic expressions ● Selection / evolution rules ● Features: – Can capture very complex behavior – Danger of artifacts / overfitting – Quite expensive procedure WB1440 Engineering Optimization – Concepts and Applications Simplified physical models ● Goal: capture trends from underlying physics through simpler model: – Lumped / Analytic / Coarse ● Parameters fitted to “high-fidelity” data ● Refinement: correction function, parameter functions ... x Simplified Correction f(x) model function WB1440 Engineering Optimization – Concepts and Applications Model simplification summary Many different approaches: – Local: Taylor series (needs derivatives) – Interpolation (exact fit): (Polynomial) fitting Kriging – Fitting: LS – Approximate modeling: reduced order / simplified models – Other: genetic programming, neural nets, etc WB1440 Engineering Optimization – Concepts and Applications Response surfaces in optimization ● Popular approach for computationally expen- sive problems: Expensive Cheap model 1. DoE, generate samples (expensive) in part of domain Optimizer 2. Build response surface (cheap) 3. Perform optimization on response surface (cheap) 4. Update domain of interest, and repeat ● Additional advantage: smoothens noisy responses ● Easy to combine with parallel computing WB1440 Engineering Optimization – Concepts and Applications Example: Multi-point Approximation Method Design domain Optimum (Expensive) simulation Sub-optimal point Trust region Response surface WB1440 Engineering Optimization – Concepts and Applications