# Engineering Optimization by yurtgc548

VIEWS: 0 PAGES: 37

• pg 1
Engineering Optimization
Concepts and Applications

Fred van Keulen
Matthijs Langelaar
CLA H21.1
A.vanKeulen@tudelft.nl

WB1440 Engineering Optimization – Concepts and Applications
Contents
● Optimization problem checking and simplification

● Model simplification

WB1440 Engineering Optimization – Concepts and Applications
Model simplification
● Basic idea:

Expensive
Cheap
model

Optimizer

● Motivation:

– Replacement of expensive function, evaluated many times

– Interaction between different disciplines

– Estimation of derivatives

– Noise

WB1440 Engineering Optimization – Concepts and Applications
Model simplification (2)
● Drawback: loss of accuracy

● Different ranges: local, mid-range, global

● Synonyms:

– Approximation models                           Procedure:
– Metamodels

– Surrogate models

– Compact models                                    Extract      Construct
information   approximation
– Reduced order models

WB1440 Engineering Optimization – Concepts and Applications
Model simplification (3)
● Information extraction: linked to techniques from
physical experiments: “plan of experiments” / DoE

● Many approaches! Covered here:

– Taylor series expansions
– Exact fitting

– Least squares fitting (response surface techniques)

– Kriging
– Reduced basis methods
– Briefly: neural nets, genetic programming, simplified physical models

● Crucial: purpose, range and level of detail

WB1440 Engineering Optimization – Concepts and Applications
Taylor series expansions
● Approximation based on local information:

1             1
f ( x  h)  f ( x )  f ' ( x ) h  f ' ' ( x ) h 2  
1

1!            2!

1 (n)
  f ( x)h n
n  0 n!                          Truncation error!
N                                  N
1 (n)
  f ( x ) h  o( h )   f ( x ) h
1 (n)       n      N                      n

n  0 n!                           n  0 n!

● Use of derivative information!

● Valid in neighbourhood of x

WB1440 Engineering Optimization – Concepts and Applications
Taylor approximation example
5           x cos(x / 5)
f       x / 2
   
1 e          5     3

Function
Approximation
(x = 20)

th20th order
45th order
rdorder
st order
312nd order
order

x

WB1440 Engineering Optimization – Concepts and Applications
Exact fitting (interpolation)
● # datapoints = # fitting parameters

● Every datapoint reproduced exactly

● Example:

f  a0  a1 x                        f2

1 x1  a0   f1 
1 x   a    f 
f1
   2  1     2
x1   x2
WB1440 Engineering Optimization – Concepts and Applications
Exact fitting (2)
● Easy for intrinsically linear functions:
n
f  a0 f 0  a1 f1  a2 f 2     ai f i
i 1

● Often used: polynomials, generalized polynomials:

f  a  bx1m x2 
n
log( f  a )  log b  m log x1  n log x2

● No smoothing / filtering / noise reduction

● Danger of oscillations with high-order polynomials

WB1440 Engineering Optimization – Concepts and Applications
Oscillations             9th order
polynomial

● Referred to as “Runge phenomenon”

1
5th order
9th order                         1  25x 2

● In practice: use order 6 or less

WB1440 Engineering Optimization – Concepts and Applications
Least squares fitting
● Less fitting parameters than datapoints

● Smoothing / filtering behaviour

● “Best fit”? Minimize sum of deviations:
squared deviations:

                  
N
~
min  | f ( xii )  f ( xi ) |
2

a
i 1                                     f

~
f

x

WB1440 Engineering Optimization – Concepts and Applications
Least squares fitting (2)
● Choose fitting function linear in parameters ai :
~
f ( x)  a0 f0 ( x)  a1 f1 ( x)  a2 f 2 ( x)   am f m ( x)  
~
 f ( x0 )   f 0 ( x0 )              f1 ( x0 )     f 2 ( x0 )      f m ( x0 )   a0    0 
~          
 f ( x1 )   f 0 ( x1 )              f1 ( x1 )     f 2 ( x1 )       f m ( x1 )   a1   1 
   
~                                                                                    
  f ( x2 )    f 0 ( x2 )            f1 ( x2 )     f 2 ( x2 )       f m ( x2 )    a2     2 
                                                                      
   

~                                                                                   
 f ( x N )  f 0 ( x N )
                                   f1 ( xN )      f 2 ( xN )      f m ( x N ) am   N 
   
~
● Short notation:                        f  Ma  ε

WB1440 Engineering Optimization – Concepts and Applications
LS fitting (3)
● Minimize sum of squared errors:

a
~
min L  ε ε  f  Ma
T
            
T   ~
f  Ma   
(Optimization problem!)

L
a
T ~          T~

 0  2M f  Ma  2M f  2MT Ma 
M Ma  M f
T     ~         T
                    ~
a M M M f  T

1   T

WB1440 Engineering Optimization – Concepts and Applications
Polynomial LS fitting
● Polynomial of degree m:
~
f ( x)  a0 x0  a1x1  a2 x2   am xm  
~                                                      ~
 N  f ( x0 ) xi 1 0 xi 2 x0 2 x0i m  a00    0 f i 
 2     x                x       a       
    ~                                   m 
m 1       
   ~ 
 xm2  a   1 ~ 2 
  xi f ( x i 1 1 xi x1   xi1  a11    f i xi 
3   2
       1) x      x                               
  x2 ~ ( x  1  x 4 x 2
   f 2) i    x
3
x2 i 2
 
 xi2   a2     i i
x
m 
 a 2    f x 
        
i                                                     2

                                               
          
 x  m~        m 1                    2m 
  
   ~ m 
   xx
  f ( xm )xi 1  mi xm   xm am   m f i xi 
xi   am  
m2 2            m
  
              

i         

WB1440 Engineering Optimization – Concepts and Applications
Polynomial LS example
1.2
samples
1
6th degree

0.8

0.6

0.4

0.2

0

-0.2
-1   -0.8   -0.6   -0.4   -0.2   0     0.2   0.4   0.6   0.8       1

WB1440 Engineering Optimization – Concepts and Applications
Multidimensional LS fitting
● Polynomial in multiple dimensions:
~
f ( x, y )  a0  a1 x  a2 y  
a3 x 2  a4 y 2  a5 xy  
a6 x 3  a7 y 3  a8 x 2 y  a9 xy 2    
  ai f i  
● Number of coefficients ai for quadratic polynomial in Rn:
1
m  (n  1)(n  2)
2
Curse of dimensionality!
WB1440 Engineering Optimization – Concepts and Applications
Response surface
● Generate datapoints through sampling:

– Generate design points through Design of Experiments

– Evaluate responses
x3
● Fit analytical model

● Check accuracy
x2

x1
2n full factorial design
Fractional factorial design
WB1440 Engineering Optimization – Concepts and Applications
Latin Hypercube Sampling (LHS)
● Popular method: LHS

● Based on idea of Latin square:

● Properties:                                           1

– Space-filling                                  0.8

– Any number of design points
0.6

0.4
– Intended for box-like domains
0.2

– Matlab: lhsdesign
0
0   0.2   0.4   0.6   0.8   1

WB1440 Engineering Optimization – Concepts and Applications
(LS) Fit quality indicators
● Accuracy? More / fewer terms?

● Examine the residuals

0
– Small

– Random!
xi
● Statistical quality indicators:
  i 2
– R2 correlation measure:                         R2  1
                
~            2       Okay: >0.6
fi  f

– F-ratio (signal to noise):                      F
~  f 
f          i
2
m
Okay: >>1
   N  m  1
2
i

WB1440 Engineering Optimization – Concepts and Applications
Nonlinear LS
● Linear LS: intrinsically linear functions (linear in ai):

f ( x)  a0 x 0  a1 x1  a2 x 2    aT p

f ( x)  a0e x  a1 log x  a2 x    aT p

● Nonlinear LS: more complicated functions of ai:

a0 x
f ( x) 
a1 x  a2 x 2  1
● More difficult to fit! (Nonlinear optimization problem)

● Matlab: lsqnonlin

WB1440 Engineering Optimization – Concepts and Applications
LS pitfalls
● Scattered data:
f

x
● Wrong choice of
basis functions:                            f

x

WB1440 Engineering Optimization – Concepts and Applications
Kriging
● Named after D.C. Krige, mining engineer, 1951

● Statistical approach: correlation between neighbouring
points
– Interpolation by weighted sum:
N
y ( x)   i ( x, xi ) yi
i 1
– Weights depend on distance

– Certain spatial correlation
function is assumed
(usually Gaussian)

WB1440 Engineering Optimization – Concepts and Applications
Kriging properties
● Kriging interpolation is “most likely” in some sense
(based on assumptions of the method)
● Interpolation: no smoothing / filtering

● Many variations exist!

● Advantage: no need to assume form of
interpolation function
● Fitting process more elaborate than LS procedure

WB1440 Engineering Optimization – Concepts and Applications
Kriging example
● Results depend strongly on statistical assumptions and
method used:

Dataset z(x,y)                                 Kriging interpolation

WB1440 Engineering Optimization – Concepts and Applications
Reduced order model
● Idea: describing system in reduced basis:

– Example: structural dynamics
~    ~    ~
Mu  Ku  f 
                                  Mw  Kw  f
● Select small number of “modes” to build basis

– Example: eigenmodes

WB1440 Engineering Optimization – Concepts and Applications
Reduced order model (2)
k  N
● Reduced basis:                    u     ω w
i 1
i   i

B  ω1 ω 2  ω k                                 u  Bw
N1   Nk   k1
● Reduced system equations:

Mu  Ku  f
                                                ~
M  BT MB        kN   NN   Nk
MB w  KBw  f
                                           ~
K  B KB
T

BT MB w  BT KBw  BT f
                                         ~
 f  BT f        kN   N1
u  Bw


WB1440 Engineering Optimization – Concepts and Applications
Reduced order models
● Many approaches!

– Selection of type and number of basis vectors

– Dealing with nonlinearity / multiple disciplines

● Active research topic

● No interpolation / fitting, but approximate modeling

WB1440 Engineering Optimization – Concepts and Applications
Example:Aircraft model

Structural model

Mass model

Aerodynamic model

WB1440 Engineering Optimization – Concepts and Applications
Neural nets

WB1440 Engineering Optimization – Concepts and Applications
Neural nets

x                                           f(x)

output

S(input)
To determine internal neuron parameters,
neural nets must be trained on data.

WB1440 Engineering Optimization – Concepts and Applications
Neural net features
● Versatile, can capture complex behavior

● Filtering, smoothing

● Many variations possible

– Network

– Number of neurons, layers

– Transfer functions

● Many training steps might be required
(nonlinear optimization)
● Matlab: see e.g. nndtoc
WB1440 Engineering Optimization – Concepts and Applications
Genetic programming
● Building mathematical functions using
evolution-like approach

● Approach good fit by crossover and
mutation of expressions

^2
2
 x1   
  x3 
x     
+
 2                            /           x3

x1        x2

WB1440 Engineering Optimization – Concepts and Applications
Genetic programming
● LS fitting with population of analytic expressions

● Selection / evolution rules

● Features:

– Can capture very complex
behavior
– Danger of artifacts /
overfitting
– Quite expensive procedure

WB1440 Engineering Optimization – Concepts and Applications
Simplified physical models
● Goal: capture trends from underlying physics through
simpler model:
– Lumped / Analytic / Coarse

● Parameters fitted to “high-fidelity” data

● Refinement: correction function, parameter functions ...

x              Simplified                    Correction   f(x)
model                        function

WB1440 Engineering Optimization – Concepts and Applications
Model simplification summary
Many different approaches:
– Local: Taylor series (needs derivatives)

– Interpolation (exact fit):

 (Polynomial) fitting

 Kriging

– Fitting: LS

– Approximate modeling: reduced order / simplified models

– Other: genetic programming, neural nets, etc

WB1440 Engineering Optimization – Concepts and Applications
Response surfaces in optimization
●     Popular approach for
computationally expen-
sive problems:
Expensive
Cheap
model
1. DoE, generate samples
(expensive) in part of domain
Optimizer
2. Build response surface (cheap)

3. Perform optimization on response surface (cheap)

4. Update domain of interest, and repeat

●     Easy to combine with parallel computing
WB1440 Engineering Optimization – Concepts and Applications
Example: Multi-point
Approximation Method
Design domain

Optimum
(Expensive) simulation

Sub-optimal point

Trust region
Response surface

WB1440 Engineering Optimization – Concepts and Applications

To top