Docstoc

Engineering Optimization

Document Sample
Engineering Optimization Powered By Docstoc
					                    Engineering Optimization
                           Concepts and Applications




                                                              Fred van Keulen
                                                              Matthijs Langelaar
                                                              CLA H21.1
                                                              A.vanKeulen@tudelft.nl

WB1440 Engineering Optimization – Concepts and Applications
                                           Contents
       ● Optimization problem checking and simplification

       ● Model simplification




WB1440 Engineering Optimization – Concepts and Applications
                            Model simplification
       ● Basic idea:

                                                              Expensive
                                                               Cheap
                                                                model



                                                              Optimizer

       ● Motivation:

             – Replacement of expensive function, evaluated many times

             – Interaction between different disciplines

             – Estimation of derivatives

             – Noise


WB1440 Engineering Optimization – Concepts and Applications
                       Model simplification (2)
       ● Drawback: loss of accuracy

       ● Different ranges: local, mid-range, global

       ● Synonyms:

             – Approximation models                           Procedure:
             – Metamodels

             – Surrogate models

             – Compact models                                    Extract      Construct
                                                              information   approximation
             – Reduced order models


WB1440 Engineering Optimization – Concepts and Applications
                       Model simplification (3)
       ● Information extraction: linked to techniques from
           physical experiments: “plan of experiments” / DoE

       ● Many approaches! Covered here:

             – Taylor series expansions
             – Exact fitting

             – Least squares fitting (response surface techniques)

             – Kriging
             – Reduced basis methods
             – Briefly: neural nets, genetic programming, simplified physical models

       ● Crucial: purpose, range and level of detail


WB1440 Engineering Optimization – Concepts and Applications
                    Taylor series expansions
       ● Approximation based on local information:

                                          1             1
                    f ( x  h)  f ( x )  f ' ( x ) h  f ' ' ( x ) h 2  
                                                      1

                                          1!            2!
                                  
                                       1 (n)
                                 f ( x)h n
                                 n  0 n!                          Truncation error!
                                  N                                  N
                                                                          1 (n)
                                 f ( x ) h  o( h )   f ( x ) h
                                       1 (n)       n      N                      n

                                 n  0 n!                           n  0 n!


        ● Use of derivative information!

        ● Valid in neighbourhood of x

WB1440 Engineering Optimization – Concepts and Applications
            Taylor approximation example
                                         5           x cos(x / 5)
                                  f       x / 2
                                                     
                                      1 e          5     3

                                                                    Function
                                                                    Approximation
                                                                    (x = 20)



                                               th20th order
                                             45th order
                                                rdorder
                                                  st order
                                              312nd order
                                                     order


                                                        x

WB1440 Engineering Optimization – Concepts and Applications
                  Exact fitting (interpolation)
       ● # datapoints = # fitting parameters

       ● Every datapoint reproduced exactly

       ● Example:


            f  a0  a1 x                        f2


            1 x1  a0   f1 
            1 x   a    f 
                                                 f1
               2  1     2
                                                          x1   x2
WB1440 Engineering Optimization – Concepts and Applications
                                  Exact fitting (2)
       ● Easy for intrinsically linear functions:
                                                               n
                  f  a0 f 0  a1 f1  a2 f 2     ai f i
                                                              i 1



        ● Often used: polynomials, generalized polynomials:

         f  a  bx1m x2 
                       n
                                             log( f  a )  log b  m log x1  n log x2

        ● No smoothing / filtering / noise reduction

        ● Danger of oscillations with high-order polynomials

WB1440 Engineering Optimization – Concepts and Applications
                                       Oscillations             9th order
                                                                polynomial

       ● Referred to as “Runge phenomenon”

                                                        1
                  5th order
                  9th order                         1  25x 2




        ● In practice: use order 6 or less

WB1440 Engineering Optimization – Concepts and Applications
                                Least squares fitting
       ● Less fitting parameters than datapoints

       ● Smoothing / filtering behaviour

       ● “Best fit”? Minimize sum of deviations:
                                     squared deviations:

                                              
                      N
                                 ~
             min  | f ( xii )  f ( xi ) |
                                          2

               a
                     i 1                                     f

                                                                  ~
                                                                  f

                                                                      x

WB1440 Engineering Optimization – Concepts and Applications
                      Least squares fitting (2)
       ● Choose fitting function linear in parameters ai :
                 ~
                 f ( x)  a0 f0 ( x)  a1 f1 ( x)  a2 f 2 ( x)   am f m ( x)  
     ~
    f ( x0 )   f 0 ( x0 )              f1 ( x0 )     f 2 ( x0 )      f m ( x0 )   a0    0 
   ~          
    f ( x1 )   f 0 ( x1 )              f1 ( x1 )     f 2 ( x1 )       f m ( x1 )   a1   1 
                                                                                        
   ~                                                                                    
   f ( x2 )    f 0 ( x2 )            f1 ( x2 )     f 2 ( x2 )       f m ( x2 )    a2     2 
                                                                         
                                                                                        
                                                                                                   
   ~                                                                                   
    f ( x N )  f 0 ( x N )
                                      f1 ( xN )      f 2 ( xN )      f m ( x N ) am   N 
                                                                                        
                                                 ~
        ● Short notation:                        f  Ma  ε

WB1440 Engineering Optimization – Concepts and Applications
                                         LS fitting (3)
       ● Minimize sum of squared errors:


                         a
                                     ~
                       min L  ε ε  f  Ma
                                         T
                                                             
                                                              T   ~
                                                                  f  Ma   
           (Optimization problem!)


                  L
                  a
                              T ~          T~
                                             
                      0  2M f  Ma  2M f  2MT Ma 
                  M Ma  M f
                     T     ~         T
                                                                          ~
                                                                   a M M M f  T
                                                                                   
                                                                                   1   T




WB1440 Engineering Optimization – Concepts and Applications
                          Polynomial LS fitting
       ● Polynomial of degree m:
                           ~
                           f ( x)  a0 x0  a1x1  a2 x2   am xm  
               ~                                                      ~
        N  f ( x0 ) xi 1 0 xi 2 x0 2 x0i m  a00    0 f i 
                     2     x                x       a       
           ~                                   m 
                                                m 1       
                                                               ~ 
                                             xm2  a   1 ~ 2 
         xi f ( x i 1 1 xi x1   xi1  a11    f i xi 
                                   3   2
                   1) x      x                               
         x2 ~ ( x  1  x 4 x 2
         f 2) i    x
                         3
                              x2 i 2
                                                             
                                             xi2   a2     i i
                                              x
                                                  m 
                                                      a 2    f x 
               
            i                                                     2

                                                      
                                                   
        x  m~        m 1                    2m 
                                                         
                                                         ~ m 
                      xx
         f ( xm )xi 1  mi xm   xm am   m f i xi 
                                              xi   am  
                                 m2 2            m
                                                         
                                                                     
            
            i         


WB1440 Engineering Optimization – Concepts and Applications
                      Polynomial LS example
                     1.2
                                                                           samples
                                                                             quadratic
                      1
                                                                             6th degree


                     0.8


                     0.6


                     0.4


                     0.2


                      0


                    -0.2
                       -1   -0.8   -0.6   -0.4   -0.2   0     0.2   0.4   0.6   0.8       1


WB1440 Engineering Optimization – Concepts and Applications
                  Multidimensional LS fitting
       ● Polynomial in multiple dimensions:
                      ~
                      f ( x, y )  a0  a1 x  a2 y  
                                  a3 x 2  a4 y 2  a5 xy  
                                  a6 x 3  a7 y 3  a8 x 2 y  a9 xy 2    
                                    ai f i  
        ● Number of coefficients ai for quadratic polynomial in Rn:
                          1
                       m  (n  1)(n  2)
                          2
           Curse of dimensionality!
WB1440 Engineering Optimization – Concepts and Applications
                              Response surface
       ● Generate datapoints through sampling:

             – Generate design points through Design of Experiments

             – Evaluate responses
                                                          x3
        ● Fit analytical model

        ● Check accuracy
                                                                               x2



                                                                      x1
                                                                  2n full factorial design
                                                               Fractional factorial design
WB1440 Engineering Optimization – Concepts and Applications
         Latin Hypercube Sampling (LHS)
       ● Popular method: LHS

       ● Based on idea of Latin square:

       ● Properties:                                           1




             – Space-filling                                  0.8




             – Any number of design points
                                                              0.6



                                                              0.4
             – Intended for box-like domains
                                                              0.2

             – Matlab: lhsdesign
                                                               0
                                                                    0   0.2   0.4   0.6   0.8   1




WB1440 Engineering Optimization – Concepts and Applications
                    (LS) Fit quality indicators
      ● Accuracy? More / fewer terms?

      ● Examine the residuals
                                                                    
                                                                    0
            – Small

            – Random!
                                                                                                          xi
      ● Statistical quality indicators:
                                                                              i 2
            – R2 correlation measure:                         R2  1
                                                                                        
                                                                                 ~            2       Okay: >0.6
                                                                                 fi  f


            – F-ratio (signal to noise):                      F
                                                                      ~  f 
                                                                     f          i
                                                                                          2
                                                                                                  m
                                                                                                      Okay: >>1
                                                                      N  m  1
                                                                             2
                                                                        i



WB1440 Engineering Optimization – Concepts and Applications
                                     Nonlinear LS
       ● Linear LS: intrinsically linear functions (linear in ai):

                                f ( x)  a0 x 0  a1 x1  a2 x 2    aT p

                                f ( x)  a0e x  a1 log x  a2 x    aT p

        ● Nonlinear LS: more complicated functions of ai:

                                                a0 x
                                 f ( x) 
                                          a1 x  a2 x 2  1
        ● More difficult to fit! (Nonlinear optimization problem)

        ● Matlab: lsqnonlin

WB1440 Engineering Optimization – Concepts and Applications
                                          LS pitfalls
       ● Scattered data:
                                                       f




                                                              x
       ● Wrong choice of
           basis functions:                            f




                                                              x

WB1440 Engineering Optimization – Concepts and Applications
                                              Kriging
       ● Named after D.C. Krige, mining engineer, 1951

       ● Statistical approach: correlation between neighbouring
           points
             – Interpolation by weighted sum:
                             N
                  y ( x)   i ( x, xi ) yi
                            i 1
             – Weights depend on distance

             – Certain spatial correlation
                function is assumed
                (usually Gaussian)


WB1440 Engineering Optimization – Concepts and Applications
                              Kriging properties
       ● Kriging interpolation is “most likely” in some sense
           (based on assumptions of the method)
       ● Interpolation: no smoothing / filtering

       ● Many variations exist!

       ● Advantage: no need to assume form of
           interpolation function
       ● Fitting process more elaborate than LS procedure



WB1440 Engineering Optimization – Concepts and Applications
                                 Kriging example
       ● Results depend strongly on statistical assumptions and
           method used:




               Dataset z(x,y)                                 Kriging interpolation

WB1440 Engineering Optimization – Concepts and Applications
                         Reduced order model
       ● Idea: describing system in reduced basis:

             – Example: structural dynamics
                                                              ~    ~    ~
                         Mu  Ku  f 
                                                            Mw  Kw  f
        ● Select small number of “modes” to build basis

             – Example: eigenmodes




WB1440 Engineering Optimization – Concepts and Applications
                    Reduced order model (2)
                                                 k  N
       ● Reduced basis:                    u     ω w
                                                   i 1
                                                          i   i



                    B  ω1 ω 2  ω k                                 u  Bw
                                                                      N1   Nk   k1
      ● Reduced system equations:

              Mu  Ku  f
                                                               ~
                                                                M  BT MB        kN   NN   Nk
              MB w  KBw  f
                                                             ~
                                                                K  B KB
                                                                        T

              BT MB w  BT KBw  BT f
                                                             ~
                                                                 f  BT f        kN   N1
                                                                u  Bw
                                                                

WB1440 Engineering Optimization – Concepts and Applications
                       Reduced order models
       ● Many approaches!

             – Selection of type and number of basis vectors

             – Dealing with nonlinearity / multiple disciplines

       ● Active research topic

       ● No interpolation / fitting, but approximate modeling




WB1440 Engineering Optimization – Concepts and Applications
      Example:Aircraft model

                                                              Structural model




                                                              Mass model




                                                              Aerodynamic model




WB1440 Engineering Optimization – Concepts and Applications
                                       Neural nets




WB1440 Engineering Optimization – Concepts and Applications
                                       Neural nets



                                   x                                           f(x)



                   output




                  S(input)
                                           To determine internal neuron parameters,
                                           neural nets must be trained on data.

WB1440 Engineering Optimization – Concepts and Applications
                            Neural net features
       ● Versatile, can capture complex behavior

       ● Filtering, smoothing

       ● Many variations possible

             – Network

             – Number of neurons, layers

             – Transfer functions

       ● Many training steps might be required
           (nonlinear optimization)
       ● Matlab: see e.g. nndtoc
WB1440 Engineering Optimization – Concepts and Applications
                         Genetic programming
       ● Building mathematical functions using
           evolution-like approach

       ● Approach good fit by crossover and
           mutation of expressions

                                                  ^2
                          2
             x1   
              x3 
            x     
                                                  +
             2                            /           x3

                                      x1        x2


WB1440 Engineering Optimization – Concepts and Applications
                         Genetic programming
       ● LS fitting with population of analytic expressions

       ● Selection / evolution rules

       ● Features:

             – Can capture very complex
                behavior
             – Danger of artifacts /
                overfitting
             – Quite expensive procedure

WB1440 Engineering Optimization – Concepts and Applications
                  Simplified physical models
       ● Goal: capture trends from underlying physics through
           simpler model:
             – Lumped / Analytic / Coarse

       ● Parameters fitted to “high-fidelity” data


        ● Refinement: correction function, parameter functions ...


                 x              Simplified                    Correction   f(x)
                                  model                        function


WB1440 Engineering Optimization – Concepts and Applications
             Model simplification summary
       Many different approaches:
             – Local: Taylor series (needs derivatives)

             – Interpolation (exact fit):

                   (Polynomial) fitting

                   Kriging

             – Fitting: LS

             – Approximate modeling: reduced order / simplified models

             – Other: genetic programming, neural nets, etc


WB1440 Engineering Optimization – Concepts and Applications
       Response surfaces in optimization
       ●     Popular approach for
             computationally expen-
             sive problems:
                                                              Expensive
                                                               Cheap
                                                                model
             1. DoE, generate samples
                  (expensive) in part of domain
                                                              Optimizer
             2. Build response surface (cheap)

             3. Perform optimization on response surface (cheap)

             4. Update domain of interest, and repeat

       ●     Additional advantage: smoothens noisy responses

       ●     Easy to combine with parallel computing
WB1440 Engineering Optimization – Concepts and Applications
                        Example: Multi-point
                       Approximation Method
                                                              Design domain

                                                                        Optimum
           (Expensive) simulation


                                                              Sub-optimal point

                                                              Trust region
              Response surface

WB1440 Engineering Optimization – Concepts and Applications

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:2/9/2012
language:
pages:37