Lecture 18 - Numerical Differentiation

							Lecture 18 - Numerical
    Differentiation
       CVEN 302
      July 13, 2001
               Lecture’s Goals
• Finite Difference Schemes
• Taylor Series Expansion for Differentiation
• Basic Numerical Integration
  –   Trapezoidal Rule
  –   Simpson’s Rule
  –   Midpoint
  –   Gaussian Quadrature
Calculus - Numerical Methods
Differentiation              Integration

v                       v


            dv
       dt
                    t                      t
a                       y
                                b
               dv           y   vdt
            a                  a
               dt
                    t
     Numerical differentiation
• Estimate the derivatives (slope, curvature, etc.)
  of a function by using the function values at
  only a set of discrete points
• Ordinary differential equation (ODE)
• Partial differential equation (PDE)
• Represent the function by Taylor polynomials
  or Lagrange interpolation
• Evaluate the derivatives of the interpolation
  polynomial at selected nodal points
           Numerical differentiation
 A Taylor series or Lagrange interpolation of points
can be used to find the derivatives. The Taylor
series expansion is defined as:

f xi   f x0   x
                       df
                                  
                                    x  d 2 f
                                         2
                                                           
                                                             x 3 d 3 f             
                       dx x  x 0     2! dx 2      x x0
                                                               3!    dx   3
                                                                              x x0

  x  xi  x0

f xi   f x0   xi  x0  f x0  
                                          xi  x0 2   f x0  
                                                                    xi  x0 3       f x0   
                                              2!                          3!
First Derivative at a Point

              f ( x )




  i-2   i-1    i         i+1   i+2
           Numerical differentiation
 Use the Taylor series expansion to represent three
 points about single location:


f xi 1   f xi   xi 1    x  f x  
                                                xi1  xi 2   f xi    1
                                   i       i
                                                     2!
  f xi   f xi                                                           2
f xi-1   f xi   xi 1     x  f x  
                                                xi1  xi 2   f xi    3
                                   i       i
                                                     2!
             Numerical differentiation

   Assume that the data points are equally
   spaced and the equations can be written as:

f xi 1   f xi   x  f xi  
                                        x 2   f x  
                                                            x 3   f xi    1
                                                      i
                                         2!                  3!
 f xi   f xi                                                                   2
f xi-1   f xi   x  f xi  
                                       x 2    f x  
                                                            x 3   f xi     3
                                                      i
                                         2!                   3!
                Forward differentiation
For a forward first derivative, subtract eqn[2] from eqn[1]:

  f  xi 1   f  xi   x  f  xi  
                                             x  2
                                                         f  x  
                                                                     x  3
                                                                               f  xi   
                                                                i
                                              2!                      3!
Rearrange the equation:

x  f xi   f xi1     f x   x 2       f  x  
                                                                 x 3      f  xi   
                                    i                       i
                                            2!                      3!

                f xi 1   f xi   x          x f x   
                                                     2                     3
    f xi                               f xi  
                         x
                                                                    i
                                        2!              3!
        Forward differentiation

As the x gets smaller the error will get smaller
                  f  xi 1   f  xi  
     f  xi                             Error
                           x            
The error is defined as:


   Error 
               x  f x   x    2
                                            f  xi   
                           i
                2!                 3!
      Backward differentiation

Subtract eqn[3] from eqn[2]:
                   f  xi   f  xi-1  
      f  xi                            Error
                           x            
The error is defined as:

    Error 
                x  f x   x     2
                                              f  xi   
                             i
                  2!                 3!
        Central differentiation

Subtract eqn[3] from eqn[1]:
                   f  xi 1   f  xi-1  
      f  xi                               Error
                            2x             
The error is defined as:

            Error 
                         x    2
                                     f  xi   
                            3!
            Differential Error

Notice that the errors of the forward and backward 1st
derivative of the equations have an error of the order
of O(x) and the central differentiation has an error
of order O(x2). The central difference has an better
accuracy and lower error that the others. This can be
improved by using more terms to model the first
derivative.
              Numerical differentiation
  If you want to improve the accuracy and
  decrease the error you will need to eliminate
  terms :
  A * f xi   f xi 

B * f  xi-1   f xi     x  f xi    
                                                   x 2   f xi  
                                                                          x 3   f xi   
                                                   2!                  3!
                                                4x               8x 
                                                      2                   3
C * f xi-2   f xi   2x  f  xi              f xi           f xi   
                                                  2!                  3!

                  =                                    =
                  0              0                      0
            Higher Order Errors in
               Differentiation
The terms become :
                           A B C  0
                            B  2C  # 2 
                               B  4C  0
  The terms become A=-3, B= 4 and C=-1

              3 f  xi   4 f  xi-1   f  xi-2  
f  xi   
                              2x
                                                          
                                                          E O x 2
                                                      
          Higher order 1st derivative
                                          f ( x )
                                                                   Parabolic curve




                 i-2     i-1                i                i+1               i+2

                                    3 f  xi  2   4 f  xi 1   f xi  
Forward difference     f  xi   
                                                      2x
                                                                                  
                                                                                E O x
                                                                                         2

                                                                             
                                     3 f  xi   4 f  xi-1   f  xi-2  
Backward difference    f  xi   
                                                     2x
                                                                                   
                                                                                E O x
                                                                                         2

                                                                             
     Higher Order Derivatives

To find higher derivatives, use the Taylor series
expansions of term and eliminate the terms from the
sum of equations. To improve the error in the
problem add additional terms.
         2nd Derivative of the function
 It will require three terms to get a central 2nd derivative
 of discrete set of data.

A * f xi 1   f xi     x  f xi    
                                                  x 2     f xi    
                                                                             x 3   f xi   
                                                    2!                         3!
 B * f xi   f xi 

C * f xi-1   f xi      x  f xi    
                                                  x 2     f xi    
                                                                             x 3   f xi   
                                                    2!                         3!

                 =                =                      =
                 0                 0                     #
     2nd order Central Difference
The terms become :
                  A B C  0
                       AC  0
                       A  C  # 2
The terms become A=1,B=-2 and C=1. Therefore


              f  xi 1   2 f xi   f  xi-1  
f  xi   
                              x
                                                        
                                                      E O x
                                                               2

                                                   
                                  2
             Lagrange Differentiation
Another form of differentiation is to use the
Langrange interpolation between three points. The
values can be determine for unevenly spaced points.
Given:

Lx   L1 x  y1  L2 x  y2  L3 x  y3

      
         x  x2 x  x3  y  x  x1 x  x3  y  x  x1 x  x2  y
        x1  x2 x1  x3  1 x2  x1 x2  x3  2 x3  x2 x3  x1  3
         Lagrange Differentiation
Differentiate the Lagrange interpolation
                           2 x  x2  x3
     f x   Lx                          y1
                         x1  x2 x1  x3 
                 2 x  x1  x3             2 x  x1  x2
                                  y2                       y3
              x2  x1 x2  x3       x3  x2 x3  x1 
Assume a constant spacing
            2 x  x2  x3      2 x  x1  x3      2 x  x1  x2
  f x                 y1                y2                y3
                2x  2
                                   x  2
                                                      2x  2
               Lagrange Differentiation
Differentiate the Lagrange interpolation
                2 x  x2  x3      2 x  x1  x3      2 x  x1  x2
    f x                   y1                y2                y3
                    2x  2
                                       x  2
                                                          2x  2


Various locations
           2x  x  x   2x  x  x   2x  x  x                      3 y1  4 y2  y3
 f x1   1 2 2 3 y1  1 1 2 3 y2  1 1 2 2                y3 
                 2x               x              2x                    2x
              2 x2  x2  x3     2x  x  x   2x  x  x  y y
 f x2                    y1  2 1 2 3 y2  2 12 2 y3  3 1
                   2x 2            x          2x       2x

              2 x3  x2  x3     2x  x  x   2x  x  x   y  4 y2  3 y3
 f x3                    y1  3 1 2 3 y2  3 1 2 2 y3  1
                   2x 2            x          2x           2x
         Lagrange Differentiation
To find a higher order derivative from the Lagrange
interpolation for a three point Lagrange

            2 x  x2  x3      2 x  x1  x3      2 x  x1  x2
  f x                 y1                y2                y3
                2x  2
                                   x  2
                                                      2x  2



 Take the derivative

                                      y1  2 y2  y3
   f x   2 y1 
              1        2       1
                           y  2 y3 
             x       x 2 2
                              x           x 2
     Partial Derivatives
• Straightforward extension of one-
  dimensional formula
                          (i, j+2)


               (i-1, j+1) (i, j+1)    (i+1, j+1)


    (i-2, j)   (i-1, j)   (i, j)      (i+1, j)     (i+2, j)


               (i-1, j-1) (i, j-1)    (i+1, j-1)


                           (i, j-2)
       u
ux        
              1
                  1  0  1 
        x 2h
         2u
              2  1  2  1                  
                 1
u xx 
        x 2
                h
                      (i, j+2)


           (i-1, j+1) (i, j+1)    (i+1, j+1)


(i-2, j)   (i-1, j)   (i, j)      (i+1, j)         (i+2, j)


           (i-1, j-1) (i, j-1)    (i+1, j-1)


                       (i, j-2)
Partial Derivatives
   Laplacian Operator
 u  u xx  u yy 
 2


          1             j+1
          |            
 1                    
                        
  2 
      1  4  1         j
h                      
           |
                       
    
          1             j-1
                        
    i-1     i     i+1
       Partial Derivatives
          Mixed Derivative


               1  0  1  j+1
              |     |      
                          | 
          1                j
u xy      2 
                 0  0  0 
         4h 
                |    |    | 
                           
              1  0   1  j-1
                           
             i-1      i      i+1
Bi-harmonic Operator
       4u       2u     4u
 4u       2               
       x 4
               x y
                 2   2
                         x 4


                 1            j+2
                 |           
                             
           2  8  2         j+1
                             
           |     |    |      
  1                          
   4 
       1   8  20   8  1  j
 4h                          
            |     |    |
                             
           2  8  2         j-1
                             
                 |           
     
                 1            j-2
                              
      i-2    i-1      i     i+1   i+2
            Richardson Extrapolation
      This technique uses the concept of variable grid
      sizes to reduce the error. The technique uses a
      simple method for eliminating the error. Consider a
      second order central difference technique. Write the
      equation in the form:

              f  xi 1   2 f xi   f  xi-1  
f  xi                                           a1x  a 2 x  
                                                            2        4

                             x 2                  
        Richardson Extrapolation
   The central difference can be defined as

              f  xi 1   2 f xi   f  xi-1  
f  xi                                           a1x  a 2 x  
                                                            2        4

                             x 2                  
  Write the equation with different grid sizes

   A f xi   Ax   a1x 2  a 2 x 4  
                     x        x       x 
                                          2              4

   A  f xi   A      a1       a2      
                     2         2        2 
      Richardson Extrapolation
Expand the terms:

A  f  xi   Ax   a 1x 2  a 2 x 4      1
                   x        x 2      x 4 
A f  xi   A
                             4   a 2  16    2
                        a1                 
                   2                       
    Richardson Extrapolation
Multiply eqn [2] by 4 and subtract eqn [1] from it.
             x          x 2       x 4 
   4A  4 A               4   4a 2  16   
                   4a 1                  
             2                           
    A   Ax         a 1x 2  a 2 x 4  


            x                x 4 
   3A  4 A   Ax   3a 2 
                                16   
                                      
             2                     
     Richardson Extrapolation
 The equation can be rewritten as:
        x           
       4 A   Ax  
          2                x 4 
                         a 
   A                       2
                                     
             3              16  
                       
                       
It can be rewritten in the form

       A  Bx   b1x  b 2 x  
                           4         6
     Richardson Extrapolation
The technique can be extrapolated to include the
higher order error elimination by using a finer grid.


           x        
       16B   Bx  
   A 
      
            2 
               15
                          O x 6  
                        
                                       
                       
                       
       Richardson Extrapolation
               Example
 The function is given:

             f x   x  2 x  4 x  8
                          3         2



Find the first derivative at x=1.25 using a central
difference scheme and h = 0.25.

f  x   3x 2  4 x  4  31.25 2  41.25   4  3.6875
       Richardson Extrapolation
               Example
The data points are:                  x        f(x)
                                      1         -5
                                    1.125    -4.607
                                    1.25     -4.172
                                    1.375    -3.682
                                     1.5     -3.125
  The derivatives using central difference
            f 1.5  f 1.0  3.125  5
f 1.25                                3.75
                20.25           0.5
            f 1.375  f 1.125  3.6816  4.6074
f 1.25                                          3.7032
                  20.125                0.25
        Richardson Extrapolation
                Example
  The results of the central difference scheme are:

            f 1.25   3.75        Error  1.69 %
            f 1.25   3.7032      Error  0.425 %

The Richardson Extrapolation uses these results to find
a better solution
             x 
           A   Ax 
                           43.7032  3.75
  1.25   
              2
f                                           3.6876   Error  0.003%
                  3               3
                Summary

• Finite Difference Techniques
  – Taylor Series
  – Lagrange Polynomials

• Error Calculation
                Summary

• Finite Difference Techniques
  – Forward Difference Scheme
  – Backward Difference Scheme
  – Central Difference Scheme
             Homework
• Check the Homework webpage