Lecture Numerical Errors by liaoqinmei

VIEWS: 3 PAGES: 30

									Lecture 17 - Eigen-analysis

         CVEN 302
       October 3, 2001
           Lecture’s Goals

– QR Factorization
– Householder
– Hessenberg Method
          QR Factorization
The technique can be used to find the
 eigenvalue using a successive iteration
 using Householder transformation to find an
 equivalent matrix to [A] having an
 eigenvalues on the diagonal
        QR factorization
• Another form of factorization

•            A = Q*R
• Produces an orthogonal matrix (“Q”) and a
  right upper triangular matrix (“R”)
• Orthogonal matrix - inverse is transpose

                 1
             Q        Q   T
         QR factorization
  Why do we care?
  We can use Q and R to find eigenvalues
  1. Get Q and R (A = Q*R)
  2. Let A = R*Q
  3. Diagonal elements of A are eigenvalue
     approximations
  4. Iterate until converged
Note: QR eigenvalue method gives all eigenvalues
      simultaneously, not just the dominant 
   QR Eigenvalue Method
In practice, QR factorization on any given
matrix requires a number of steps
First transform A into Hessenberg form
   x x x x x x x x 
   x x x x x x x x 
                   
    x x x x x x x
   
        x x x x x x
                       Hessenberg matrix -
 M
   
   
          x x x x x
                    
                        upper triangular plus
           x x x x
             x x x
                        first subdiagonal
                   
   
               x x

Special properties of Hessenberg matrix
make it easier to find Q, R, and eigenvalues
          QR Factorization
• Construction of QR Factorization
                     A  QR
                         1
     Q : orthogonal Q  Q ; Q Q  I
                                T     T


     R : upper triangular matrix

• Use Householder Reflections and Given Rotations to
  reduce certain elements of a vector to zero
• Use QR factorization that preserve the eigenvalues
• The eigenvalues of the transformed matrix are much
  easier to obtain
   Jordon Canonical Form
• Any square matrix is orthogonally
  similar to a triangular matrix with the
  eigenvalues on the diagonal

J 1     
  
            1    1              
                 2   1          
                                 
J r                           an n  n m atrix
                                 
                         n1  1
           
                              n 
                                  
Similarity Transformation
 • Transformation of the matrix A of the form
   H-1AH is known as similarity transformation
 • A real matrix Q is orthogonal if QTQ = I
 • If Q is orthogonal, then A and Q -1AQ are said to
   be orthogonally similar
               If Ax  x , then
              Ax  AQQ 1 x  x
        Q 1 AQQ 1 x  Q 1 Ax  Q 1 x
 • The eigenvalues are preserved under the
   similarity transformation
    Upper Triangular Matrix
    • The diagonal elements Rii of the upper
      triangular matrix R are the eigenvalues
             r11    r12         r1 n 
                              r13
            0       r22 r23  r2 n 
                                      
          R0
            
                      0 r33  r3 n 
                                                               i  rii
                           
            0
                     0    0  rnn    
                         r11     r12              r13            r1 n
                                0      r22        r23            r2 n
Det  A  I   A  I        0         0       r33            r3 n
                                                              
                                0         0          0       rnn  
                            ( r11   )(r22   )(r33   ) ( rnn   )  0
     Householder Reflector
• Householder reflector is a matrix of the form

             Q  I  2 ww     T


             w w  w 2 1
              T           2



• It is straightforward to verify that Q is
  symmetric and orthogonal

         
  Q  I  2 ww
    T
                     
                   T T
                          I  2 ww  Q
                                    T


  Q T Q  I  4 ww T  4 ww T ww T  I
      Householder Matrix
  • Householder matrix reduces zk+1 ,…,zn to zero

                                      v
                 H  I  2 ww  ; w 
                                      v2
      x  x 1     x2      xk         x k 1  x n  
     y  Hx   y 1        y2          yk    0  0 

  • To achieve the above operation, v must be a
    linear combination of x and ek

                   ek    0,0,...,0,1,0....,0 T
v  x   e k  x 1 , x 2 ,, x k 1 , x k   , x k  1 ,, x n 
    Householder Transformation
                vv           2v x 
   Hx   I  2       x  x         v
                v v          v v 
    x  x 1 x 2  x k x k  1  x n  
   
   v   x 1 x 2  x k  α x k  1  x n    x   e k
   v v  x 1  x 2    ( x k   ) 2    x n  x 2  2x k   2
              2     2                             2
                                                       2
   
     v x  x 1  x 2    ( x k   ) x k    x n  x 2  x k
                                                          2
   
              2     2                               2
   
                                                  2 x 2  2xk
                                                      2
                                         2vx
       If we choose    x                   2                    1
                                   2
                                         vv   x 2  2xk  x
                                                                 2
                                                                 2

                  2vx 
       Hx  x        v  x  v  0  0  α 0  0
                  vv 
choose α  sign(xk ) x         2
                                       to avoid cancellation of xk  α
       Householder matrix
• Corollary (kth Householder matrix): Let
  A be an nxn matrix and x any vector. If k is an
  integer with 1< k<n-1 we can construct a
  vector w(k) and matrix H(k) = I - 2w(k)w’(k) so
  that
                            x1   x1 
                                
                                    
                            xk 1   xk 
                      (k )          
           H x  H  xk       y
              (k )

                           x   0 
                            k 1   
                                
                           x   
                            n   0 
       Householder matrix
• Define the value  so that                  x   2
                                                       y    2
    2  x k  x k 1    x n 
           2     2            2

       2
   x   2
            x 1  x 2    x k 1  x k  x k 1    x n
               2     2         2        2     2            2



                                         y
                                                      2
            x  x  x
               2
               1
                       2
                       2
                                2
                                k 1
                                         2
                                                      2
                                                           g2
• The vector w is found by
    x y
           0  0 ( x k   ) x k  1  x n  
           1
 w
    x y 2 s
 s 2   x k     x k 1    x n
                   2   2            2
                                              ww   1
     2 x k   2  x k  x k  1    x n  2 x k  1  2 2
                       2     2             2



• Choose  = sign(xk)g to reduce round-off error
            Householder Matrices
 Step 1 : set w 1  w 2    w k  1  0
 Step 2 : com pute g           x k  x k 1    x n
                                  2     2            2


                         sign x k  g
 Step 3 : com pute s   x k     x k  1    x n
                         2             2   2         2


                              g 2  2x k   2  g 2  2x k  g 2
                         s  2 g 2  2 x k  2 g g  x k          
 Step 4 : set w k   x k    /s   x k  sign x k g  /s ;
                wi  xi / s ,       i  k  1 , , n

                   x k  sign x k g     x k 1        xn 
                                                             ; w w  w
                                                                           2
w  0 ,  , 0 ,                       ,          , ,                         1
                                                         s               2
                            s                s
 Example: Householder matrix
               1   2  1                 1
           A  2
                   2  1 ; x  A:,1   2 
                                          
               2  1
                       2
                                         2
                                           
g   x1  x 2  x 3  12  2 2  2 2  3;
      2     2     2
                                              sign x 1 g  3
      s  2 g( g  x 1 )  2( 3 )( 3  1 )  24
     
     
      w  w(1) 
                       1
                           1  3 2 2  1 4 2 2
                      24                   24
                              1 0 0       4 
      H ( 1 )  I  2 ww    0 1 0  
                                         1  
                                     24  2  4 2 2 
                              0 0 1
                                          2
                                             
                           1/3  2/3  2/3
                          2/3
                                  2/3  1/3
                                            
                           2/3  1/3
                                        2/3
                                            
  Example: Householder matrix
              1/3  2/3  2/3  1   2  1   3  4/3  1/3
H ( 1 ) A    2/3
                     2/3  1/3  2
                                     2  1   0
                                                    1/3  2/3
                                                               
              2/3  1/3
                           2/3  2  1
                                         2   0  8/3
                                                          7/3
                                                               
g  (1/3)2  ( 8/3)2  65 /3;   sign x 2  g  65 /3;
s  2 g ( g  x 2 )  4.0294
                                        
              1    1  65            
                                    0 0.7497  0.6618
                                    8
w  w( 2 )    0
              s       3            3
                      1    0   0          0
H(2)    I  2 ww    0
                           1   0    0.7497  0 0.7497  0.6618
                                             
                      0
                           0   1   0.6618
                                             
                      1           0      0
                     0
                            0.1240 0.9923
                                           
                      0
                             0.9923 0.1240
                                           
    Basic QR Factorization
•    [A] = [Q] [R]
•    [Q] is orthogonal, QTQ = I
•    [R] is upper triangular
•    QR factorization using Householder matrices
•    Q = H(1)H(2)….H(n-1)

Define R ( 0 )  A                              Define Q  I
    for k  1 : n  1                           for k  n - 1 : -1 : 1
     Find H ( k ) with positions k  1 , , n
                                                  Q  H (k)Q
                 in R (k - 1) reduce to zero
                                     ( k 1 )
                                                end
  Define R     (k)
                     H   (k )
                                 R
end
                                                Define R  R (n-1)
Example: QR Factorization
                1     2  1
  R( 0 )  A  2
                      2  1
                            
                2  1
                          2
                            
  H ( 1 )  I  2 ww 
  R( 1 )  H ( 1 ) R( 0 )  H ( 1 ) A
  H ( 2 )  I  2 ww'
  R( 2 )  H ( 2 ) R( 1 )  H ( 2 ) H ( 1 ) A
  Q  H ( 1 ) H ( 2 )
  
   R  R( 2 )
          
  QR  H ( 1 ) H ( 2 )    H   (2)
                                      H (1)A  A
           QR Factorization
                         3     1.3333  0.3333
R  H ( 2 )H (1) A   0        2.6874   2.3980
                                                 
                         0
                                      0  0.3721
                                                 
                       2/3  2/3  1
                        1/3                0      0
                        2/3  1/3 0  0.1240 0.9923
Q  H ( 1 ) H ( 2 )    2/3
                                                  
                        2/3
                       1/3
                               2/3 0
                                      0.9923 0.1240
                                                     
    0.3333  0.5788  0.7442
   0.6667  0.4134   0.6202
  
    0.6667
                              
               0.7029  0.2481
                                             QR = A
                             

• Similarity transformation B = QTAQ
  preserve the eigenvalues  B   A
     Finding Eigenvalues Using
          QR Factorization
• Generate a sequence A(m) that are
  orthogonally similar to A
• Use Householder transformation H-1AH
• the iterates converge to an upper triangular
  matrix with the eigenvalues on the
  diagonal
                A (k)  Q ( k ) R ( k )
    In general  ( k  1 )
               A          R Q (k ) (k )



   Find all eigenvalues simultaneously!
     QR Eigenvalue Method
    A( 1 )  Q ( 1 ) R ( 1 )
    (2)                                 A  QR
   A  R Q
                (1) (1)

                                               1
   A  Q R
       (2)       (2)    (2)     RQ AQ A                 T

    (3)                                  ( k 1 )
    A  R ( 2 )Q ( 2 )               A               RQ
                                    ( k 1 )
   A  Q R
       (3)       (3)    (3)
                                A              Q A  T   (k )
                                                                Q
    (4)
    A  R ( 3 )Q ( 3 )

• QR factorization: A = QR
• Similarity transformation: A(new) = RQ
    Example: QR Eigenvalue
              1      2  1
A  A(0 )    2
                     2  1  QR
                           
              2  1
                         2
                           
                 0.3333  0.5788  0.7442   3  1.3333  0.3333
               0.6667  0.4134
                                     0.6202  0  2.6874
                                                            2.3980
                                                                    
                 0.6667
                            0.7029  0.2481  0
                                                        0  0.3721
                                                                    

                 3  1.3333  0.3333   0.3333  0.5788  0.7442
A( 1)    RQ   0  2.6874
                                2.3980   0.6667  0.4134
                                                             0.6202
                                                                     
                0
                           0  0.3721   0.6667
                                                    0.7029  0.2481
                                                                     
                 2.1111   2.0535    1.4884
              0.1929
                          2.7966  2.2615  
                 0.2481  0.2615
                                    0.0923 
            Example: QR Eigenvalue
            2.4636   1.8104  1.3865            2.4140   1.8600   1.3933
A( 2 )      0.0310
                     3.0527   1.7694 A ( 5 )   0.0002
                                                          2.9996  1.8982
            0.0616  0.1047  0.5161
                                                0.0003  0.0007  0.4136 
                                                                           

            2.4056   1.8691   1.3930                2.4143   1.8596  1.3934
A( 3 )     0.0056
                     2.9892  1.9203
                                         A( 6 )    0.0000
                                                               3.0001   1.8974
                                                                               
            0.0099  0.0191  0.3948
                                                    0.0001  0.0001  0.4143
                                                                              

            2.4157   1.8579  1.3937            2.4142 1.8597   1.3934
A( 4 )      0.0010
                     3.0021   1.8930 A ( 7 )   0.0000 3.0000  1.8974
                                                                       
            0.0017  0.0038  0.4178
                                                0.0000 0.0000  0.4142
                                                                        

          1  2.4142 ,  2  3.0000 ,  3  0.4142
 MATLAB Example
                                       A =
                                              2.4634    1.8104   -1.3865
                                             -0.0310    3.0527    1.7694
» A=[1 2 -1; 2 2 -1; 2 -1 2]
                                              0.0616   -0.1047   -0.5161
A =                                    A =
     1     2    -1                            2.4056    1.8691    1.3930
     2     2    -1                            0.0056    2.9892   -1.9203
     2    -1     2                            0.0099   -0.0191   -0.3948
                            QR         A =
» [Q,R]=QR_factor(A)                          2.4157    1.8579   -1.3937
                       factorization         -0.0010    3.0021    1.8930
Q =
                                              0.0017   -0.0038   -0.4178
    -0.3333  -0.5788     -0.7442       A =
    -0.6667  -0.4134      0.6202              2.4140    1.8600    1.3933
    -0.6667   0.7029     -0.2481              0.0002    2.9996   -1.8982
R =                                           0.0003   -0.0007   -0.4136
    -3.0000  -1.3333     -0.3333       A =
     0.0000  -2.6874      2.3980              2.4143    1.8596   -1.3934
     0.0000   0.0000     -0.3721              0.0000    3.0001    1.8972
                                              0.0001   -0.0001   -0.4143
» e=QR_eig(A,6);     eigenvalue        e =
A =                                           2.4143
    2.1111    2.0535     1.4884               3.0001
    0.1929    2.7966    -2.2615              -0.4143
    0.2481   -0.2615     0.0923
    Improved QR Method
• Using similarity transformation to form an upper
  Hessenberg Matrix (upper triangular matrix &
  one nonzero band below diagonal)
• More efficient to form Hessenberg matrix
  without explicitly forming the Householder
  matrices (not given in textbook)

   function A = Hessenberg(A)
   [n,nn] = size(A);
   for k = 1:n-2
        H = Householder(A(:,k),k+1);
        A = H*A*H;
   end
Improved QR Method                  A =
                                           2.4056   -2.1327    0.9410
» A=[1 2 -1; 2 2 -1; 2 -1 2]              -0.0114   -0.4056   -1.9012
A =                                        0.0000    0.0000    3.0000
      1      2    -1                A =
      2      2    -1                       2.4157    2.1194   -0.9500
      2     -1     2                      -0.0020   -0.4157   -1.8967
» [Q,R]=QR_factor_g(A) Hessenberg          0.0000    0.0000    3.0000
Q =                       matrix    A =
     0.4472     0.5963   -0.6667           2.4140   -2.1217    0.9485
     0.8944    -0.2981    0.3333          -0.0003   -0.4140   -1.8975
          0    -0.7454   -0.6667           0.0000    0.0000    3.0000
R =                                 A =
     2.2361     2.6833   -1.3416           2.4143    2.1213   -0.9487
    -1.4907     1.3416   -1.7889          -0.0001   -0.4143   -1.8973
    -1.3333          0   -1.0000           0.0000    0.0000    3.0000
» e=QR_eig_g(A,6);     eigenvalue   e =
A =                                     2.4143
     2.1111    -2.4356    0.7071       -0.4143
    -0.3143    -0.1111   -2.0000        3.0000
          0     0.0000    3.0000    » eig(A)
A =                                 ans =               MATLAB
     2.4634     2.0523   -0.9939        2.4142          function
    -0.0690    -0.4634   -1.8741       -0.4142
     0.0000     0.0000    3.0000        3.0000
                Summary

• QR Factorization
  – Householder matrix
  – Hessenberg matrix
             Homework
• Check the Homework webpage

								
To top