# Lecture Numerical Errors by liaoqinmei

VIEWS: 3 PAGES: 30

• pg 1
```									Lecture 17 - Eigen-analysis

CVEN 302
October 3, 2001
Lecture’s Goals

– QR Factorization
– Householder
– Hessenberg Method
QR Factorization
The technique can be used to find the
eigenvalue using a successive iteration
using Householder transformation to find an
equivalent matrix to [A] having an
eigenvalues on the diagonal
QR factorization
• Another form of factorization

•            A = Q*R
• Produces an orthogonal matrix (“Q”) and a
right upper triangular matrix (“R”)
• Orthogonal matrix - inverse is transpose

1
Q        Q   T
QR factorization
Why do we care?
We can use Q and R to find eigenvalues
1. Get Q and R (A = Q*R)
2. Let A = R*Q
3. Diagonal elements of A are eigenvalue
approximations
4. Iterate until converged
Note: QR eigenvalue method gives all eigenvalues
simultaneously, not just the dominant 
QR Eigenvalue Method
In practice, QR factorization on any given
matrix requires a number of steps
First transform A into Hessenberg form
x x x x x x x x 
x x x x x x x x 
                
 x x x x x x x

x x x x x x
   Hessenberg matrix -
M


x x x x x

upper triangular plus
        x x x x
          x x x
first subdiagonal
                

            x x

Special properties of Hessenberg matrix
make it easier to find Q, R, and eigenvalues
QR Factorization
• Construction of QR Factorization
A  QR
1
Q : orthogonal Q  Q ; Q Q  I
T     T

R : upper triangular matrix

• Use Householder Reflections and Given Rotations to
reduce certain elements of a vector to zero
• Use QR factorization that preserve the eigenvalues
• The eigenvalues of the transformed matrix are much
easier to obtain
Jordon Canonical Form
• Any square matrix is orthogonally
similar to a triangular matrix with the
eigenvalues on the diagonal

J 1     

 1    1              
      2   1          
                      
J r                           an n  n m atrix
                      
              n1  1

                   n 

Similarity Transformation
• Transformation of the matrix A of the form
H-1AH is known as similarity transformation
• A real matrix Q is orthogonal if QTQ = I
• If Q is orthogonal, then A and Q -1AQ are said to
be orthogonally similar
If Ax  x , then
Ax  AQQ 1 x  x
Q 1 AQQ 1 x  Q 1 Ax  Q 1 x
• The eigenvalues are preserved under the
similarity transformation
Upper Triangular Matrix
• The diagonal elements Rii of the upper
triangular matrix R are the eigenvalues
 r11    r12         r1 n 
r13
0       r22 r23  r2 n 
                          
R0

0 r33  r3 n 
                        i  rii
               
0
         0    0  rnn    
r11     r12              r13            r1 n
0      r22        r23            r2 n
Det  A  I   A  I        0         0       r33            r3 n
                              
0         0          0       rnn  
 ( r11   )(r22   )(r33   ) ( rnn   )  0
Householder Reflector
• Householder reflector is a matrix of the form

Q  I  2 ww     T

w w  w 2 1
T           2

• It is straightforward to verify that Q is
symmetric and orthogonal


Q  I  2 ww
T

T T
 I  2 ww  Q
T

Q T Q  I  4 ww T  4 ww T ww T  I
Householder Matrix
• Householder matrix reduces zk+1 ,…,zn to zero

v
H  I  2 ww  ; w 
v2
x  x 1     x2      xk         x k 1  x n  
y  Hx   y 1        y2          yk    0  0 

• To achieve the above operation, v must be a
linear combination of x and ek

ek    0,0,...,0,1,0....,0 T
v  x   e k  x 1 , x 2 ,, x k 1 , x k   , x k  1 ,, x n 
Householder Transformation
      vv           2v x 
Hx   I  2       x  x         v
      v v          v v 
 x  x 1 x 2  x k x k  1  x n  

v   x 1 x 2  x k  α x k  1  x n    x   e k
v v  x 1  x 2    ( x k   ) 2    x n  x 2  2x k   2
2     2                             2
                                                    2

v x  x 1  x 2    ( x k   ) x k    x n  x 2  x k
2

2     2                               2

2 x 2  2xk
2
2vx
If we choose    x                   2                    1
2
vv   x 2  2xk  x
2
2

 2vx 
 Hx  x        v  x  v  0  0  α 0  0
 vv 
choose α  sign(xk ) x         2
to avoid cancellation of xk  α
Householder matrix
• Corollary (kth Householder matrix): Let
A be an nxn matrix and x any vector. If k is an
integer with 1< k<n-1 we can construct a
vector w(k) and matrix H(k) = I - 2w(k)w’(k) so
that
 x1   x1 
     
         
 xk 1   xk 
(k )          
H x  H  xk       y
(k )

x   0 
 k 1   
     
x   
 n   0 
Householder matrix
• Define the value  so that                  x   2
 y    2
 2  x k  x k 1    x n 
2     2            2

2
x   2
 x 1  x 2    x k 1  x k  x k 1    x n
2     2         2        2     2            2

  y
2
 x  x  x
2
1
2
2
2
k 1
2
2
 g2
• The vector w is found by
x y
 0  0 ( x k   ) x k  1  x n  
1
w
x y 2 s
s 2   x k     x k 1    x n
2   2            2
 ww   1
 2 x k   2  x k  x k  1    x n  2 x k  1  2 2
2     2             2

• Choose  = sign(xk)g to reduce round-off error
Householder Matrices
Step 1 : set w 1  w 2    w k  1  0
Step 2 : com pute g           x k  x k 1    x n
2     2            2

  sign x k  g
Step 3 : com pute s   x k     x k  1    x n
2             2   2         2

 g 2  2x k   2  g 2  2x k  g 2
s  2 g 2  2 x k  2 g g  x k          
Step 4 : set w k   x k    /s   x k  sign x k g  /s ;
wi  xi / s ,       i  k  1 , , n

               x k  sign x k g     x k 1        xn 
 ; w w  w
2
w  0 ,  , 0 ,                       ,          , ,                         1
                                                     s               2
s                s
Example: Householder matrix
1   2  1                 1
A  2
    2  1 ; x  A:,1   2 
                 
2  1
        2
                2
 
g   x1  x 2  x 3  12  2 2  2 2  3;
2     2     2
  sign x 1 g  3
 s  2 g( g  x 1 )  2( 3 )( 3  1 )  24


 w  w(1) 
1
1  3 2 2  1 4 2 2
                 24                   24
1 0 0       4 
H ( 1 )  I  2 ww    0 1 0  
1  
       24  2  4 2 2 
0 0 1
            2
 
  1/3  2/3  2/3
   2/3
         2/3  1/3

  2/3  1/3
               2/3

Example: Householder matrix
  1/3  2/3  2/3  1   2  1   3  4/3  1/3
H ( 1 ) A    2/3
         2/3  1/3  2
      2  1   0
       1/3  2/3

  2/3  1/3
               2/3  2  1
          2   0  8/3
             7/3

g  (1/3)2  ( 8/3)2  65 /3;   sign x 2  g  65 /3;
s  2 g ( g  x 2 )  4.0294

1    1  65            
   0 0.7497  0.6618
8
w  w( 2 )    0
s       3            3
1    0   0          0
H(2)    I  2 ww    0
     1   0    0.7497  0 0.7497  0.6618
           
0
     0   1   0.6618
           
1           0      0
 0
      0.1240 0.9923

0
       0.9923 0.1240

Basic QR Factorization
•    [A] = [Q] [R]
•    [Q] is orthogonal, QTQ = I
•    [R] is upper triangular
•    QR factorization using Householder matrices
•    Q = H(1)H(2)….H(n-1)

Define R ( 0 )  A                              Define Q  I
for k  1 : n  1                           for k  n - 1 : -1 : 1
Find H ( k ) with positions k  1 , , n
Q  H (k)Q
in R (k - 1) reduce to zero
( k 1 )
end
Define R     (k)
H   (k )
R
end
Define R  R (n-1)
Example: QR Factorization
1     2  1
R( 0 )  A  2
      2  1

2  1
          2

H ( 1 )  I  2 ww 
R( 1 )  H ( 1 ) R( 0 )  H ( 1 ) A
H ( 2 )  I  2 ww'
R( 2 )  H ( 2 ) R( 1 )  H ( 2 ) H ( 1 ) A
Q  H ( 1 ) H ( 2 )

 R  R( 2 )

QR  H ( 1 ) H ( 2 )    H   (2)
H (1)A  A
QR Factorization
 3     1.3333  0.3333
R  H ( 2 )H (1) A   0        2.6874   2.3980

 0
              0  0.3721

 2/3  2/3  1
  1/3                0      0
2/3  1/3 0  0.1240 0.9923
Q  H ( 1 ) H ( 2 )    2/3
                            
  2/3
 1/3
         2/3 0
   0.9923 0.1240

  0.3333  0.5788  0.7442
   0.6667  0.4134   0.6202

  0.6667

0.7029  0.2481
QR = A
                           

• Similarity transformation B = QTAQ
preserve the eigenvalues  B   A
Finding Eigenvalues Using
QR Factorization
• Generate a sequence A(m) that are
orthogonally similar to A
• Use Householder transformation H-1AH
• the iterates converge to an upper triangular
matrix with the eigenvalues on the
diagonal
 A (k)  Q ( k ) R ( k )
In general  ( k  1 )
A          R Q (k ) (k )

Find all eigenvalues simultaneously!
QR Eigenvalue Method
 A( 1 )  Q ( 1 ) R ( 1 )
 (2)                                 A  QR
A  R Q
(1) (1)

1
A  Q R
(2)       (2)    (2)     RQ AQ A                 T

 (3)                                  ( k 1 )
 A  R ( 2 )Q ( 2 )               A               RQ
( k 1 )
A  Q R
(3)       (3)    (3)
A              Q A  T   (k )
Q
 (4)
 A  R ( 3 )Q ( 3 )

• QR factorization: A = QR
• Similarity transformation: A(new) = RQ
Example: QR Eigenvalue
1      2  1
A  A(0 )    2
       2  1  QR

2  1
           2

  0.3333  0.5788  0.7442   3  1.3333  0.3333
   0.6667  0.4134
                      0.6202  0  2.6874
                2.3980

  0.6667
             0.7029  0.2481  0
            0  0.3721


  3  1.3333  0.3333   0.3333  0.5788  0.7442
A( 1)    RQ   0  2.6874
                 2.3980   0.6667  0.4134
                     0.6202

 0
            0  0.3721   0.6667
            0.7029  0.2481

 2.1111   2.0535    1.4884
 0.1929
          2.7966  2.2615  
 0.2481  0.2615
                    0.0923 
Example: QR Eigenvalue
 2.4636   1.8104  1.3865            2.4140   1.8600   1.3933
A( 2 )      0.0310
          3.0527   1.7694 A ( 5 )   0.0002
                     2.9996  1.8982
 0.0616  0.1047  0.5161
                                     0.0003  0.0007  0.4136 
                          

 2.4056   1.8691   1.3930                2.4143   1.8596  1.3934
A( 3 )     0.0056
          2.9892  1.9203
    A( 6 )    0.0000
          3.0001   1.8974

 0.0099  0.0191  0.3948
                                         0.0001  0.0001  0.4143
                         

 2.4157   1.8579  1.3937            2.4142 1.8597   1.3934
A( 4 )      0.0010
          3.0021   1.8930 A ( 7 )   0.0000 3.0000  1.8974
                                 
 0.0017  0.0038  0.4178
                                     0.0000 0.0000  0.4142
                       

1  2.4142 ,  2  3.0000 ,  3  0.4142
MATLAB Example
A =
2.4634    1.8104   -1.3865
-0.0310    3.0527    1.7694
» A=[1 2 -1; 2 2 -1; 2 -1 2]
0.0616   -0.1047   -0.5161
A =                                    A =
1     2    -1                            2.4056    1.8691    1.3930
2     2    -1                            0.0056    2.9892   -1.9203
2    -1     2                            0.0099   -0.0191   -0.3948
QR         A =
» [Q,R]=QR_factor(A)                          2.4157    1.8579   -1.3937
factorization         -0.0010    3.0021    1.8930
Q =
0.0017   -0.0038   -0.4178
-0.3333  -0.5788     -0.7442       A =
-0.6667  -0.4134      0.6202              2.4140    1.8600    1.3933
-0.6667   0.7029     -0.2481              0.0002    2.9996   -1.8982
R =                                           0.0003   -0.0007   -0.4136
-3.0000  -1.3333     -0.3333       A =
0.0000  -2.6874      2.3980              2.4143    1.8596   -1.3934
0.0000   0.0000     -0.3721              0.0000    3.0001    1.8972
0.0001   -0.0001   -0.4143
» e=QR_eig(A,6);     eigenvalue        e =
A =                                           2.4143
2.1111    2.0535     1.4884               3.0001
0.1929    2.7966    -2.2615              -0.4143
0.2481   -0.2615     0.0923
Improved QR Method
• Using similarity transformation to form an upper
Hessenberg Matrix (upper triangular matrix &
one nonzero band below diagonal)
• More efficient to form Hessenberg matrix
without explicitly forming the Householder
matrices (not given in textbook)

function A = Hessenberg(A)
[n,nn] = size(A);
for k = 1:n-2
H = Householder(A(:,k),k+1);
A = H*A*H;
end
Improved QR Method                  A =
2.4056   -2.1327    0.9410
» A=[1 2 -1; 2 2 -1; 2 -1 2]              -0.0114   -0.4056   -1.9012
A =                                        0.0000    0.0000    3.0000
1      2    -1                A =
2      2    -1                       2.4157    2.1194   -0.9500
2     -1     2                      -0.0020   -0.4157   -1.8967
» [Q,R]=QR_factor_g(A) Hessenberg          0.0000    0.0000    3.0000
Q =                       matrix    A =
0.4472     0.5963   -0.6667           2.4140   -2.1217    0.9485
0.8944    -0.2981    0.3333          -0.0003   -0.4140   -1.8975
0    -0.7454   -0.6667           0.0000    0.0000    3.0000
R =                                 A =
2.2361     2.6833   -1.3416           2.4143    2.1213   -0.9487
-1.4907     1.3416   -1.7889          -0.0001   -0.4143   -1.8973
-1.3333          0   -1.0000           0.0000    0.0000    3.0000
» e=QR_eig_g(A,6);     eigenvalue   e =
A =                                     2.4143
2.1111    -2.4356    0.7071       -0.4143
-0.3143    -0.1111   -2.0000        3.0000
0     0.0000    3.0000    » eig(A)
A =                                 ans =               MATLAB
2.4634     2.0523   -0.9939        2.4142          function
-0.0690    -0.4634   -1.8741       -0.4142
0.0000     0.0000    3.0000        3.0000
Summary

• QR Factorization
– Householder matrix
– Hessenberg matrix
Homework
• Check the Homework webpage

```
To top