4. The solution of a set of linear equations by lsg16921

VIEWS: 26 PAGES: 10

									                                                                      EE103 (Spring 2004-05)


      4. The solution of a set of linear equations



• nonsingular matrices, the inverse of a matrix

• the condition of a set of linear equations

• matrix norm

• condition number




                                                                                        4–1




                                            Nonsingular matrices

n linear equations in n variables

                                 a11x1 + a12x2 + · · · + a1nxn = b1
                                 a21x1 + a22x2 + · · · + a2nxn = b2
                                                               .
                                                               .
                                an1x1 + an2x2 + · · · + annxn = bn

in matrix form: Ax = b where b ∈ Rn, A ∈ Rn×n, x ∈ Rn

definition: A is nonsingular if for all b there is a unique solution x

in Matlab: if A is nonsingular, then

                                                  x = A\b

returns the unique solution; if A is singular, it generates an error message

The solution of a set of linear equations                                               4–2
             Equivalent characterizations of nonsingularity
the following statements are equivalent (for A ∈ Rn×n):
• A is nonsingular
• AT is nonsingular
• det A = 0
• there exists a unique matrix A−1 ∈ Rn×n such that

                                                   A−1A = AA−1 = I

     A−1 is called the inverse of A
• the columns of A are linearly independent:
                                                    n
                                            Ax =          xiai = 0 =⇒ x = 0
                                                    i=1

     where A =               a1 a2 · · · a n                (ai is the ith column of A)


The solution of a set of linear equations                                                 4–3




example
                                                            1 −1
                                                   A=
                                                            1  2

• det A = 3

                                  2 1
• A−1 = (1/3)
                                 −1 1

• columns are linearly independent:

                                  1                 −1
                         x1                 + x2             =0    =⇒     x 1 = x2 = 0
                                  1                  2


• rows are linearly independent:

                                    1                   1
                          y1                 + y2            =0    =⇒     y 1 = y2 = 0
                                   −1                   2


The solution of a set of linear equations                                                 4–4
example
                                                       1 −1
                                                A=
                                                      −2  2

• det A = 0

• columns are linearly dependent:

                                                 1         −1
                                                      +         =0
                                                −2          2


• rows are linearly dependent:

                                                  1        −2
                                            2          +        =0
                                                 −1         2




The solution of a set of linear equations                                4–5




              Determining whether a matrix is nonsingular

methods that are not recommended

• calculate the determinant of A (as expensive as solving Ax = b)
• compute the inverse of A (more expensive than solving Ax = b; in
  practice, the inverse is computed by solving n systems

                                                      Ax = ei

     where ei is the ith unit vector)

some practical methods

• if A is given numerically: choose any b and try to solve Ax = b; the
  Matlab command x = A\b will generate an error message if A is
  singular
• if A is given by a formula: check if there exists an x = 0 with Ax = 0

The solution of a set of linear equations                                4–6
example: polynomial interpolation (page 3–4)

                                                          n−1
                                         1 t 1 t2 · · · t 1
                                                             
                                                1
                                        1 t2 t2 · · · tn−1 
                                     A= . .    2         2
                                        . .    . ...
                                                .           . 
                                                            . 
                                         1 t n t2 · · · t n
                                                n
                                                          n−1


is nonsingular if ti = tj for i = j

proof. we show that Ax = 0 ⇒ x = 0

                                                                n−1
                                                                   
                                    x 1 + x 2 t1 + · · · + x n t1
                                   x1 + x2t2 + · · · + xntn−1 
                             Ax =                              2   =0
                                                  .
                                                   .                
                                                                n−1
                                    x 1 + x 2 tn + · · · + x n tn

means that t1, . . . , tn are roots of the polynomial x1 + x2t + · · · + xntn−1

a polynomial of degree n − 1 can’t have more than n − 1 real roots, hence,
Ax = 0 ⇒ x = 0, i.e., A is nonsingular

The solution of a set of linear equations                                              4–7



         PSfrag replacements

example (linear static circuit)
                                                     R1        R2
                                            x1                         x2
                                     b1                   R3                b2



b = Ax with
                                  R1 + R 3   R3
                     A=                                             (R1, R2, R3 > 0)
                                    R3     R2 + R 3

A is nonsingular, i.e., there is no nonzero x with Ax = 0
       Ax = 0 means voltage sources are short-circuited
proof. PSfrag replacements
                                                     R1        R2


                                       x1                 R3           x2



x = 0 is impossible if at least two of the three Ri’s are positive

The solution of a set of linear equations                                              4–8
                                        Ax = b with A singular

has no solutions or infinitely many solutions, depending on b
example

                             1 −1                        1                 2
               A=                           ,   b(1) =       ,   b(2) =
                            −2  2                        0                −4


• Ax = b(1) has no solutions
• Ax = b(2) has infinitely many solutions (x1 = 2 + t, x2 = t, any t)

related questions when A is singular:
• if there are no solutions, what is the x that minimizes Ax − b ?
• if there are infinitely many solutions, what is the solution x that is best
  in some sense (for example, smallest x ) ?

in this course we’ll only cover the case where A is nonsingular

The solution of a set of linear equations                                       4–9




                       Condition of a set of linear equations

consider Ax = b, with A ∈ Rn×n nonsingular

• suppose we have an error or noise in b, i.e., b becomes b + ∆b

• x becomes x + ∆x, with A(x + ∆x) = b + ∆b, i.e.,

                                                ∆x = A−1∆b


• of course ∆b (and hence ∆x) is not known; however, suppose we have
  a bound on the size of ∆b, can we bound ∆x ?

we say A is ill-conditioned if the solution x is very sensitive to errors in b




The solution of a set of linear equations                                      4–10
example

                1          1          1                                1 − 1010  1010
       A=                     −10                          ,   A−1 =
                2      1 + 10     1 − 10−10                            1 + 1010 −1010

• exact rhs b = (1, 1), exact solution x = (1, 1)
• suppose |∆b1| ≤ 0.01, |∆b2| ≤ 0.01 (i.e., error is less than 1%)



                             ∆x1                  1 − 1010  1010         ∆b1
                                             =
                             ∆x2                  1 + 1010 −1010         ∆b2
                                                  ∆b1 − 1010(∆b1 − ∆b2)
                                             =
                                                  ∆b1 + 1010(∆b1 − ∆b2)


small error ∆b (for example, ∆b1 = 0.01, ∆b2 = −0.01) may lead to very
large error ∆x


The solution of a set of linear equations                                               4–11




                                             Norm of a matrix

A ∈ Rm×n defines a linear function f (x) = Ax
 PSfrag replacements

                                     x                 A               y = Ax



• for x ∈ Rn, Ax / x gives the amplification factor or gain in the
  direction x
• obviously, in general the gain depends on the direction x
• we define the norm of A as the maximum gain (over all directions), i.e.,

                                                       Ax
                                            A = max       = max Ax
                                                 x=0    x   x =1


     (also called spectral norm or 2-norm)


The solution of a set of linear equations                                               4–12
                            Computing the norm of a matrix

general idea: find x = 0 that maximizes Ax / x
simple examples where it is easy to maximize Ax / x :
• zero matrix: 0 = 0
• identity matrix: I = 1
• diagonal matrix: if
                                                             
                                                α1 0 · · · 0
                                               0 α2 · · · 0 
                                            A= .
                                               .  . ... . 
                                                   .        . 
                                                0  0 · · · αn

     then A = maxi=1,...,n |αi|

for general A, we have to use a numerical algorithm to calculate A
(in Matlab: use norm(A))

The solution of a set of linear equations                                 4–13




                                            Other matrix norms

there are many other possible definitions of matrix norms, e.g., the
Frobenius norm
                                                          m   n
                                              A   F   =             a2
                                                                     ij
                                                          i=1 j=1



• simple explicit expression
• in Matlab: use norm(A,’fro’)


in this course: A stands for the ‘operator’ norm (page 4–12)

• no simple explicit expression (except for special A)
• readily computed numerically (in Matlab: use norm(A))



The solution of a set of linear equations                                 4–14
                               Properties of the matrix norm

• αA = |α| A

• A ≥ 0 for all A; A = 0 iff A = 0

• A+B ≤ A + B


• Ax ≤ A x for all x ∈ Rn

• AB ≤ A B

for A ∈ Rn×n and nonsingular:
                                            Ax
• 1/ A−1 = minx=0                            x   (i.e., minimum gain over all directions)

• A A−1 ≥ 1

(proofs follow directly from the definition)

The solution of a set of linear equations                                                   4–15




                                             Condition number

Ax = b (A ∈ Rn×n and nonsingular) with error ∆b in rhs b (page 4–10)
bound on absolute error ∆x :

                                                 ∆x ≤ A−1       ∆b

(follows from ∆x = A−1∆b and property 4 on page 4–15)


• an upper bound on the absolute error ∆x : A−1 large doesn’t
  necessarily mean that ∆x is large
• A−1 small means ∆x is small when ∆b is small
• for any A, there exists ∆b such that

                                                 ∆x = A−1        ∆b

     (no proof)

The solution of a set of linear equations                                                   4–16
bound on relative error ∆x / x :

                                            ∆x             ∆b
                                               ≤ A A−1
                                            x              b

(follows from ∆x ≤ A−1                      ∆b and b ≤ A     x )

κ(A) = A A−1 is called the condition number of A

• κ(A) ≥ 1 for all A (last property on page page 4–15)
• an upper bound on the relative error in x: κ(A) large doesn’t
  necessarily mean that ∆x / x is large
• for any A, there exists b, ∆b such that
                                              ∆x        ∆b
                                                 = κ(A)
                                               x        b

      (no proof)

The solution of a set of linear equations                                      4–17




• A is a well-conditioned matrix if κ(A) is small (i.e., close to 1):
     the relative error in x is not much larger than the relative error in b

• A is badly conditioned or ill-conditioned if κ(A) is large:
     the relative error in x can be much larger than the relative error in b




The solution of a set of linear equations                                      4–18
                          Determining the condition number

in practice, how do we determine whether a matrix is ill-conditioned?


condition number

• in Matlab: cond(A) returns the condition number κ(A)
• expensive to calculate (more expensive than solving Ax = b)



condition number estimator

• in Matlab: condest(A) returns an estimate of κ(A)
• much cheaper to evaluate than κ(A)



The solution of a set of linear equations                               4–19

								
To top