# 4. The solution of a set of linear equations by lsg16921

VIEWS: 26 PAGES: 10

• pg 1
```									                                                                      EE103 (Spring 2004-05)

4. The solution of a set of linear equations

• nonsingular matrices, the inverse of a matrix

• the condition of a set of linear equations

• matrix norm

• condition number

4–1

Nonsingular matrices

n linear equations in n variables

a11x1 + a12x2 + · · · + a1nxn = b1
a21x1 + a22x2 + · · · + a2nxn = b2
.
.
an1x1 + an2x2 + · · · + annxn = bn

in matrix form: Ax = b where b ∈ Rn, A ∈ Rn×n, x ∈ Rn

deﬁnition: A is nonsingular if for all b there is a unique solution x

in Matlab: if A is nonsingular, then

x = A\b

returns the unique solution; if A is singular, it generates an error message

The solution of a set of linear equations                                               4–2
Equivalent characterizations of nonsingularity
the following statements are equivalent (for A ∈ Rn×n):
• A is nonsingular
• AT is nonsingular
• det A = 0
• there exists a unique matrix A−1 ∈ Rn×n such that

A−1A = AA−1 = I

A−1 is called the inverse of A
• the columns of A are linearly independent:
n
Ax =          xiai = 0 =⇒ x = 0
i=1

where A =               a1 a2 · · · a n                (ai is the ith column of A)

The solution of a set of linear equations                                                 4–3

example
1 −1
A=
1  2

• det A = 3

2 1
• A−1 = (1/3)
−1 1

• columns are linearly independent:

1                 −1
x1                 + x2             =0    =⇒     x 1 = x2 = 0
1                  2

• rows are linearly independent:

1                   1
y1                 + y2            =0    =⇒     y 1 = y2 = 0
−1                   2

The solution of a set of linear equations                                                 4–4
example
1 −1
A=
−2  2

• det A = 0

• columns are linearly dependent:

1         −1
+         =0
−2          2

• rows are linearly dependent:

1        −2
2          +        =0
−1         2

The solution of a set of linear equations                                4–5

Determining whether a matrix is nonsingular

methods that are not recommended

• calculate the determinant of A (as expensive as solving Ax = b)
• compute the inverse of A (more expensive than solving Ax = b; in
practice, the inverse is computed by solving n systems

Ax = ei

where ei is the ith unit vector)

some practical methods

• if A is given numerically: choose any b and try to solve Ax = b; the
Matlab command x = A\b will generate an error message if A is
singular
• if A is given by a formula: check if there exists an x = 0 with Ax = 0

The solution of a set of linear equations                                4–6
example: polynomial interpolation (page 3–4)

n−1
1 t 1 t2 · · · t 1
            
1
 1 t2 t2 · · · tn−1 
A= . .    2         2
 . .    . ...
.           . 
. 
1 t n t2 · · · t n
n
n−1

is nonsingular if ti = tj for i = j

proof. we show that Ax = 0 ⇒ x = 0

n−1
                           
x 1 + x 2 t1 + · · · + x n t1
 x1 + x2t2 + · · · + xntn−1 
Ax =                              2   =0
                .
.                
n−1
x 1 + x 2 tn + · · · + x n tn

means that t1, . . . , tn are roots of the polynomial x1 + x2t + · · · + xntn−1

a polynomial of degree n − 1 can’t have more than n − 1 real roots, hence,
Ax = 0 ⇒ x = 0, i.e., A is nonsingular

The solution of a set of linear equations                                              4–7

PSfrag replacements

example (linear static circuit)
R1        R2
x1                         x2
b1                   R3                b2

b = Ax with
R1 + R 3   R3
A=                                             (R1, R2, R3 > 0)
R3     R2 + R 3

A is nonsingular, i.e., there is no nonzero x with Ax = 0
Ax = 0 means voltage sources are short-circuited
proof. PSfrag replacements
R1        R2

x1                 R3           x2

x = 0 is impossible if at least two of the three Ri’s are positive

The solution of a set of linear equations                                              4–8
Ax = b with A singular

has no solutions or inﬁnitely many solutions, depending on b
example

1 −1                        1                 2
A=                           ,   b(1) =       ,   b(2) =
−2  2                        0                −4

• Ax = b(1) has no solutions
• Ax = b(2) has inﬁnitely many solutions (x1 = 2 + t, x2 = t, any t)

related questions when A is singular:
• if there are no solutions, what is the x that minimizes Ax − b ?
• if there are inﬁnitely many solutions, what is the solution x that is best
in some sense (for example, smallest x ) ?

in this course we’ll only cover the case where A is nonsingular

The solution of a set of linear equations                                       4–9

Condition of a set of linear equations

consider Ax = b, with A ∈ Rn×n nonsingular

• suppose we have an error or noise in b, i.e., b becomes b + ∆b

• x becomes x + ∆x, with A(x + ∆x) = b + ∆b, i.e.,

∆x = A−1∆b

• of course ∆b (and hence ∆x) is not known; however, suppose we have
a bound on the size of ∆b, can we bound ∆x ?

we say A is ill-conditioned if the solution x is very sensitive to errors in b

The solution of a set of linear equations                                      4–10
example

1          1          1                                1 − 1010  1010
A=                     −10                          ,   A−1 =
2      1 + 10     1 − 10−10                            1 + 1010 −1010

• exact rhs b = (1, 1), exact solution x = (1, 1)
• suppose |∆b1| ≤ 0.01, |∆b2| ≤ 0.01 (i.e., error is less than 1%)

∆x1                  1 − 1010  1010         ∆b1
=
∆x2                  1 + 1010 −1010         ∆b2
∆b1 − 1010(∆b1 − ∆b2)
=
∆b1 + 1010(∆b1 − ∆b2)

small error ∆b (for example, ∆b1 = 0.01, ∆b2 = −0.01) may lead to very
large error ∆x

The solution of a set of linear equations                                               4–11

Norm of a matrix

A ∈ Rm×n deﬁnes a linear function f (x) = Ax
PSfrag replacements

x                 A               y = Ax

• for x ∈ Rn, Ax / x gives the ampliﬁcation factor or gain in the
direction x
• obviously, in general the gain depends on the direction x
• we deﬁne the norm of A as the maximum gain (over all directions), i.e.,

Ax
A = max       = max Ax
x=0    x   x =1

(also called spectral norm or 2-norm)

The solution of a set of linear equations                                               4–12
Computing the norm of a matrix

general idea: ﬁnd x = 0 that maximizes Ax / x
simple examples where it is easy to maximize Ax / x :
• zero matrix: 0 = 0
• identity matrix: I = 1
• diagonal matrix: if
           
α1 0 · · · 0
 0 α2 · · · 0 
A= .
 .  . ... . 
.        . 
0  0 · · · αn

then A = maxi=1,...,n |αi|

for general A, we have to use a numerical algorithm to calculate A
(in Matlab: use norm(A))

The solution of a set of linear equations                                 4–13

Other matrix norms

there are many other possible deﬁnitions of matrix norms, e.g., the
Frobenius norm
m   n
A   F   =             a2
ij
i=1 j=1

• simple explicit expression
• in Matlab: use norm(A,’fro’)

in this course: A stands for the ‘operator’ norm (page 4–12)

• no simple explicit expression (except for special A)
• readily computed numerically (in Matlab: use norm(A))

The solution of a set of linear equations                                 4–14
Properties of the matrix norm

• αA = |α| A

• A ≥ 0 for all A; A = 0 iﬀ A = 0

• A+B ≤ A + B

• Ax ≤ A x for all x ∈ Rn

• AB ≤ A B

for A ∈ Rn×n and nonsingular:
Ax
• 1/ A−1 = minx=0                            x   (i.e., minimum gain over all directions)

• A A−1 ≥ 1

(proofs follow directly from the deﬁnition)

The solution of a set of linear equations                                                   4–15

Condition number

Ax = b (A ∈ Rn×n and nonsingular) with error ∆b in rhs b (page 4–10)
bound on absolute error ∆x :

∆x ≤ A−1       ∆b

(follows from ∆x = A−1∆b and property 4 on page 4–15)

• an upper bound on the absolute error ∆x : A−1 large doesn’t
necessarily mean that ∆x is large
• A−1 small means ∆x is small when ∆b is small
• for any A, there exists ∆b such that

∆x = A−1        ∆b

(no proof)

The solution of a set of linear equations                                                   4–16
bound on relative error ∆x / x :

∆x             ∆b
≤ A A−1
x              b

(follows from ∆x ≤ A−1                      ∆b and b ≤ A     x )

κ(A) = A A−1 is called the condition number of A

• κ(A) ≥ 1 for all A (last property on page page 4–15)
• an upper bound on the relative error in x: κ(A) large doesn’t
necessarily mean that ∆x / x is large
• for any A, there exists b, ∆b such that
∆x        ∆b
= κ(A)
x        b

(no proof)

The solution of a set of linear equations                                      4–17

• A is a well-conditioned matrix if κ(A) is small (i.e., close to 1):
the relative error in x is not much larger than the relative error in b

• A is badly conditioned or ill-conditioned if κ(A) is large:
the relative error in x can be much larger than the relative error in b

The solution of a set of linear equations                                      4–18
Determining the condition number

in practice, how do we determine whether a matrix is ill-conditioned?

condition number

• in Matlab: cond(A) returns the condition number κ(A)
• expensive to calculate (more expensive than solving Ax = b)

condition number estimator

• in Matlab: condest(A) returns an estimate of κ(A)
• much cheaper to evaluate than κ(A)

The solution of a set of linear equations                               4–19

```
To top