VIEWS: 26 PAGES: 10 CATEGORY: Education POSTED ON: 1/19/2010
EE103 (Spring 2004-05) 4. The solution of a set of linear equations • nonsingular matrices, the inverse of a matrix • the condition of a set of linear equations • matrix norm • condition number 4–1 Nonsingular matrices n linear equations in n variables a11x1 + a12x2 + · · · + a1nxn = b1 a21x1 + a22x2 + · · · + a2nxn = b2 . . an1x1 + an2x2 + · · · + annxn = bn in matrix form: Ax = b where b ∈ Rn, A ∈ Rn×n, x ∈ Rn deﬁnition: A is nonsingular if for all b there is a unique solution x in Matlab: if A is nonsingular, then x = A\b returns the unique solution; if A is singular, it generates an error message The solution of a set of linear equations 4–2 Equivalent characterizations of nonsingularity the following statements are equivalent (for A ∈ Rn×n): • A is nonsingular • AT is nonsingular • det A = 0 • there exists a unique matrix A−1 ∈ Rn×n such that A−1A = AA−1 = I A−1 is called the inverse of A • the columns of A are linearly independent: n Ax = xiai = 0 =⇒ x = 0 i=1 where A = a1 a2 · · · a n (ai is the ith column of A) The solution of a set of linear equations 4–3 example 1 −1 A= 1 2 • det A = 3 2 1 • A−1 = (1/3) −1 1 • columns are linearly independent: 1 −1 x1 + x2 =0 =⇒ x 1 = x2 = 0 1 2 • rows are linearly independent: 1 1 y1 + y2 =0 =⇒ y 1 = y2 = 0 −1 2 The solution of a set of linear equations 4–4 example 1 −1 A= −2 2 • det A = 0 • columns are linearly dependent: 1 −1 + =0 −2 2 • rows are linearly dependent: 1 −2 2 + =0 −1 2 The solution of a set of linear equations 4–5 Determining whether a matrix is nonsingular methods that are not recommended • calculate the determinant of A (as expensive as solving Ax = b) • compute the inverse of A (more expensive than solving Ax = b; in practice, the inverse is computed by solving n systems Ax = ei where ei is the ith unit vector) some practical methods • if A is given numerically: choose any b and try to solve Ax = b; the Matlab command x = A\b will generate an error message if A is singular • if A is given by a formula: check if there exists an x = 0 with Ax = 0 The solution of a set of linear equations 4–6 example: polynomial interpolation (page 3–4) n−1 1 t 1 t2 · · · t 1 1 1 t2 t2 · · · tn−1 A= . . 2 2 . . . ... . . . 1 t n t2 · · · t n n n−1 is nonsingular if ti = tj for i = j proof. we show that Ax = 0 ⇒ x = 0 n−1 x 1 + x 2 t1 + · · · + x n t1 x1 + x2t2 + · · · + xntn−1 Ax = 2 =0 . . n−1 x 1 + x 2 tn + · · · + x n tn means that t1, . . . , tn are roots of the polynomial x1 + x2t + · · · + xntn−1 a polynomial of degree n − 1 can’t have more than n − 1 real roots, hence, Ax = 0 ⇒ x = 0, i.e., A is nonsingular The solution of a set of linear equations 4–7 PSfrag replacements example (linear static circuit) R1 R2 x1 x2 b1 R3 b2 b = Ax with R1 + R 3 R3 A= (R1, R2, R3 > 0) R3 R2 + R 3 A is nonsingular, i.e., there is no nonzero x with Ax = 0 Ax = 0 means voltage sources are short-circuited proof. PSfrag replacements R1 R2 x1 R3 x2 x = 0 is impossible if at least two of the three Ri’s are positive The solution of a set of linear equations 4–8 Ax = b with A singular has no solutions or inﬁnitely many solutions, depending on b example 1 −1 1 2 A= , b(1) = , b(2) = −2 2 0 −4 • Ax = b(1) has no solutions • Ax = b(2) has inﬁnitely many solutions (x1 = 2 + t, x2 = t, any t) related questions when A is singular: • if there are no solutions, what is the x that minimizes Ax − b ? • if there are inﬁnitely many solutions, what is the solution x that is best in some sense (for example, smallest x ) ? in this course we’ll only cover the case where A is nonsingular The solution of a set of linear equations 4–9 Condition of a set of linear equations consider Ax = b, with A ∈ Rn×n nonsingular • suppose we have an error or noise in b, i.e., b becomes b + ∆b • x becomes x + ∆x, with A(x + ∆x) = b + ∆b, i.e., ∆x = A−1∆b • of course ∆b (and hence ∆x) is not known; however, suppose we have a bound on the size of ∆b, can we bound ∆x ? we say A is ill-conditioned if the solution x is very sensitive to errors in b The solution of a set of linear equations 4–10 example 1 1 1 1 − 1010 1010 A= −10 , A−1 = 2 1 + 10 1 − 10−10 1 + 1010 −1010 • exact rhs b = (1, 1), exact solution x = (1, 1) • suppose |∆b1| ≤ 0.01, |∆b2| ≤ 0.01 (i.e., error is less than 1%) ∆x1 1 − 1010 1010 ∆b1 = ∆x2 1 + 1010 −1010 ∆b2 ∆b1 − 1010(∆b1 − ∆b2) = ∆b1 + 1010(∆b1 − ∆b2) small error ∆b (for example, ∆b1 = 0.01, ∆b2 = −0.01) may lead to very large error ∆x The solution of a set of linear equations 4–11 Norm of a matrix A ∈ Rm×n deﬁnes a linear function f (x) = Ax PSfrag replacements x A y = Ax • for x ∈ Rn, Ax / x gives the ampliﬁcation factor or gain in the direction x • obviously, in general the gain depends on the direction x • we deﬁne the norm of A as the maximum gain (over all directions), i.e., Ax A = max = max Ax x=0 x x =1 (also called spectral norm or 2-norm) The solution of a set of linear equations 4–12 Computing the norm of a matrix general idea: ﬁnd x = 0 that maximizes Ax / x simple examples where it is easy to maximize Ax / x : • zero matrix: 0 = 0 • identity matrix: I = 1 • diagonal matrix: if α1 0 · · · 0 0 α2 · · · 0 A= . . . ... . . . 0 0 · · · αn then A = maxi=1,...,n |αi| for general A, we have to use a numerical algorithm to calculate A (in Matlab: use norm(A)) The solution of a set of linear equations 4–13 Other matrix norms there are many other possible deﬁnitions of matrix norms, e.g., the Frobenius norm m n A F = a2 ij i=1 j=1 • simple explicit expression • in Matlab: use norm(A,’fro’) in this course: A stands for the ‘operator’ norm (page 4–12) • no simple explicit expression (except for special A) • readily computed numerically (in Matlab: use norm(A)) The solution of a set of linear equations 4–14 Properties of the matrix norm • αA = |α| A • A ≥ 0 for all A; A = 0 iﬀ A = 0 • A+B ≤ A + B • Ax ≤ A x for all x ∈ Rn • AB ≤ A B for A ∈ Rn×n and nonsingular: Ax • 1/ A−1 = minx=0 x (i.e., minimum gain over all directions) • A A−1 ≥ 1 (proofs follow directly from the deﬁnition) The solution of a set of linear equations 4–15 Condition number Ax = b (A ∈ Rn×n and nonsingular) with error ∆b in rhs b (page 4–10) bound on absolute error ∆x : ∆x ≤ A−1 ∆b (follows from ∆x = A−1∆b and property 4 on page 4–15) • an upper bound on the absolute error ∆x : A−1 large doesn’t necessarily mean that ∆x is large • A−1 small means ∆x is small when ∆b is small • for any A, there exists ∆b such that ∆x = A−1 ∆b (no proof) The solution of a set of linear equations 4–16 bound on relative error ∆x / x : ∆x ∆b ≤ A A−1 x b (follows from ∆x ≤ A−1 ∆b and b ≤ A x ) κ(A) = A A−1 is called the condition number of A • κ(A) ≥ 1 for all A (last property on page page 4–15) • an upper bound on the relative error in x: κ(A) large doesn’t necessarily mean that ∆x / x is large • for any A, there exists b, ∆b such that ∆x ∆b = κ(A) x b (no proof) The solution of a set of linear equations 4–17 • A is a well-conditioned matrix if κ(A) is small (i.e., close to 1): the relative error in x is not much larger than the relative error in b • A is badly conditioned or ill-conditioned if κ(A) is large: the relative error in x can be much larger than the relative error in b The solution of a set of linear equations 4–18 Determining the condition number in practice, how do we determine whether a matrix is ill-conditioned? condition number • in Matlab: cond(A) returns the condition number κ(A) • expensive to calculate (more expensive than solving Ax = b) condition number estimator • in Matlab: condest(A) returns an estimate of κ(A) • much cheaper to evaluate than κ(A) The solution of a set of linear equations 4–19