Review of Matrix Algebra
1 Basic Definitions
A matrix is a two-dimensional arrangement of
elements.
The elements of a matrix can be numbers, variables,
or symbols.
The elements are indexed by the row and then the
column in which they occur. For example, A22
represents the item of matrix A located on the second
row, second column
The size of a matrix – called the order – is denoted by
the number of rows and then the number of columns.
For instance a 2x3 matrix is a matrix that has two
rows and three columns.
For example,
a b c
f
A2 x 3
d e
1
A vector is a matrix that has only a single row or a
matrix that has only a single column.
A row-vector has only a single row, such as
B1x 4 1 2 2
A column-vector has only a single column, such as
1
2
C4 x1
3
4
If we don’t specify row or column, we assume that a
vector is a column vector.
A scalar is a matrix (or row-vector or column-vector)
that has only a single element.
Scalars are non-dimensional numbers, variables, or
symbols like you’re used to seeing
and have seen since grade school.
You already know scalar arithmetic and algebra, but
you may not have known that it
2
was scalar arithmetic and algebra.
For example, [1] or [a] or [] are all scalars.
The transpose of a k x n matrix A is the n x k matrix
obtained by interchanging the rows and columns of A
(it is typically denoted as AT or A’). To transpose a
matrix (or vector), each row is made into a column,
which means that each column will become a row in
the transposed matrix.
1 4
1 2 3
If A , then A ' 2 5
4 5 6
3 6
If A = A’, then we say that A is symmetric. Thus,
only square matrices may be symmetric since the
sizes would not work out for rectangular matrices.
3
Exercises
1) Find A’
A
2) Find B’
3
7
B
66
21
3) Find C’
C 2 0 1 0
4) Find D’
c d e f
D
g h i j
4
2 Arithmetic
2.1 Scalar Arithmetic
You’ve known how to perform these operations for a
long time. . .
• Addition (& subtraction): 1 + 2 = 3
• Multiplication: 2*3 = 6
• Division: 3/4 = 3* (1/4) = 3* (4)-1
2.2 Vector Arithmetic
For vector arithmetic, we have to make sure that the
vectors are conformable for that particular operation.
• Addition (& Subtraction): To be conformable, the
vectors must both be the “same size”
(that is, they must both be row-vectors with the same
number of columns or both be
column-vectors with the same number of rows).
5
Example:
1 4 5
2 5 7
3 6 9
Exercises
2 2
Let A 4 , and B a
6
5
1) Find A+B
2) Find A’+B’
• Multiplication: To be conformable, we can either
multiply a row-vector times a column-vector or a
column-vector times a row-vector, but we cannot
multiply two row vectors or two column vectors.
Unlike scalar multiplication, the order is important:
RC ≠ CR
6
• Inner Product: row-vector * column-vector =
scalar. In addition, to be conformable for inner-
product multiplication, the number of columns in
the row-vector must equal the number of rows in
the column-vector.
4
1 2 3 5 1* 4 2 *5 3* 6 32
6
c1
More generally, if R r1 r2 r3 and C c2
c3
then
RC ri * ci
7
• Outer Product: column-vector * row-vector =
matrix. The vectors may be of any size, but their
sizes will determine the size of the matrix yielded by
their outer product.
4 4 8 12
5 1 2 3 5 10 15
6
6 12 18
c1
More generally, if R r1 r2 r3 and C c2
c3
then
r1 * c1 r1 * c2 r1 * c3
CR r2 * c1 r2 * c2 r2 * c3
r3 * c1
r3 * c2 r3 * c3
We can generalize the formula by indicating the
formula for each element of the product. If we call
the product matrix M, mij refers to the element in
the ith row and jth column of M.
mij = ri · cj
8
• Division: There is no convenient “division”
operator for vectors.
Exercises
2
Let A 4 , and B 4 2 1
6
1) Find AB
2) Find BA
3) Find A’A
4) Find B’B
9
2.3 Matrix Arithmetic
For matrix arithmetic, we must also make sure the
matrices are conformable as well.
Laws of Matrix Algebra:
Associative Laws: (A+B)+C=A+(B+C)
(AB)C=A(BC)
(AB)’=B’A’
Commutative Law for Addition: A+B=B+A
Note however that this law does not hold for
multiplication, AB≠BA
10
• Addition (& subtraction): To be conformable, the
matrices must have both the same number of rows
and the same number of columns. Like vector
addition, matrix addition is an element by element
operator (i.e., corresponding elements are simply
added together (or subtracted as the case may be).
For example
1 4 7 1 2 0
A 2 5 8 and B 2 1 1 , then
3 6 9
1 2 2
1 1 4 2 7 0 2 6 7
A B 2 2 5 1 8 1 4 6 9
3 1 6 2 9 2 4 8 11
11
Exercises
4 8 12 1 0 1
1) Let A 5 10 15 , B 1 2 2 . Find
6 12 18
3 2 1
A+B
2) Find A-B
3) Find A’+B’
12
• Multiplication: To be conformable for
multiplication, the number of columns of the first
matrix must equal the number of rows of the second
matrix. The resulting product will have as many rows
as the first matrix and as many columns as the second
matrix.
As with vector multiplication, the order of
multiplication is important. In fact, we have some
special nomenclature to indicate the order of
multiplication:
• A is premultiplied by B = BA
• B is premultiplied by A = AB
• A is postmultiplied by B = AB
• B is postmultiplied by A = BA
The most elegant way to express the process of
matrix multiplication is to say that (i,j) element is
given by the inner product of the ith row of the first
matrix and the jth column of the second matrix.
13
1 2
1 2 3
For example, if A and B 1 2
4 5 6
1 2
then
1 2
1 2 3 1*1 2*1 3*1 1* 2 2* 2 3* 2
AB 1 2
4 5 6 1 2 4*1 5*1 6*1 4* 2 5* 2 6* 2
6 12
15 30
1 2 1*1 2* 4 1* 2 2*5 1*3 2*6
BA 1 2 1 2 3 1*1 2* 4 1* 2 2*5 1*3 2*6
4 5 6
1*1 2* 4 1* 2 2*5 1*3 2*6
1 2
9 12 15
9 12 15
9 12 15
Clearly, AB ≠ BA.
14
Exercises
5 8
c f
1) Let A 2 6 , and B
d e
. Find AB
g h i j
7 3
1 0 0 0
0 1 0 0
2) Let C 7 5 4 2, and D .
0 0 1 0
0 0 0 1
Find CD
a b
A B
3) Let E c d , and F . Find EF
C D
e
f
5 8
4) Let A 2 6 .
7 3
a) Find A’A
b) Find AA’
15
• Inverses: The matrix analogue of division is
multiplying by an inverse (which is another way of
thinking about division in scalar arithmetic).
Properties of the Inverse of a Matrix:
1) For any nonsingular matrix A, A1 A
1
2) The inverse of a matrix A is unique
3) For any non-singular matrix A, A ' A1 '
1
4) If A and B are non-singular and of the same
dimension, then AB is non-singular and
AB
1
A1 B 1
Finding the inverse of a matrix is (relatively) easy for
square matrices and (relatively) complex for
rectangular matrices. .
For scalars, the inverse of a number times that
number is one. A matrix inverse times that matrix
should be equal to “one” – but we need to find a
matrix version of “one.”
16
The matrix equivalent of one is called the “Identity
Matrix.” The identity matrix has one’s on the main
diagonal (elements with row and column indices that
are equal) and zeroes elsewhere.
1 0 0 0
0 1 0 0
For instance, I 4 x 4
0 0 1 0
0 0 0 1
Thus, for matrix inverses, A · A-1 = I = A-1 · A
We have a fairly simple formula for 2x2 matrices.
a b 1 1 d b
If A then A ad cb c a
c d
2 1
For example, if A then
3 4
1 1 4 1 4 / 5 1/ 5
A 3 2 3/ 5 2 / 5
2* 4 1*3
Let’s check to make sure that AA-1 = I
17
2 1 0.8 0.2 1.6 0.6 0.4 0.4 1 0
3 4 0.6 0.4 2.4 2.4 0.6 1.6 0 1
This formula applies only to 2x2 matrices, although
there is a similar (but much more complicated)
formula for 3x3 matrices. Instead of developing a
series of formulas, it is desirable to find a more
general approach.
Not all matrixes are invertible. The question of
whether a matrix is nonsingular (hence, invertible) is
linked to the value of its determinant. The
determinant of a square matrix A, written |A|, is a
scalar associated with that matrix. Determinants are
only defined for square matrices, and they are found
using a recursive algorithm.
• The determinant of a scalar is that scalar: |a| = a.
• The determinant of a 2x2 matrix is found by
multiplying each element in a given row times the
18
determinant of the reduced matrix formed by deleting
the row and column of this element, then multiplying
by (-1)i+j . These products are then added together to
yield the determinant.
2 1
2* 4 1 3*1 1
11 1 2
A 83 5
3 4
• The determinant of a 3x3 matrix is found the same
way, except that the determinant of the reduced
matrix after eliminating the row & column is the
determinant of a 2x2 matrix rather than the
degenerate case previously when it was the
determinant of a scalar.
4 3 2
5 3 1 3 1 5
1 1 1
11 1 2 1 3
A 1 5 3 4 3 2
1 4 2 4 2 1
2 1 4
4(17)(1) 3(2)(1) 2(9)(1) 56
This is a Laplace expansion of the matrix A along its
first row.
19
The determinant of a matrix can be evaluated with a
Laplace expansion along any one of its rows or any
one of its columns.
• The determinants of larger matrices are found the
same way, except that it takes more steps to find the
determinant of the reduced matrix. Since we define
determinants are linear combinations of the
determinants of smaller sub-matrices, we call the
algorithm a “recursive” algorithm.
The Laplace expansion method, discussed here in the
context of a 3x3 matrix, can be expressed in another
way that enables us to generalize this method to
matrices of higher dimensions. We first need to
consider two definitions:
Minor: Let A be an n x n matrix. Let Mij be the (n-
1)x(n-1) matrix obtained by deleting the ith row and
the jth column of A. The determinant of that matrix,
denoted |Mij|, is called the minor of aij.
20
Cofactor: Let A be an n x n matrix. The cofactor Cij
is a minor multiplied by either 1, if (i + j) is an even
integer, or -1 if (i + j) is an odd integer. That is,
Cij (1)i j M ij .
The Laplace expansion can be expressed in a
compact way using cofactors. For example, the
evaluation of the determinant of A through a Laplace
expansion along its first row can be written as
3
A a1 j C1 j
j 1
The general rule for inverse matrixes is
1
A1 (adj A)
A
where adj A is the adjoint of A.
Let A be an n x n matrix. Define the n x n matrix in
which the (i, j)th element is the cofactor Cij of A as the
21
matrix of cofactors. The adjoint matrix is an n x n
matrix that is the transpose of the matrix of cofactors.
The inverse of a matrix can be found following these
four steps:
1) Get the determinant of your matrix. We take here
our example of matrix A above; that is |A|=56.
17 2 9
2) Get the matrix of cofactors 10 12 2
1 12 17
3) Transpose the matrix of cofactors to get
17 10 1
adj ( A) 2 12 12
9 2
17
4) Divide adj (A) by the determinant and to
0.304 0.179 0.018
get A1 0.036 0.214 0.214
0.161 0.036 0.304
Let’s check to make sure that AA-1 = I
22
4 3 2 0.304 0.179 0.018
1 5 3 0.036 0.214 0.214
2 1 4 0.161 0.036 0.304
4(0.304) 3(0.036) 2(0.161) 4( 0.179) 3(0.214) 2(0.036) 4(0.018) 3(0.214) 2(0.304)
1(0.304) 5(0.036) 3(0.161) 1(0.179) 5(0.214) 3(0.036) 1(0.018) 5(0.214) 3(0.304)
2(0.304) 1(0.036) 4(0.161) 2(0.179) 1(0.214) 4(0.036) 2(0.018) 1( 0.214) 4(0.304)
1 0 0
0 1 0
0 0 1
Exercises
1 0
1) Let A 0
1 . Find A-1
1 0
1
2 4 5
2) Let B 0 3 0
. Find B-1
1 0 1
23