Matrix Algebra
Matrix algebra is a means of expressing
large numbers of calculations made
upon ordered sets of numbers.
Often referred to as Linear Algebra
Why use it?
Matrix algebra is used primarily to
facilitate mathematical expression.
Many equations would be completely
intractable if scalar mathematics had to
be used. It is also important to note
that the scalar algebra is under there
somewhere.
Definitions - scalar
scalar - a number
denoted with regular type as is scalar
algebra
[1] or [a]
Definitions - vector
vector - a single row or column of
numbers
denoted with bold small letters
row vector 1 2 3 4 5
1
column vector 2
3
4
5
Definitions - Matrix
A matrix is a set of rows and columns of
numbers
1 2 3
4 5 6
Denoted with a bold Capital letter
All matrices (and vectors) have an order
- that is the number of rows x the
number of columns. Thus A is
1 2 3
4 5 6
2 x 3
Matrix Equality
Thus two matrices are equal iff (if and
only if) all of their elements are
identical
Note: your data set is a matrix.
Matrix Operations
Addition and Subtraction
Multiplication
Transposition
Inversion
Addition and Subtraction
Two matrices may be added iff they are
the same order.
Simply add the corresponding elements
a11 a12 b11 b12 c11 c12
a a22 b21 b22 c 21 c 22
21
a31 a32 b31 b32 c31 c32
Addition and Subtraction
(cont.)
Where
a11 b11 c11
a12 b12 c12
a21 b21 c 21
a22 b22 c 22
a31 b31 c31
a32 b32 c32
Hence
1 2 4 6 5 8
3 4 4 6 7 10
5 6 4 6 9 12
Matrix Multiplication
To multiply a scalar times a matrix,
simply multiply each element of the
matrix by the scalar quantity
a11 a12 2a11 2a12
2 2a
a21 a22 21 2a22
Matrix Multiplication (cont.)
To multiply a matrix times a matrix, we
write
A times B as AB
This is pre-multiplying B by A, or post-
multiplying A by B.
Matrix Multiplication (cont.)
In order to multiply matrices, they must
be conformable (the number of columns
in A must equal the number of rows in
B.)
an (mxn) x (nxp) = (mxp)
an (mxn) x (pxn) = cannot be done
a (1xn) x (nx1) = a scalar (1x1)
Matrix Multiplication (cont.)
Thus a11 a12 a13 b11 b12 c11 c12
a a22 a23 x b21 b22 c 21 c 22
21
a31 a32 a33 b31 b32 c31 c32
where
c11 a11b11 a12 b21 a13b31
c12 a11b12 a12 b22 a13b32
c 21 a21b11 a22 b21 a23b31
c 22 a21b12 a22 b22 a23b32
c31 a31b11 a32 b21 a33b31
c32 a31b12 a32 b22 a33b32
Matrix Multiplication- an
example
Thus 1 4 7 1 4 c11 c12 30 66
2 5 8 x 2 5 c c 22 36 81
21
3 6 9 3 6 c31 c32 42 96
where
c11 1 * 1 4 * 2 7 * 3 30
c12 1 * 4 4 * 5 7 * 6 66
c 21 2 * 1 5 * 2 8 * 3 36
c 22 2 * 4 5 * 5 8 * 6 81
c31 3 * 1 6 * 2 9 * 3 42
c32 3 * 4 6 * 5 9 * 6 96
Matrix multiplication is not
Commutative
AB does not necessarily equal BA
(BA may even be an impossible
operation)
Yet matrix multiplication is
Associative
A(BC) = (AB)C
Special matrices
There are a number of special matrices
Diagonal
Null
Identity
Diagonal Matrices
A diagonal matrix is a square matrix that
has values on the diagonal with all off-
diagonal entities being zero.
a11 0 0 0
0 a 0 0
22
0 0 a33 0
0 0 0 a44
Identity Matrix
An identity matrix is a diagonal matrix
where the diagonal elements all equal
one. It is used in a fashion analogous
to multiplying through by "1" in scalar
math.
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
Null Matrix
A square matrix where all elements equal
zero.
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
Not usually „used‟ so much as sometimes the
result of a calculation.
Analogous to “a+b=0”
The Transpose of a Matrix A'
Taking the transpose is an operation
that creates a new matrix based on an
existing one.
The rows of A = the columns of A'
Hold upper left and lower right corners
and rotate 180 degrees.
Example of a transpose
1 4
2 5, A' 1 2 3
A 4 5 6
3 6
The Transpose of a Matrix A'
If A = A', then A is symmetric (i.e. correlation
matrix)
If AA‟ = A then A' is idempotent (and A' = A)
The transpose of a sum = sum of transposes
( A B C )' A'B'C'
The transpose of a product = the product of
the transposes in reverse order
( ABC )' C' B' A'
An example:
Suppose that you wish to obtain the
sum of squared errors from the vector
e. Simply pre-multiply e by its
transpose e'.
which, in matrices looks like
e' e e1 e2 ..en
2 2 2
An example - cont
Since the matrix product is a scalar
found by summing the elements of the
vector squared.
The Determinant of a Matrix
The determinant of a matrix A is
denoted by |A|.
Determinants exist only for square
matrices.
They are a matrix characteristic, and
they are also difficult to compute
The Determinant for a 2x2 matrix
If A = a11 a12
a
21 a22
Then
A a11a22 a12a21
That one is easy
The Determinant for a 3x3 matrix
If A =
a11 a12 a13
a
21 a22 a23
a31 a32
a33
Then
A a11a22a33 a11a23a32 a12a23a31 a12a21a33 a13a21a32 a13a22a31
Determinants
For 4 x 4 and up don't try. For those
interested, expansion by minors and
cofactors is the preferred method.
(However the spaghetti method works
well! Simply duplicate all but the last
column of the matrix next to the
original and sum the products of the
diagonals along the following pattern.)
Spaghetti Method of |A|
a11 a12 a13 a11 a12
a a22 a
a23 21 a22
21
a31 a32
a33 a31 a32
Properties of Determinates
Determinants have several mathematical
properties which are useful in matrix
manipulations.
1 |A|=|A'|.
2. If a row of A = 0, then |A|= 0.
3. If every value in a row is multiplied by k,
then |A| = k|A|.
4. If two rows (or columns) are interchanged
the sign, but not value, of |A| changes.
5. If two rows are identical, |A| = 0.
Properties of Determinates
6. |A| remains unchanged if each
element of a row or each element
multiplied by a constant, is added to any
other row.
7. Det of product = product of Det's
|A| = |A| |B|
8. Det of a diagonal matrix = product
of the diagonal elements
The Inverse of a Matrix (A-1)
For an nxn matrix A, there may be a B such
that AB = I = BA.
The inverse is analogous to a reciprocal)
A matrix which has an inverse is nonsingular.
A matrix which does not have an inverse is
singular.
An inverse exists only if A 0
Inverse by Row or column
operations
Set up a tableau matrix
A tableau for inversions consists of the
matrix to be inverted post multiplied by
a conformable identity matrix.
Matrix Inversion by Tableau
Method
Rules:
You may interchange rows.
You may multiply a row by a scalar.
You may replace a row with the sum of that row
and another row multiplied by a scalar.
Every operation performed on A must be
performed on I
When you are done; A = I & I = A-1
The Tableau Method of Matrix
Inversion: An Example
Step 1: Set up Tableau
1 4 3 1 0 0
2 5 4 0 1 0
1 3 2 0 0 1
Matrix Inversion – cont.
Step 2: Add –2(Row 1) to Row 2
1 4 3 1 0 0
0 3 2 2 1 0
1 3 2 0 0 1
Step 3: Add –1(Row 1) to Row 3
1 4 3 1 0 0
0 3 2 2 1 0
0 7 5 1 0 1
Matrix Inversion – cont.
Step 4: Multiply Row 2 by –1/3
1 4 3 1 0 0
0 1 2 / 3 2 / 3 1 / 3 0
0 7 5 1
0 1
Step 5: Add –4 (Row 2) to Row 1
1 0 1 / 3 5 / 3 4 / 3 0
0 1 2 / 3 2 / 3 1 / 3 0
0 7 5 1
0 1
Matrix Inversion – cont.
Step 6: Add 7(Row 2) to Row 3
1 0 1 / 3 5 / 3 4 / 3 0
0 1 2 / 3 2 / 3 1 / 3 0
0 0 1 / 3 11 / 3 7 / 3 1
Step 7: Add Row 3 to Row 1
1 0 0 2 1 1
0 1 2 / 3 2 / 3 1 / 3 0
0 0 1 / 3 11 / 3 7 / 3 1
Matrix Inversion – cont.
Step 9: Add 2(Row 3) to Row 2
1 0 0 2 1 1
0 1 0 8 5 2
0 0 1 / 3 11 / 3 7 / 3 1
Step 9: Multiply Row 3 by -3
1 0 0 2 1 1
0 1 0 8 5 2
0 0 1 11 7 3
Checking the calculation
Remember AA-1=I
1 4 3 2 1 1 1 0 0
2 5 4 8 5 2 0 1 0
1 3 2 11 7 3 0 0 1
Thus
1 * 2 4 * 8 3 * 11 1
1 * 1 4 * 5 3 * 7 0
etc
The Matrix Model
The multiple regression model may be
easily represented in matrix terms.
Y XB e
Where the Y, X, B and e are all matrices
of data, coefficients, or residuals
The Matrix Model (cont.)
The matrices in Y XB e are
represented by
Y1 X 11 X 12 ... X ik B1 e1
Y X X ... X 2 k B e
Y
2
X 21 22 B
2
e
2
... ... ... ...
Yn X n1 X n 2 ... X nk Bk en
Note that we postmultiply X by B since this
order makes them conformable.
The Assumptions of the Model
Scalar Version
1. The ei's are normally distributed.
2. E(ei) = 0
3. E(ei2) = 2
4. E(eiej) = 0 (ij)
5. X's are nonstochastic with values fixed in repeated
samples and (Xik-Xbark)2/n is a finite nonzero
number.
6. The number of observations is greater than the
number of coefficients estimated.
7. No exact linear relationship exists between any of
the explanatory variables.
The Assumptions of the
Model: The Matrix Version
These same assumptions expressed in
matrix format are:
1. e N(0,)
2. = 2I
3. The elements of X are fixed in repeated
samples and (1/ n)X'X is nonsingular and
its elements are finite