# 10.2 Regular Markov Chains

```					 Learning Objectives for Section 9.2
Regular Markov Chains

 The student will be able to determine the stationary matrix for
a given transition matrix.
 The student will be able to identify regular Markov chains.
 The student will be able to solve applications using Markov
chains.
 The student will be able to find graphing utility
approximations of the stationary matrix.

Barnett/Ziegler/Byleen Finite Mathematics 11e                       1
Regular Markov Chains

In this section, we will study what happens to the entries in
the kth state matrix as the number of trials increases. We
wish to determine the long-run behavior of the both the state
matrices and the powers of the transition matrix P.

Barnett/Ziegler/Byleen Finite Mathematics 11e                         2
The Stationary Matrix

 When we computed the fourth state matrix of a previous
problem we saw that the numbers appeared to approaching
fixed values. Recall,
4
0.98 0.02
S 4  S0 P  0.90 0.10 
4
  0.97488 0.02512
0.78 0.22

 If we calculated the 5th , 6th and and kth state matrix, we would
find that they approach a limiting matrix of
[0.975 0.025]
This final matrix is called the stationary matrix.

Barnett/Ziegler/Byleen Finite Mathematics 11e                     3
Stationary Matrix
(continued)

 The stationary matrix S for a Markov chain with transition
matrix P has the property that
SP = S.
 To prove that the matrix [0.975 0.025] is the stationary matrix,
we need to verify this property:

0.98 0.02
0.975 0.025            0.975 0.025
0.78 0.22
 We find this statement to be true, so the stationary matrix is
indeed [0.975 0.025].

Barnett/Ziegler/Byleen Finite Mathematics 11e                      4
Stationary Matrix
Interpretation

 The stationary matrix means that in the long-run the system
will be at a steady state. Later states will change very little, if
at all. In the example, in the long run the number of low-risk
drivers will be 0.975 and the number of high-risk drivers will
be 0.025.
 The question of whether or not every Markov chain has a
unique stationary matrix can be answered - it is no. However,
if a Markov chain is regular, then it will have a unique
stationary matrix and successive state matrices will always
approach this stationary matrix.

Barnett/Ziegler/Byleen Finite Mathematics 11e                       5
Regular Markov Chains

 A transition matrix P is regular if some power of P has only
positive entries. A Markov chain is a regular Markov chain
if its transition matrix is regular.
 For example, for the matrix D given below, D2 has only
positive entries, so D is regular.
 Note that the entries in P, and all powers of P, are always ≥ 0,
since they represent probabilities. So, “positive” basically
means “nonzero”.

0.3                0.7               .79 .210
D                                 D 
2

 1                  0               0.30 0.70
Barnett/Ziegler/Byleen Finite Mathematics 11e                        6
Properties of
Regular Markov Chains

 Theorem
If a Markov chain is regular, then there is a unique stationary
matrix S that satisfies
 SP = S
 The sum of its entries is 1

 The stationary matrix can be found by solving these equations.

Barnett/Ziegler/Byleen Finite Mathematics 11e                         7
Finding the Stationary Matrix
Example

0.3 0.7
 Find the stationary matrix for P  
 1   0

Barnett/Ziegler/Byleen Finite Mathematics 11e    8
Finding the Stationary Matrix
Example

0.3 0.7
 Find the stationary matrix for P  
 1   0
 Solution: s1 s2  
0.3 0.7
1         s1 s2 
      0
0.3s1  s2        0.7 s1  0  s1   s2 

 Look at the first entry: 0.3s1  s2  s1  s2  0.7 s1
 The second entry leads to the same result. We use the other
property s1  s2  1 to find the solution
S  0.5882        0.4118 
Barnett/Ziegler/Byleen Finite Mathematics 11e                     9
Limiting Matrix P*

 According to Theorem 1 of this section, the state matrices Sk
will approach the stationary matrix S, and the matrices given
by successive powers of P approach a limiting matrix P*
where each row of P* is equal to the stationary matrix S.

 Example:
S=        0.5882     0.4118

0.5881 0.4112
P*  
 0.5881 0.4112

Barnett/Ziegler/Byleen Finite Mathematics 11e                     10
Application

A company rates every employee as below average, average, or
above average. Past performance indicates that each year 10%
of the below-average employees will raise their rating to
average and 25% of the average employees will raise their
rating to above average. On the other hand, 15% of the average
employees will lower their rating to below average and 15% of
the above-average employees will lower their rating to average.
Company policy prohibits rating changes from below average
to above average, or conversely, in a single year. Over the long
run, what percentage of employees will receive below-average
ratings? Average ratings? Above-average ratings?

Barnett/Ziegler/Byleen Finite Mathematics 11e                  11
Application
(continued)
Next year
First, we find the transition matrix.                            A- A A+
A-    0.9 0.1    0 
A- = below average                            This              0.15 0.6 0.25
A                   
A = average                                   year
A+    0
      0.15 0.85

A+ = above average
To determine what happens over the long run, we find the
stationary matrix by solving the following system:
 0.9 0.1   0 
 s1    s2    s3  0.15 0.6 0.25   s1
                               s2   s3  , s1  s2  s3  1
 0
     0.15 0.85


Barnett/Ziegler/Byleen Finite Mathematics 11e                                         12
Application
(continued)

This is equivalent to                          .9 s1  .15s2      =s1

.1s1  .6 s2  .15s3  s2
.2s 2 +.85s 3 =s 3
s1  s2  s3  1

Using Gauss-Jordan elimination to solve this system of four
equations with three variables, we obtain
s1 = 0.36 s2 = 0.24 s3 = 0.4
Thus, in the long run, 36% of the employees will be rated as
below average, 24% as average, and 40% as above average.
Barnett/Ziegler/Byleen Finite Mathematics 11e                               13
Approximating the Stationary
Matrix by Calculator

Suppose we want to
approximate the stationary
matrix S for the transition
matrix A on the left by
computing powers of A with a
graphing calculator.
We can see that when we
compute A20, the values in
each column are the same to 4
decimal places, so this is the
stationary matrix.
Barnett/Ziegler/Byleen Finite Mathematics 11e                            14

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 15 posted: 1/17/2012 language: English pages: 14