10.2 Regular Markov Chains

					 Learning Objectives for Section 9.2
       Regular Markov Chains

 The student will be able to determine the stationary matrix for
  a given transition matrix.
 The student will be able to identify regular Markov chains.
 The student will be able to solve applications using Markov
  chains.
 The student will be able to find graphing utility
  approximations of the stationary matrix.




Barnett/Ziegler/Byleen Finite Mathematics 11e                       1
                   Regular Markov Chains




      In this section, we will study what happens to the entries in
       the kth state matrix as the number of trials increases. We
      wish to determine the long-run behavior of the both the state
           matrices and the powers of the transition matrix P.



Barnett/Ziegler/Byleen Finite Mathematics 11e                         2
                   The Stationary Matrix

 When we computed the fourth state matrix of a previous
  problem we saw that the numbers appeared to approaching
  fixed values. Recall,
                                                4
                                0.98 0.02
       S 4  S0 P  0.90 0.10 
                 4
                                            0.97488 0.02512
                                0.78 0.22

 If we calculated the 5th , 6th and and kth state matrix, we would
  find that they approach a limiting matrix of
               [0.975 0.025]
  This final matrix is called the stationary matrix.

Barnett/Ziegler/Byleen Finite Mathematics 11e                     3
                         Stationary Matrix
                            (continued)

 The stationary matrix S for a Markov chain with transition
  matrix P has the property that
                              SP = S.
 To prove that the matrix [0.975 0.025] is the stationary matrix,
  we need to verify this property:

                      0.98 0.02
        0.975 0.025            0.975 0.025
                      0.78 0.22
 We find this statement to be true, so the stationary matrix is
  indeed [0.975 0.025].

Barnett/Ziegler/Byleen Finite Mathematics 11e                      4
                         Stationary Matrix
                          Interpretation

 The stationary matrix means that in the long-run the system
  will be at a steady state. Later states will change very little, if
  at all. In the example, in the long run the number of low-risk
  drivers will be 0.975 and the number of high-risk drivers will
  be 0.025.
 The question of whether or not every Markov chain has a
  unique stationary matrix can be answered - it is no. However,
  if a Markov chain is regular, then it will have a unique
  stationary matrix and successive state matrices will always
  approach this stationary matrix.


Barnett/Ziegler/Byleen Finite Mathematics 11e                       5
                  Regular Markov Chains

 A transition matrix P is regular if some power of P has only
  positive entries. A Markov chain is a regular Markov chain
  if its transition matrix is regular.
 For example, for the matrix D given below, D2 has only
  positive entries, so D is regular.
 Note that the entries in P, and all powers of P, are always ≥ 0,
  since they represent probabilities. So, “positive” basically
  means “nonzero”.

             0.3                0.7               .79 .210
           D                                 D 
                                                 2
                                                             
              1                  0               0.30 0.70
Barnett/Ziegler/Byleen Finite Mathematics 11e                        6
                      Properties of
                  Regular Markov Chains

 Theorem
    If a Markov chain is regular, then there is a unique stationary
    matrix S that satisfies
      SP = S
      The sum of its entries is 1


 The stationary matrix can be found by solving these equations.




Barnett/Ziegler/Byleen Finite Mathematics 11e                         7
         Finding the Stationary Matrix
                   Example

                                     0.3 0.7
 Find the stationary matrix for P  
                                      1   0




Barnett/Ziegler/Byleen Finite Mathematics 11e    8
         Finding the Stationary Matrix
                   Example

                                      0.3 0.7
 Find the stationary matrix for P  
                                       1   0
 Solution: s1 s2  
                       0.3 0.7
                      1         s1 s2 
                            0
                    0.3s1  s2        0.7 s1  0  s1   s2 

 Look at the first entry: 0.3s1  s2  s1  s2  0.7 s1
 The second entry leads to the same result. We use the other
  property s1  s2  1 to find the solution
                                S  0.5882        0.4118 
Barnett/Ziegler/Byleen Finite Mathematics 11e                     9
                         Limiting Matrix P*

 According to Theorem 1 of this section, the state matrices Sk
  will approach the stationary matrix S, and the matrices given
  by successive powers of P approach a limiting matrix P*
  where each row of P* is equal to the stationary matrix S.

 Example:
                          S=        0.5882     0.4118

                                0.5881 0.4112
                           P*  
                                 0.5881 0.4112
                                               
Barnett/Ziegler/Byleen Finite Mathematics 11e                     10
                                  Application

A company rates every employee as below average, average, or
above average. Past performance indicates that each year 10%
of the below-average employees will raise their rating to
average and 25% of the average employees will raise their
rating to above average. On the other hand, 15% of the average
employees will lower their rating to below average and 15% of
the above-average employees will lower their rating to average.
Company policy prohibits rating changes from below average
to above average, or conversely, in a single year. Over the long
run, what percentage of employees will receive below-average
ratings? Average ratings? Above-average ratings?

Barnett/Ziegler/Byleen Finite Mathematics 11e                  11
                                  Application
                                  (continued)
                                                                    Next year
 First, we find the transition matrix.                            A- A A+
                                                            A-    0.9 0.1    0 
  A- = below average                            This              0.15 0.6 0.25
                                                            A                   
  A = average                                   year
                                                            A+    0
                                                                       0.15 0.85
                                                                                 
  A+ = above average
 To determine what happens over the long run, we find the
 stationary matrix by solving the following system:
                       0.9 0.1   0 
    s1    s2    s3  0.15 0.6 0.25   s1
                                                     s2   s3  , s1  s2  s3  1
                       0
                           0.15 0.85
                                     

Barnett/Ziegler/Byleen Finite Mathematics 11e                                         12
                                  Application
                                  (continued)

 This is equivalent to                          .9 s1  .15s2      =s1

                                                .1s1  .6 s2  .15s3  s2
                                                      .2s 2 +.85s 3 =s 3
                                                s1  s2  s3  1

  Using Gauss-Jordan elimination to solve this system of four
  equations with three variables, we obtain
                 s1 = 0.36 s2 = 0.24 s3 = 0.4
  Thus, in the long run, 36% of the employees will be rated as
  below average, 24% as average, and 40% as above average.
Barnett/Ziegler/Byleen Finite Mathematics 11e                               13
         Approximating the Stationary
             Matrix by Calculator

                                        Suppose we want to
                                        approximate the stationary
                                        matrix S for the transition
                                        matrix A on the left by
                                        computing powers of A with a
                                        graphing calculator.
                                        We can see that when we
                                        compute A20, the values in
                                        each column are the same to 4
                                        decimal places, so this is the
                                        stationary matrix.
Barnett/Ziegler/Byleen Finite Mathematics 11e                            14

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:15
posted:1/17/2012
language:English
pages:14