Learning Objectives for Section 9.2 Regular Markov Chains The student will be able to determine the stationary matrix for a given transition matrix. The student will be able to identify regular Markov chains. The student will be able to solve applications using Markov chains. The student will be able to find graphing utility approximations of the stationary matrix. Barnett/Ziegler/Byleen Finite Mathematics 11e 1 Regular Markov Chains In this section, we will study what happens to the entries in the kth state matrix as the number of trials increases. We wish to determine the long-run behavior of the both the state matrices and the powers of the transition matrix P. Barnett/Ziegler/Byleen Finite Mathematics 11e 2 The Stationary Matrix When we computed the fourth state matrix of a previous problem we saw that the numbers appeared to approaching fixed values. Recall, 4 0.98 0.02 S 4 S0 P 0.90 0.10 4 0.97488 0.02512 0.78 0.22 If we calculated the 5th , 6th and and kth state matrix, we would find that they approach a limiting matrix of [0.975 0.025] This final matrix is called the stationary matrix. Barnett/Ziegler/Byleen Finite Mathematics 11e 3 Stationary Matrix (continued) The stationary matrix S for a Markov chain with transition matrix P has the property that SP = S. To prove that the matrix [0.975 0.025] is the stationary matrix, we need to verify this property: 0.98 0.02 0.975 0.025 0.975 0.025 0.78 0.22 We find this statement to be true, so the stationary matrix is indeed [0.975 0.025]. Barnett/Ziegler/Byleen Finite Mathematics 11e 4 Stationary Matrix Interpretation The stationary matrix means that in the long-run the system will be at a steady state. Later states will change very little, if at all. In the example, in the long run the number of low-risk drivers will be 0.975 and the number of high-risk drivers will be 0.025. The question of whether or not every Markov chain has a unique stationary matrix can be answered - it is no. However, if a Markov chain is regular, then it will have a unique stationary matrix and successive state matrices will always approach this stationary matrix. Barnett/Ziegler/Byleen Finite Mathematics 11e 5 Regular Markov Chains A transition matrix P is regular if some power of P has only positive entries. A Markov chain is a regular Markov chain if its transition matrix is regular. For example, for the matrix D given below, D2 has only positive entries, so D is regular. Note that the entries in P, and all powers of P, are always ≥ 0, since they represent probabilities. So, “positive” basically means “nonzero”. 0.3 0.7 .79 .210 D D 2 1 0 0.30 0.70 Barnett/Ziegler/Byleen Finite Mathematics 11e 6 Properties of Regular Markov Chains Theorem If a Markov chain is regular, then there is a unique stationary matrix S that satisfies SP = S The sum of its entries is 1 The stationary matrix can be found by solving these equations. Barnett/Ziegler/Byleen Finite Mathematics 11e 7 Finding the Stationary Matrix Example 0.3 0.7 Find the stationary matrix for P 1 0 Barnett/Ziegler/Byleen Finite Mathematics 11e 8 Finding the Stationary Matrix Example 0.3 0.7 Find the stationary matrix for P 1 0 Solution: s1 s2 0.3 0.7 1 s1 s2 0 0.3s1 s2 0.7 s1 0 s1 s2 Look at the first entry: 0.3s1 s2 s1 s2 0.7 s1 The second entry leads to the same result. We use the other property s1 s2 1 to find the solution S 0.5882 0.4118 Barnett/Ziegler/Byleen Finite Mathematics 11e 9 Limiting Matrix P* According to Theorem 1 of this section, the state matrices Sk will approach the stationary matrix S, and the matrices given by successive powers of P approach a limiting matrix P* where each row of P* is equal to the stationary matrix S. Example: S= 0.5882 0.4118 0.5881 0.4112 P* 0.5881 0.4112 Barnett/Ziegler/Byleen Finite Mathematics 11e 10 Application A company rates every employee as below average, average, or above average. Past performance indicates that each year 10% of the below-average employees will raise their rating to average and 25% of the average employees will raise their rating to above average. On the other hand, 15% of the average employees will lower their rating to below average and 15% of the above-average employees will lower their rating to average. Company policy prohibits rating changes from below average to above average, or conversely, in a single year. Over the long run, what percentage of employees will receive below-average ratings? Average ratings? Above-average ratings? Barnett/Ziegler/Byleen Finite Mathematics 11e 11 Application (continued) Next year First, we find the transition matrix. A- A A+ A- 0.9 0.1 0 A- = below average This 0.15 0.6 0.25 A A = average year A+ 0 0.15 0.85 A+ = above average To determine what happens over the long run, we find the stationary matrix by solving the following system: 0.9 0.1 0 s1 s2 s3 0.15 0.6 0.25 s1 s2 s3 , s1 s2 s3 1 0 0.15 0.85 Barnett/Ziegler/Byleen Finite Mathematics 11e 12 Application (continued) This is equivalent to .9 s1 .15s2 =s1 .1s1 .6 s2 .15s3 s2 .2s 2 +.85s 3 =s 3 s1 s2 s3 1 Using Gauss-Jordan elimination to solve this system of four equations with three variables, we obtain s1 = 0.36 s2 = 0.24 s3 = 0.4 Thus, in the long run, 36% of the employees will be rated as below average, 24% as average, and 40% as above average. Barnett/Ziegler/Byleen Finite Mathematics 11e 13 Approximating the Stationary Matrix by Calculator Suppose we want to approximate the stationary matrix S for the transition matrix A on the left by computing powers of A with a graphing calculator. We can see that when we compute A20, the values in each column are the same to 4 decimal places, so this is the stationary matrix. Barnett/Ziegler/Byleen Finite Mathematics 11e 14

DOCUMENT INFO

Shared By:

Categories:

Tags:

Stats:

views: | 15 |

posted: | 1/17/2012 |

language: | English |

pages: | 14 |

OTHER DOCS BY ewghwehws

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.