Markov Chains & Transition Matrices: exam review Exam review

Each week a student studies in the Library (L), at Home (H) or in a Cafe (C). After a week in the library, she studies at home the next week with probability 0.3 and in the cafe with probability 0.2. After a week at home, she studies in the library with probability 0.25 and at home again with probability 0.6. After a week in the cafe, she never studies at home the next week, and she studies in the library with probability 0.4. [6]

1. (a) (a) Write down the transition matrix, with the states in the order L, H, C. [4]
2. (b) (b) Draw the transition diagram. [2]

A bank classes a loan account each month as Current (C), Late (L) or in Default (D). The transition diagram is shown. [7]

1. (a) (a) Write down the transition matrix, with the states in the order C, L, D. [3]
2. (b) (b) Write down the absorbing state and say how the matrix shows it. [1]
3. (c) (c) A loan account is current this month. Find the probability that the account is in default after 3 months. [3]

In each lesson a student is either Attentive (A) or Distracted (D). The states in the order A, D have transition matrix \(\mathbf{T}=\begin{pmatrix}0.8 & 0.35 \\ 0.2 & 0.65\end{pmatrix}\) [6]

1. (a) (a) A student is distracted in one lesson. Find the probability that the student is attentive four lessons later. [3]
2. (b) (b) At the start of a class, 70% of the students are attentive. Find the percentage of students in each state three lessons later. [3]

A bike-hire company has three zones, X, Y and Z. Each night its bikes are moved between zones. The transition matrix, with the zones in the order X, Y, Z, is \(\mathbf{T}=\begin{pmatrix}0.7 & 0.2 & 0.1 \\ 0.2 & 0.6 & 0.3 \\ 0.1 & 0.2 & 0.6\end{pmatrix}\) [7]

1. (a) (a) Write down a system of three linear equations that the steady-state vector must satisfy. Use \(\mathbf{T}\mathbf{s}=\mathbf{s}\) and the fact that the shares add to 1. [2]
2. (b) (b) Solve your system to find the exact steady-state vector. [3]
3. (c) (c) The company has 1500 bikes. In the long term, how many are in zone X? Give your answer to the nearest whole number. [2]

A commuter travels to work each day by Bus (B) or by Train (R). The choice on one day depends only on the choice on the previous day. The transition diagram is shown. [6]

1. (a) (a) Write down the transition matrix, with the states in the order B, R. [2]
2. (b) (b) The commuter takes the bus on Monday. Find the probability that she takes the train on Tuesday and on Wednesday and the bus on Thursday. [2]
3. (c) (c) The commuter takes the bus on Monday. Find the probability that she takes the train on Wednesday. [2]

The members of a gym either Attend (A) or Miss (M) each week. 80% of the members who attend in one week attend the next week. 35% of the members who miss in one week attend the next week. There are 600 members. This week 400 of them attend. [8]

1. (a) (a) Write down the transition matrix, with the states in the order A, M. [2]
2. (b) (b) Find the number of members expected to attend in 4 weeks' time. [2]
3. (c) (c) Use a high power of \(\mathbf{T}\) to find the number of members expected to attend in the long term. [2]
4. (d) (d) Show, by solving \(\mathbf{T}\mathbf{s}=\mathbf{s}\), that the long-term proportion in the first state is exactly \(\frac{7}{11}\). [2]

A football team's match results are Win (W), Draw (D) or Loss (L). The result of a match depends only on the result of the previous match. After a win, the team wins its next match with probability 0.5 and draws with probability 0.3. After a draw, it wins with probability 0.35 and loses with probability 0.25. After a loss, it wins with probability 0.2 and draws with probability 0.3. [14]

1. (a) (a) Write down the transition matrix, with the states in the order W, D, L. [4]
2. (b) (b) The team has just won a match. Find the probability that it wins the match after next. [2]
3. (c) (c) In its last match the probabilities of a win, a draw and a loss were 0.6, 0.3 and 0.1. Find the probabilities of each result in the match five matches after that one. [3]
4. (d) (d) Find the long-term matrix, giving its entries to 3 significant figures. [2]
5. (e) (e) Write down the long-term percentage of matches that the team wins. [1]
6. (f) (f) Explain why every column of the long-term matrix is the same. [2]

A beetle walks along a strip of five squares, numbered 0 to 4. Each minute it moves one square to the right with probability 0.55 and one square to the left with probability 0.45. Squares 0 and 4 are sticky traps: once the beetle is in one it stays there. The beetle starts on square 1. [14]

1. (a) (a) Write down the two absorbing states and say what absorbing means. [2]
2. (b) (b) Write down the transition matrix, with the squares in the order 0 to 4. [3]
3. (c) (c) Find the probability that the beetle is on square 2 after 3 minutes. [2]
4. (d) (d) Find the probability that the beetle is eventually caught in the trap on square 4. [3]
5. (e) (e) Explain why this chain has no single steady state. [2]
6. (f) (f) 200 beetles start on square 1. How many do you expect to end in the trap on square 4? [2]

A region has farms that are either Organic (O) or Conventional (C). Each year 15% of the organic farms change to conventional and 10% of the conventional farms change to organic. No farms leave the region and none join it. [15]

1. (a) (a) Write down the transition matrix \(\mathbf{T}\), with the states in the order O, C. [2]
2. (b) (b) Find the eigenvalues of \(\mathbf{T}\). [3]
3. (c) (c) Find an eigenvector for each eigenvalue. [3]
4. (d) (d) Write down matrices \(\mathbf{P}\) and \(\mathbf{D}\) such that \(\mathbf{T}=\mathbf{PDP}^{-1}\), where \(\mathbf{D}\) is diagonal. [2]
5. (e) (e) At the start, 60% of the farms are organic. Use \(\mathbf{T}^n=\mathbf{P}\mathbf{D}^n\mathbf{P}^{-1}\) to find an expression for the proportion of organic farms after \(n\) years. [3]
6. (f) (f) Hence write down the long-term proportion of organic farms. [2]

A museum has four rooms, A, B, C and D. Every 10 minutes each visitor stays in the same room or moves to another room. The probabilities of moving are known for a visitor in each room. From A: to B 0.2, to C 0.1, to D 0.25. From B: to A 0.3, to C 0.25, and never to D. From C: to A 0.15, to B 0.2, to D 0.35. From D: to A 0.1, to B 0.3, to C 0.2. [12]

1. (a) (a) Write down the transition matrix, with the rooms in the order A, B, C, D. [3]
2. (b) (b) A visitor is in room A. Find the probability that the visitor is in room D 30 minutes later. [2]
3. (c) (c) At 10:00 there are 400 visitors in room A, 300 in B, 200 in C and 100 in D. Find the number of visitors in each room at 10:30, assuming the same 1000 visitors stay in the museum. [3]
4. (d) (d) Find the exact steady-state vector. [4]

A farmer grows Wheat (W), Barley (B) or Oats (O) on a field. Each year the crop depends only on the crop of the previous year. After wheat, the field is never sown with wheat again: it is sown with barley with probability 0.6 and with oats otherwise. After barley, the field is sown with wheat with probability 0.5 and with barley with probability 0.2. After oats, the field is sown with wheat with probability 0.3 and with barley with probability 0.4. [9]

1. (a) (a) Write down the transition matrix, with the states in the order W, B, O. [3]
2. (b) (b) Show that the chain is regular, and state the smallest power of \(\mathbf{T}\) with every entry positive. [2]
3. (c) (c) Find the eigenvalue 1 of \(\mathbf{T}\) and a corresponding eigenvector. Hence write down the steady state vector, giving its entries to 3 decimal places. [4]

A museum guards a collection by moving a security robot between four rooms, A, B, C and D, every 10 minutes. The transition diagram is shown. [7]

1. (a) (a) Write down the transition matrix, with the states in the order A, B, C, D. [3]
2. (b) (b) Find the probability that a robot in D is in B after 2 moves. [2]
3. (c) (c) Use technology to find the long-term probability that a robot is in C. [2]