Markov Chains & Transition Matrices: exam review Exam review One question at a time

Each week a student studies in the Library (L), at Home (H) or in a Cafe (C). After a week in the library, she studies at home the next week with probability 0.3 and in the cafe with probability 0.2. After a week at home, she studies in the library with probability 0.25 and at home again with probability 0.6. After a week in the cafe, she never studies at home the next week, and she studies in the library with probability 0.4. [6]

1. (a) (a) Write down the transition matrix, with the states in the order L, H, C. [4]

   • M1 for placing the given probabilities with each start state as a column
   • A1 for column L: \(1-0.3-0.2=0.5\), so \((0.5, 0.3, 0.2)\)
   • A1 for columns H and C: \((0.25, 0.6, 0.15)\) and \((0.4, 0, 0.6)\)
   • A1 for the matrix with every column adding to 1

2. (b) (b) Draw the transition diagram. [2]

   • A1 for all 8 non-zero entries drawn as arrows, loops included, and no arrow for a 0
   • A1 for every arrow in the right direction with its probability

A bank classes a loan account each month as Current (C), Late (L) or in Default (D). The transition diagram is shown. [7]

1. (a) (a) Write down the transition matrix, with the states in the order C, L, D. [3]

   • M1 for each start state as a column
   • A1 for the first two columns
   • A1 for the third column \((0,0,1)\)

2. (b) (b) Write down the absorbing state and say how the matrix shows it. [1]

   • A1 for D, with a 1 on the diagonal of its column (it never leaves)

3. (c) (c) A loan account is current this month. Find the probability that the account is in default after 3 months. [3]

   • M1 for using \(\mathbf{T}^{3}\)
   • M1 for the entry in column C, row D
   • A1 for 0.044

In each lesson a student is either Attentive (A) or Distracted (D). The states in the order A, D have transition matrix \(\mathbf{T}=\begin{pmatrix}0.8 & 0.35 \\ 0.2 & 0.65\end{pmatrix}\) [6]

1. (a) (a) A student is distracted in one lesson. Find the probability that the student is attentive four lessons later. [3]

   • M1 for using \(\mathbf{T}^{4}\)
   • M1 for the entry in column D, row A
   • A1 for 0.61

2. (b) (b) At the start of a class, 70% of the students are attentive. Find the percentage of students in each state three lessons later. [3]

   • M1 for \(\mathbf{s}_{3}=\mathbf{T}^{3}\mathbf{s}_0\) with \(\mathbf{s}_0=\begin{pmatrix}0.7 \\ 0.3\end{pmatrix}\)
   • A1 for \(\mathbf{T}^{3}=\begin{pmatrix}0.669 & 0.578 \\ 0.331 & 0.422\end{pmatrix}\)
   • A1 for 64.2% and 35.8%

A bike-hire company has three zones, X, Y and Z. Each night its bikes are moved between zones. The transition matrix, with the zones in the order X, Y, Z, is \(\mathbf{T}=\begin{pmatrix}0.7 & 0.2 & 0.1 \\ 0.2 & 0.6 & 0.3 \\ 0.1 & 0.2 & 0.6\end{pmatrix}\) [7]

1. (a) (a) Write down a system of three linear equations that the steady-state vector must satisfy. Use \(\mathbf{T}\mathbf{s}=\mathbf{s}\) and the fact that the shares add to 1. [2]

   • M1 for \(\mathbf{T}\mathbf{s}=\mathbf{s}\) written as equations in \(s_1, s_2, s_3\)
   • A1 for two correct row equations and \(s_1+s_2+s_3=1\)

2. (b) (b) Solve your system to find the exact steady-state vector. [3]

   • M1 for solving the system on the calculator
   • A1 for two correct fractions
   • A1 for \(\mathbf{s}=\begin{pmatrix}\frac{10}{29} \\ \frac{11}{29} \\ \frac{8}{29}\end{pmatrix}\)

3. (c) (c) The company has 1500 bikes. In the long term, how many are in zone X? Give your answer to the nearest whole number. [2]

   • M1 for \(1500\times \frac{10}{29}\)
   • A1 for 517 (nearest whole number; \(517.24\) before rounding)

A commuter travels to work each day by Bus (B) or by Train (R). The choice on one day depends only on the choice on the previous day. The transition diagram is shown. [6]

1. (a) (a) Write down the transition matrix, with the states in the order B, R. [2]

   • M1 for each start state as a column
   • A1 for the matrix with every column adding to 1

2. (b) (b) The commuter takes the bus on Monday. Find the probability that she takes the train on Tuesday and on Wednesday and the bus on Thursday. [2]

   • M1 for multiplying the three entries: B to R, R to R, R to B
   • A1 for 0.056

3. (c) (c) The commuter takes the bus on Monday. Find the probability that she takes the train on Wednesday. [2]

   • M1 for using \(\mathbf{T}^{2}\)
   • A1 for the entry in column B, row R: 0.507

The members of a gym either Attend (A) or Miss (M) each week. 80% of the members who attend in one week attend the next week. 35% of the members who miss in one week attend the next week. There are 600 members. This week 400 of them attend. [8]

1. (a) (a) Write down the transition matrix, with the states in the order A, M. [2]

   • M1 for each start state as a column
   • A1 for the second column found from \(1-0.35\) and \(1-0.8\) (columns add to 1)

2. (b) (b) Find the number of members expected to attend in 4 weeks' time. [2]

   • M1 for \(\mathbf{T}^{4}\) multiplied by the initial vector \(\begin{pmatrix}400\\200\end{pmatrix}\)
   • A1 for 383 in the first state

3. (c) (c) Use a high power of \(\mathbf{T}\) to find the number of members expected to attend in the long term. [2]

   • M1 for a high power of \(\mathbf{T}\) (such as \(\mathbf{T}^{200}\)) times the initial vector, until the values stop changing
   • A1 for 382

4. (d) (d) Show, by solving \(\mathbf{T}\mathbf{s}=\mathbf{s}\), that the long-term proportion in the first state is exactly \(\frac{7}{11}\). [2]

   • M1 for \(\mathbf{T}\mathbf{s}=\mathbf{s}\) with \(\mathbf{s}=\begin{pmatrix}x\\1-x\end{pmatrix}\), giving \(0.8x+0.35(1-x)=x\)
   • A1 for \(x=\frac{7}{11}\)

A football team's match results are Win (W), Draw (D) or Loss (L). The result of a match depends only on the result of the previous match. After a win, the team wins its next match with probability 0.5 and draws with probability 0.3. After a draw, it wins with probability 0.35 and loses with probability 0.25. After a loss, it wins with probability 0.2 and draws with probability 0.3. [14]

1. (a) (a) Write down the transition matrix, with the states in the order W, D, L. [4]

   • M1 for each start state as a column
   • A1 for the first column
   • A1 for the second column
   • A1 for the third column (using column sums for the missing entry)

2. (b) (b) The team has just won a match. Find the probability that it wins the match after next. [2]

   • M1 for using \(\mathbf{T}^{2}\) and the entry in column W, row W
   • A1 for 0.395

3. (c) (c) In its last match the probabilities of a win, a draw and a loss were 0.6, 0.3 and 0.1. Find the probabilities of each result in the match five matches after that one. [3]

   • M1 for \(\mathbf{s}_{5}=\mathbf{T}^{5}\mathbf{s}_0\) with \(\mathbf{s}_0=\begin{pmatrix}0.6 \\ 0.3 \\ 0.1\end{pmatrix}\)
   • A1 for \(\mathbf{T}^{5}=\begin{pmatrix}0.358 & 0.357 & 0.356 \\ 0.333 & 0.333 & 0.333 \\ 0.309 & 0.309 & 0.311\end{pmatrix}\)
   • A1 for \(\begin{pmatrix}0.358 \\ 0.333 \\ 0.309\end{pmatrix}\)

4. (d) (d) Find the long-term matrix, giving its entries to 3 significant figures. [2]

   • M1 for a high power of \(\mathbf{T}\) such as \(\mathbf{T}^{100}\)
   • A1 for the matrix, with three identical columns

5. (e) (e) Write down the long-term percentage of matches that the team wins. [1]

   • A1 for 35.7% (first row of the long-term matrix, times 100)

6. (f) (f) Explain why every column of the long-term matrix is the same. [2]

   • R1 for the long-term probabilities do not depend on where the chain starts
   • R1 for every column being the steady-state vector

A beetle walks along a strip of five squares, numbered 0 to 4. Each minute it moves one square to the right with probability 0.55 and one square to the left with probability 0.45. Squares 0 and 4 are sticky traps: once the beetle is in one it stays there. The beetle starts on square 1. [14]

1. (a) (a) Write down the two absorbing states and say what absorbing means. [2]

   • A1 for the two squares
   • A1 for it stays there once it is in (a 1 on the diagonal of its column)

2. (b) (b) Write down the transition matrix, with the squares in the order 0 to 4. [3]

   • M1 for each start square as a column
   • A1 for the two absorbing columns
   • A1 for the other three columns

3. (c) (c) Find the probability that the beetle is on square 2 after 3 minutes. [2]

   • M1 for \(\mathbf{T}^{3}\) and the entry in column 1, row 2
   • A1 for 0.272

4. (d) (d) Find the probability that the beetle is eventually caught in the trap on square 4. [3]

   • M1 for a high power of \(\mathbf{T}\), such as \(\mathbf{T}^{200}\), until the 3 significant figures stop changing
   • M1 for the entry in column 1, row 4
   • A1 for 0.329

5. (e) (e) Explain why this chain has no single steady state. [2]

   • R1 for the columns of a high power of \(\mathbf{T}\) are not all the same
   • R1 for the long-term probability depends on the starting square

6. (f) (f) 200 beetles start on square 1. How many do you expect to end in the trap on square 4? [2]

   • M1 for \(200\times 0.329\)
   • A1 for 66 (nearest whole number)

A region has farms that are either Organic (O) or Conventional (C). Each year 15% of the organic farms change to conventional and 10% of the conventional farms change to organic. No farms leave the region and none join it. [15]

1. (a) (a) Write down the transition matrix \(\mathbf{T}\), with the states in the order O, C. [2]

   • M1 for each start state as a column
   • A1 for \(\mathbf{T}=\begin{pmatrix}0.85 & 0.1 \\ 0.15 & 0.9\end{pmatrix}\)

2. (b) (b) Find the eigenvalues of \(\mathbf{T}\). [3]

   • M1 for \(\det(\mathbf{T}-\lambda\mathbf{I})=0\): \((1-0.15-\lambda)(1-0.1-\lambda)-0.15\times0.1=0\)
   • A1 for \(\lambda=1\)
   • A1 for \(\lambda=0.75\)

3. (c) (c) Find an eigenvector for each eigenvalue. [3]

   • M1 for solving \((\mathbf{T}-\lambda\mathbf{I})\mathbf{v}=\mathbf{0}\)
   • A1 for an eigenvector for \(\lambda=1\): \(\begin{pmatrix}2 \\ 3\end{pmatrix}\)
   • A1 for an eigenvector for \(\lambda=0.75\): \(\begin{pmatrix}1 \\ -1\end{pmatrix}\)

4. (d) (d) Write down matrices \(\mathbf{P}\) and \(\mathbf{D}\) such that \(\mathbf{T}=\mathbf{PDP}^{-1}\), where \(\mathbf{D}\) is diagonal. [2]

   • A1 for \(\mathbf{P}=\begin{pmatrix}2 & 1 \\ 3 & -1\end{pmatrix}\) (eigenvectors as columns)
   • A1 for \(\mathbf{D}=\begin{pmatrix}1 & 0 \\ 0 & 0.75\end{pmatrix}\) in the same order

5. (e) (e) At the start, 60% of the farms are organic. Use \(\mathbf{T}^n=\mathbf{P}\mathbf{D}^n\mathbf{P}^{-1}\) to find an expression for the proportion of organic farms after \(n\) years. [3]

   • M1 for \(\mathbf{s}_n=\mathbf{P}\mathbf{D}^n\mathbf{P}^{-1}\mathbf{s}_0\) with \(\mathbf{s}_0=\begin{pmatrix}0.6 \\ 0.4\end{pmatrix}\)
   • A1 for \(\mathbf{D}^n=\begin{pmatrix}1 & 0 \\ 0 & (0.75)^n\end{pmatrix}\)
   • A1 for \(0.4+0.2(0.75)^n\)

6. (f) (f) Hence write down the long-term proportion of organic farms. [2]

   • M1 for \((0.75)^n\to 0\) as \(n\to\infty\)
   • A1 for 0.4

A museum has four rooms, A, B, C and D. Every 10 minutes each visitor stays in the same room or moves to another room. The probabilities of moving are known for a visitor in each room. From A: to B 0.2, to C 0.1, to D 0.25. From B: to A 0.3, to C 0.25, and never to D. From C: to A 0.15, to B 0.2, to D 0.35. From D: to A 0.1, to B 0.3, to C 0.2. [12]

1. (a) (a) Write down the transition matrix, with the rooms in the order A, B, C, D. [3]

   • M1 for each start state as a column and a missing entry from a column sum
   • A1 for two columns correct
   • A1 for the matrix

2. (b) (b) A visitor is in room A. Find the probability that the visitor is in room D 30 minutes later. [2]

   • M1 for \(\mathbf{T}^{3}\)
   • A1 for the entry in column A, row D: 0.236

3. (c) (c) At 10:00 there are 400 visitors in room A, 300 in B, 200 in C and 100 in D. Find the number of visitors in each room at 10:30, assuming the same 1000 visitors stay in the museum. [3]

   • M1 for \(\mathbf{s}_{3}=\mathbf{T}^{3}\mathbf{s}_0\) with \(\mathbf{s}_0=\begin{pmatrix}400 \\ 300 \\ 200 \\ 100\end{pmatrix}\)
   • A1 for correct values before rounding: \(\begin{pmatrix}264.9 \\ 296 \\ 207.4 \\ 231.7\end{pmatrix}\)
   • A1 for \(\begin{pmatrix}265 \\ 296 \\ 207 \\ 232\end{pmatrix}\)

4. (d) (d) Find the exact steady-state vector. [4]

   • M1 for \(\mathbf{T}\mathbf{s}=\mathbf{s}\) as equations in \(s_1, s_2, s_3, s_4\)
   • M1 for using \(s_1+s_2+s_3+s_4=1\) in place of one equation
   • M1 for solving on the calculator
   • A1 for \(\mathbf{s}=\begin{pmatrix}\frac{545}{2084} \\ \frac{155}{521} \\ \frac{437}{2084} \\ \frac{241}{1042}\end{pmatrix}\)

A farmer grows Wheat (W), Barley (B) or Oats (O) on a field. Each year the crop depends only on the crop of the previous year. After wheat, the field is never sown with wheat again: it is sown with barley with probability 0.6 and with oats otherwise. After barley, the field is sown with wheat with probability 0.5 and with barley with probability 0.2. After oats, the field is sown with wheat with probability 0.3 and with barley with probability 0.4. [9]

1. (a) (a) Write down the transition matrix, with the states in the order W, B, O. [3]

   • M1 for each given probability placed with the start state as a column
   • A1 for the column that is found by subtraction from 1
   • A1 for the matrix with every column adding to 1

2. (b) (b) Show that the chain is regular, and state the smallest power of \(\mathbf{T}\) with every entry positive. [2]

   • M1 for \(\mathbf{T}^{2}\) with every entry positive
   • A1 for 2 (\(\mathbf{T}\) itself has a zero entry, so 1 is not enough)

3. (c) (c) Find the eigenvalue 1 of \(\mathbf{T}\) and a corresponding eigenvector. Hence write down the steady state vector, giving its entries to 3 decimal places. [4]

   • M1 for using technology to find the eigenvalue 1 and its eigenvector
   • A1 for an eigenvector in proportion to (0.880, 1.160, 1.000)
   • M1 for dividing by the sum of its entries so the entries add to 1
   • A1 for (0.289, 0.382, 0.329)

A museum guards a collection by moving a security robot between four rooms, A, B, C and D, every 10 minutes. The transition diagram is shown. [7]

1. (a) (a) Write down the transition matrix, with the states in the order A, B, C, D. [3]

   • M1 for each start state as a column
   • A1 for three correct columns
   • A1 for the matrix with every column adding to 1

2. (b) (b) Find the probability that a robot in D is in B after 2 moves. [2]

   • M1 for using \(\mathbf{T}^{2}\) and the entry in column D, row B
   • A1 for 0.3

3. (c) (c) Use technology to find the long-term probability that a robot is in C. [2]

   • M1 for a high power of \(\mathbf{T}\) (such as \(\mathbf{T}^{200}\)) until the entries stop changing
   • A1 for 0.321