Transition matrices and diagrams Problem set B Worked solutions

A Markov chain has two states, P and Q. Write down the transition matrix in each case.

1. (a) \(P(P \to P) = 0.58\); \(P(Q \to P) = 0.09\).

   • Column P: \(1 − 0.58 = 0.42\), so \(P(P \to Q) = 0.42\)
   • Column Q: \(1 − 0.09 = 0.91\), so \(P(Q \to Q) = 0.91\)
   • \(\mathbf{T} = \begin{pmatrix} 0.58 & 0.09 \\ 0.42 & 0.91 \end{pmatrix}\)

2. (b) \(P(P \to Q) = 0.76\); \(P(Q \to Q) = 0.33\).

   • Column P: \(1 − 0.76 = 0.24\), so \(P(P \to P) = 0.24\)
   • Column Q: \(1 − 0.33 = 0.67\), so \(P(Q \to P) = 0.67\)
   • \(\mathbf{T} = \begin{pmatrix} 0.24 & 0.67 \\ 0.76 & 0.33 \end{pmatrix}\)

A Markov chain has three states, A, B and C. The known probabilities are: \(P(A \to A) = 0.38\); \(P(A \to B) = 0.47\); \(P(B \to A) = 0.11\); \(P(B \to B) = 0.64\); \(P(C \to B) = 0.21\); \(P(C \to C) = 0.5\). Write down the transition matrix.

• Column A: \(1 − 0.38 − 0.47 = 0.15\), so \(P(A \to C) = 0.15\)
• Column B: \(1 − 0.11 − 0.64 = 0.25\), so \(P(B \to C) = 0.25\)
• Column C: \(1 − 0.21 − 0.5 = 0.29\), so \(P(C \to A) = 0.29\)
• \(\mathbf{T} = \begin{pmatrix} 0.38 & 0.11 & 0.29 \\ 0.47 & 0.64 & 0.21 \\ 0.15 & 0.25 & 0.5 \end{pmatrix}\)

A Markov chain has four states, A, B, C and D. The probabilities of changing state are: \(P(A \to B) = 0.14\); \(P(A \to C) = 0.26\); \(P(A \to D) = 0.19\); \(P(B \to A) = 0.33\); \(P(B \to C) = 0.09\); \(P(B \to D) = 0.3\); \(P(C \to A) = 0.06\); \(P(C \to B) = 0.43\); \(P(C \to D) = 0.34\); \(P(D \to A) = 0.13\); \(P(D \to B) = 0.21\); \(P(D \to C) = 0.52\). Find the transition matrix.

• Column A: \(1 − 0.14 − 0.26 − 0.19 = 0.41\), so \(P(A \to A) = 0.41\)
• Column B: \(1 − 0.33 − 0.09 − 0.3 = 0.28\), so \(P(B \to B) = 0.28\)
• Column C: \(1 − 0.06 − 0.43 − 0.34 = 0.17\), so \(P(C \to C) = 0.17\)
• Column D: \(1 − 0.13 − 0.21 − 0.52 = 0.14\), so \(P(D \to D) = 0.14\)
• \(\mathbf{T} = \begin{pmatrix} 0.41 & 0.33 & 0.06 & 0.13 \\ 0.14 & 0.28 & 0.43 & 0.21 \\ 0.26 & 0.09 & 0.17 & 0.52 \\ 0.19 & 0.3 & 0.34 & 0.14 \end{pmatrix}\)

Draw the transition diagram for each matrix. The rows and columns are in the order of the states shown.

1. (a) States P, Q: \(\mathbf{T} = \begin{pmatrix} 0.81 & 0.46 \\ 0.19 & 0.54 \end{pmatrix}\)

   • Column P: a loop 0.81, an arrow to Q labelled 0.19.
   • Column Q: an arrow to P labelled 0.46, a loop 0.54.
   • Check: the labels leaving each vertex, loop included, add to 1.

2. (b) States A, B, C: \(\mathbf{T} = \begin{pmatrix} 0.58 & 0.27 & 0.12 \\ 0.42 & 0.23 & 0 \\ 0 & 0.5 & 0.88 \end{pmatrix}\)

   • Column A: a loop 0.58, an arrow to B labelled 0.42.
   • Column B: an arrow to A labelled 0.27, a loop 0.23, an arrow to C labelled 0.5.
   • Column C: an arrow to A labelled 0.12, a loop 0.88.
   • Check: the labels leaving each vertex, loop included, add to 1.

Write down the transition matrix for the transition diagram, with the states in the order P, Q.

• Column P (arrows leaving P): 0.41 (loop), 0.59 (P to Q)
• Column Q (arrows leaving Q): 0.18 (Q to P), 0.82 (loop)
• \(\mathbf{T} = \begin{pmatrix} 0.41 & 0.18 \\ 0.59 & 0.82 \end{pmatrix}\)
• Each column adds to 1.

The transition diagram has three states. Find its transition matrix, with the states in the order A, B, C, and name the absorbing state.

• Column A (arrows leaving A): 1 (loop)
• Column B (arrows leaving B): 0.3 (B to A), 0.5 (loop), 0.2 (B to C)
• Column C (arrows leaving C): 0.1 (C to A), 0.45 (C to B), 0.45 (loop)
• \(\mathbf{T} = \begin{pmatrix} 1 & 0.3 & 0.1 \\ 0 & 0.5 & 0.45 \\ 0 & 0.2 & 0.45 \end{pmatrix}\)
• The loop at A is 1 and nothing leaves A, so A is absorbing.

Write down the transition matrix for the transition diagram, with the states in the order W, X, Y, Z.

• Column W (arrows leaving W): 0.4 (W to X), 0.6 (W to Y)
• Column X (arrows leaving X): 0.5 (X to W), 0.2 (loop), 0.3 (X to Y)
• Column Y (arrows leaving Y): 0.15 (Y to X), 0.35 (loop), 0.5 (Y to Z)
• Column Z (arrows leaving Z): 0.35 (Z to W), 0.25 (Z to Y), 0.4 (loop)
• \(\mathbf{T} = \begin{pmatrix} 0 & 0.5 & 0 & 0.35 \\ 0.4 & 0.2 & 0.15 & 0 \\ 0.6 & 0.3 & 0.35 & 0.25 \\ 0 & 0 & 0.5 & 0.4 \end{pmatrix}\)
• Each column adds to 1.

Three radio stations, P, Q and R, compete for listeners. Each month, 65% of the listeners of P stay with P and 25% move to Q. Of the listeners of Q, 55% stay with Q and 30% move to R. Of the listeners of R, 20% move to P and 50% move to Q. Write down the transition matrix for the monthly movement of listeners, with the stations in the order P, Q, R.

• Column P: stay 0.65, to Q 0.25, so to R \(1 − 0.65 − 0.25 = 0.1\)
• Column Q: stay 0.55, to R 0.3, so to P \(1 − 0.55 − 0.3 = 0.15\)
• Column R: to P 0.2, to Q 0.5, so stay \(1 − 0.2 − 0.5 = 0.3\)
• \(\mathbf{T} = \begin{pmatrix} 0.65 & 0.15 & 0.2 \\ 0.25 & 0.55 & 0.5 \\ 0.1 & 0.3 & 0.3 \end{pmatrix}\)

Answer each part.

1. (a) Explain why \(\begin{pmatrix} 0.63 & 0.34 \\ 0.48 & 0.66 \end{pmatrix}\) cannot be a transition matrix.

   • Add each column. Column 1: \(0.63 + 0.48 = 1.11\).
   • The columns of a transition matrix must each add to 1, so this is not one.

2. (b) \(\mathbf{T} = \begin{pmatrix} 0.22 & 0.36 & a \\ 0.48 & b & 0.14 \\ c & 0.27 & 0.42 \end{pmatrix}\) is a transition matrix. Find \(a\), \(b\) and \(c\).

   • Column 1: \(0.22 + 0.48 + c = 1\), so \(c = 0.3\)
   • Column 2: \(0.36 + 0.27 + b = 1\), so \(b = 0.37\)
   • Column 3: \(0.14 + 0.42 + a = 1\), so \(a = 0.44\)

A plant is checked each week. It is healthy (H), infected (I) or dead (D). A healthy plant is still healthy the next week with probability 0.88, and infected with probability 0.12. An infected plant recovers to healthy with probability 0.17 and dies with probability 0.34. A dead plant stays dead.

1. (a) Write down the transition matrix, with the states in the order H, I, D.

   • H: stay 0.88, to I 0.12, to D 0.
   • I: to H 0.17, stay 0.49, to D 0.34.
   • D: stays, so \(P(D \to D) = 1\).
   • \(\mathbf{T} = \begin{pmatrix} 0.88 & 0.17 & 0 \\ 0.12 & 0.49 & 0 \\ 0 & 0.34 & 1 \end{pmatrix}\)

2. (b) List the loops and arrows of the transition diagram.

   • Each diagonal entry is a loop, each other non-zero entry in column X, row Y is an arrow from X to Y.
   • Loops: H 0.88, I 0.49, D 1.
   • Arrows: H to I 0.12, I to H 0.17, I to D 0.34.

3. (c) State the absorbing state and say how you can see it in the matrix.

   • D has a loop of 1 and no arrow leaves it.
   • In the matrix, column D is \(\begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix}\): a 1 on the diagonal and 0 elsewhere.