Steady state and long-term probabilities Problem set A Worked solutions

\(\mathbf{T} = \begin{pmatrix} 0.67 & 0.14 & 0.19 \\ 0.21 & 0.79 & 0.28 \\ 0.12 & 0.07 & 0.53 \end{pmatrix}\) and \(\mathbf{s}_0 = \begin{pmatrix} 0.6 \\ 0.1 \\ 0.3 \end{pmatrix}\). Find \(\mathbf{T}^{50} \mathbf{s}_0\) and \(\mathbf{T}^{100} \mathbf{s}_0\). Use them to write down the long-term state vector.

• \(\mathbf{T}^{50} \mathbf{s}_0 = \begin{pmatrix} 0.315 \\ 0.526 \\ 0.159 \end{pmatrix}\)
• \(\mathbf{T}^{100} \mathbf{s}_0 = \begin{pmatrix} 0.315 \\ 0.526 \\ 0.159 \end{pmatrix}\)
• The two agree to 3 significant figures, so this is the steady state.

\(\mathbf{T} = \begin{pmatrix} 0.52 & 0.15 & 0.22 \\ 0.29 & 0.57 & 0.31 \\ 0.19 & 0.28 & 0.47 \end{pmatrix}\). The steady-state vector is \(\mathbf{s} = (s_1, s_2, s_3)\). Use \(\mathbf{T}\mathbf{s} = \mathbf{s}\) to write down a system of three linear equations that can be solved for \(s_1, s_2, s_3\). Do not solve it.

• \(\mathbf{T}\mathbf{s} = \mathbf{s}\) gives \(0.52 s_1 + 0.15 s_2 + 0.22 s_3 = s_1\), \(0.29 s_1 + 0.57 s_2 + 0.31 s_3 = s_2\), \(0.19 s_1 + 0.28 s_2 + 0.47 s_3 = s_3\)
• Rearrange two of them: \(−0.48 s_1 + 0.15 s_2 + 0.22 s_3 = 0\), \(0.29 s_1 − 0.43 s_2 + 0.31 s_3 = 0\)
• Use the shares adding to 1 instead of the third: \(s_1 + s_2 + s_3 = 1\)

Find the exact steady-state vector by setting up and solving a system of equations.

1. (a) \(\mathbf{T} = \begin{pmatrix} 0.79 & 0.34 \\ 0.21 & 0.66 \end{pmatrix}\)

   • \(\mathbf{T}\mathbf{s} = \mathbf{s}\) with \(\mathbf{s} = (s_1, s_2)\)
   • \(−0.21 s_1 + 0.34 s_2 = 0\)
   • \(s_1 + s_2 = 1\)
   • Solve these equations on the calculator: \(\mathbf{s} = (\frac{34}{55}, \frac{21}{55})\)

2. (b) \(\mathbf{T} = \begin{pmatrix} 0.58 & 0.28 & 0.2 \\ 0.23 & 0.27 & 0.22 \\ 0.19 & 0.45 & 0.58 \end{pmatrix}\)

   • \(\mathbf{T}\mathbf{s} = \mathbf{s}\) with \(\mathbf{s} = (s_1, s_2, s_3)\)
   • \(−0.42 s_1 + 0.28 s_2 + 0.2 s_3 = 0\)
   • \(0.23 s_1 − 0.73 s_2 + 0.22 s_3 = 0\)
   • \(s_1 + s_2 + s_3 = 1\)
   • Solve these equations on the calculator: \(\mathbf{s} = (\frac{6}{17}, \frac{4}{17}, \frac{7}{17})\)

Find the exact steady-state vector for \(\mathbf{T} = \begin{pmatrix} 0.06 & 0.34 & 0.5 & 0.4 \\ 0.08 & 0.22 & 0.04 & 0.12 \\ 0.44 & 0.04 & 0.36 & 0.22 \\ 0.42 & 0.4 & 0.1 & 0.26 \end{pmatrix}\).

• \(\mathbf{T}\mathbf{s} = \mathbf{s}\) with \(\mathbf{s} = (s_1, s_2, s_3, s_4)\)
• \(−0.94 s_1 + 0.34 s_2 + 0.5 s_3 + 0.4 s_4 = 0\)
• \(0.08 s_1 − 0.78 s_2 + 0.04 s_3 + 0.12 s_4 = 0\)
• \(0.44 s_1 + 0.04 s_2 − 0.64 s_3 + 0.22 s_4 = 0\)
• \(s_1 + s_2 + s_3 + s_4 = 1\)
• Solve these equations on the calculator: \(\mathbf{s} = (\frac{7}{22}, \frac{1}{11}, \frac{7}{22}, \frac{3}{11})\)

Use your calculator to find the steady-state vector. Give the entries to 3 significant figures.

1. (a) \(\mathbf{T} = \begin{pmatrix} 0.74 & 0.07 & 0.09 \\ 0.14 & 0.87 & 0.13 \\ 0.12 & 0.06 & 0.78 \end{pmatrix}\)

   • Solve \((\mathbf{T} - \mathbf{I})\mathbf{s} = \mathbf{0}\) with the sum of the entries equal to 1.
   • \(\mathbf{s} = \begin{pmatrix} 0.228 \\ 0.509 \\ 0.263 \end{pmatrix}\)

2. (b) \(\mathbf{T} = \begin{pmatrix} 0.38 & 0.04 & 0.22 & 0.04 \\ 0.16 & 0.02 & 0.1 & 0.5 \\ 0.4 & 0.44 & 0.48 & 0.2 \\ 0.06 & 0.5 & 0.2 & 0.26 \end{pmatrix}\)

   • Solve \((\mathbf{T} - \mathbf{I})\mathbf{s} = \mathbf{0}\) with the sum of the entries equal to 1.
   • \(\mathbf{s} = \begin{pmatrix} 0.167 \\ 0.194 \\ 0.389 \\ 0.250 \end{pmatrix}\)

\(\mathbf{T} = \begin{pmatrix} 0.12 & 0.16 & 0.22 & 0.06 \\ 0.5 & 0.12 & 0.46 & 0.44 \\ 0.28 & 0.38 & 0.02 & 0.04 \\ 0.1 & 0.34 & 0.3 & 0.46 \end{pmatrix}\) and \(\mathbf{s}_0 = \begin{pmatrix} 0.25 \\ 0.25 \\ 0.4 \\ 0.1 \end{pmatrix}\). Find \(\mathbf{s}_5\).

• \(\mathbf{s}_5 = \mathbf{T}^5 \mathbf{s}_0\)
• \(\mathbf{T}^5 = \begin{pmatrix} 0.132 & 0.131 & 0.132 & 0.132 \\ 0.344 & 0.339 & 0.343 & 0.343 \\ 0.182 & 0.188 & 0.183 & 0.182 \\ 0.341 & 0.342 & 0.342 & 0.343 \end{pmatrix}\)
• \(\mathbf{s}_5 = \begin{pmatrix} 0.132 \\ 0.343 \\ 0.184 \\ 0.342 \end{pmatrix}\)

The transition matrix \(\mathbf{T} = \begin{pmatrix} 0.72 & 0.19 \\ 0.28 & 0.81 \end{pmatrix}\) gives the probability that a customer's next phone is made by Aero (A) or by Brio (B), with the brands in the order A, B. Write down the probability that a customer who now has a Brio phone buys an Aero phone next.

• Start Brio: column B. End Aero: row A.
• Column B, row A: \(0.19\)

Three types of land use, 1, 2 and 3, change each decade with transition matrix \(\mathbf{T} = \begin{pmatrix} 0.77 & 0.11 & 0.09 \\ 0.16 & 0.74 & 0.21 \\ 0.07 & 0.15 & 0.7 \end{pmatrix}\), in that order.

1. (a) Find the long-term probability matrix, with entries to 3 significant figures.

   • \(\mathbf{T}^{100} = \begin{pmatrix} 0.307 & 0.307 & 0.307 \\ 0.414 & 0.414 & 0.414 \\ 0.279 & 0.279 & 0.279 \end{pmatrix}\)
   • Every column is the same: the long-term probabilities do not depend on the start.

2. (b) Hence state the long-term percentage of type 2.

   • Read any column: the entry for type 2 is \(0.414\)
   • As a percentage: 41.4%

A hockey team wins (W) or does not win (N) each match. If it wins a match, the probability that it wins the next is 0.68. If it does not win a match, the probability that it wins the next is 0.37.

1. (a) Write down the transition matrix.

   • Column 1: \(0.68, 0.32\). Column 2: \(0.37, 0.63\)
   • \(\mathbf{T} = \begin{pmatrix} 0.68 & 0.37 \\ 0.32 & 0.63 \end{pmatrix}\)

2. (b) The team did not win its last match. Find the probability that it wins its third match after that.

   • \(\mathbf{T}^3 = \begin{pmatrix} 0.550 & 0.520 \\ 0.450 & 0.480 \end{pmatrix}\)
   • Start in state 2 (column 2), end in state 1 (row 1): \(0.520\)

3. (c) Find the exact steady-state vector, and state the long-term probability that the team wins a match.

   • \(\mathbf{T}\mathbf{s} = \mathbf{s}\) with \(\mathbf{s} = (s_1, s_2)\)
   • \(−0.32 s_1 + 0.37 s_2 = 0\)
   • \(s_1 + s_2 = 1\)
   • Solve these equations on the calculator: \(\mathbf{s} = (\frac{37}{69}, \frac{32}{69})\)
   • The long-term probability of the first state is \(\frac{37}{69}\).

\(\mathbf{T} = \begin{pmatrix} 0.74 & p \\ 0.26 & 1 − p \end{pmatrix}\) is a transition matrix with steady-state vector \(\mathbf{s} = \begin{pmatrix} 0.6 \\ 0.4 \end{pmatrix}\).

1. (a) Use \(\mathbf{T}\mathbf{s} = \mathbf{s}\) to write down an equation for p.

   • The first row of \(\mathbf{T}\mathbf{s} = \mathbf{s}\) gives \(0.74 \times 0.6 + p \times 0.4 = 0.6\)

2. (b) Solve it to find p.

   • \(p = 0.39\)
   • Check: the second row gives the same value of p.