Steady state and long-term probabilities Problem set B One question at a time

\(\mathbf{T} = \begin{pmatrix} 0.58 & 0.17 & 0.08 \\ 0.29 & 0.68 & 0.12 \\ 0.13 & 0.15 & 0.8 \end{pmatrix}\) and \(\mathbf{s}_0 = \begin{pmatrix} 0.2 \\ 0.5 \\ 0.3 \end{pmatrix}\). Find \(\mathbf{T}^{50} \mathbf{s}_0\) and \(\mathbf{T}^{100} \mathbf{s}_0\). Use them to write down the long-term state vector.

• \(\mathbf{T}^{50} \mathbf{s}_0 = \begin{pmatrix} 0.225 \\ 0.360 \\ 0.416 \end{pmatrix}\)
• \(\mathbf{T}^{100} \mathbf{s}_0 = \begin{pmatrix} 0.225 \\ 0.360 \\ 0.416 \end{pmatrix}\)
• The two agree to 3 significant figures, so this is the steady state.

\(\mathbf{T} = \begin{pmatrix} 0.43 & 0.24 & 0.33 \\ 0.37 & 0.46 & 0.09 \\ 0.2 & 0.3 & 0.58 \end{pmatrix}\). The steady-state vector is \(\mathbf{s} = (s_1, s_2, s_3)\). Use \(\mathbf{T}\mathbf{s} = \mathbf{s}\) to write down a system of three linear equations that can be solved for \(s_1, s_2, s_3\). Do not solve it.

• \(\mathbf{T}\mathbf{s} = \mathbf{s}\) gives \(0.43 s_1 + 0.24 s_2 + 0.33 s_3 = s_1\), \(0.37 s_1 + 0.46 s_2 + 0.09 s_3 = s_2\), \(0.2 s_1 + 0.3 s_2 + 0.58 s_3 = s_3\)
• Rearrange two of them: \(−0.57 s_1 + 0.24 s_2 + 0.33 s_3 = 0\), \(0.37 s_1 − 0.54 s_2 + 0.09 s_3 = 0\)
• Use the shares adding to 1 instead of the third: \(s_1 + s_2 + s_3 = 1\)

Find the exact steady-state vector by setting up and solving a system of equations.

1. (a) \(\mathbf{T} = \begin{pmatrix} 0.73 & 0.38 \\ 0.27 & 0.62 \end{pmatrix}\)

   • \(\mathbf{T}\mathbf{s} = \mathbf{s}\) with \(\mathbf{s} = (s_1, s_2)\)
   • \(−0.27 s_1 + 0.38 s_2 = 0\)
   • \(s_1 + s_2 = 1\)
   • Solve these equations on the calculator: \(\mathbf{s} = (\frac{38}{65}, \frac{27}{65})\)

2. (b) \(\mathbf{T} = \begin{pmatrix} 0.35 & 0.22 & 0.12 \\ 0.28 & 0.58 & 0.32 \\ 0.37 & 0.2 & 0.56 \end{pmatrix}\)

   • \(\mathbf{T}\mathbf{s} = \mathbf{s}\) with \(\mathbf{s} = (s_1, s_2, s_3)\)
   • \(−0.65 s_1 + 0.22 s_2 + 0.12 s_3 = 0\)
   • \(0.28 s_1 − 0.42 s_2 + 0.32 s_3 = 0\)
   • \(s_1 + s_2 + s_3 = 1\)
   • Solve these equations on the calculator: \(\mathbf{s} = (\frac{4}{19}, \frac{8}{19}, \frac{7}{19})\)

Find the exact steady-state vector for \(\mathbf{T} = \begin{pmatrix} 0.2 & 0.38 & 0.22 & 0.14 \\ 0.3 & 0.02 & 0.2 & 0.22 \\ 0.1 & 0.48 & 0.48 & 0.44 \\ 0.4 & 0.12 & 0.1 & 0.2 \end{pmatrix}\).

• \(\mathbf{T}\mathbf{s} = \mathbf{s}\) with \(\mathbf{s} = (s_1, s_2, s_3, s_4)\)
• \(−0.8 s_1 + 0.38 s_2 + 0.22 s_3 + 0.14 s_4 = 0\)
• \(0.3 s_1 − 0.98 s_2 + 0.2 s_3 + 0.22 s_4 = 0\)
• \(0.1 s_1 + 0.48 s_2 − 0.52 s_3 + 0.44 s_4 = 0\)
• \(s_1 + s_2 + s_3 + s_4 = 1\)
• Solve these equations on the calculator: \(\mathbf{s} = (\frac{3}{13}, \frac{5}{26}, \frac{5}{13}, \frac{5}{26})\)

Use your calculator to find the steady-state vector. Give the entries to 3 significant figures.

1. (a) \(\mathbf{T} = \begin{pmatrix} 0.83 & 0.06 & 0.11 \\ 0.04 & 0.79 & 0.18 \\ 0.13 & 0.15 & 0.71 \end{pmatrix}\)

   • Solve \((\mathbf{T} - \mathbf{I})\mathbf{s} = \mathbf{0}\) with the sum of the entries equal to 1.
   • \(\mathbf{s} = \begin{pmatrix} 0.332 \\ 0.342 \\ 0.326 \end{pmatrix}\)

2. (b) \(\mathbf{T} = \begin{pmatrix} 0.48 & 0.26 & 0.34 & 0.16 \\ 0.42 & 0.24 & 0.08 & 0.3 \\ 0.04 & 0.1 & 0.38 & 0.12 \\ 0.06 & 0.4 & 0.2 & 0.42 \end{pmatrix}\)

   • Solve \((\mathbf{T} - \mathbf{I})\mathbf{s} = \mathbf{0}\) with the sum of the entries equal to 1.
   • \(\mathbf{s} = \begin{pmatrix} 0.310 \\ 0.293 \\ 0.121 \\ 0.276 \end{pmatrix}\)

\(\mathbf{T} = \begin{pmatrix} 0.4 & 0.02 & 0.32 & 0.12 \\ 0.26 & 0.36 & 0.12 & 0.4 \\ 0.32 & 0.24 & 0.08 & 0.28 \\ 0.02 & 0.38 & 0.48 & 0.2 \end{pmatrix}\) and \(\mathbf{s}_0 = \begin{pmatrix} 0.5 \\ 0.1 \\ 0.1 \\ 0.3 \end{pmatrix}\). Find \(\mathbf{s}_4\).

• \(\mathbf{s}_4 = \mathbf{T}^4 \mathbf{s}_0\)
• \(\mathbf{T}^4 = \begin{pmatrix} 0.199 & 0.183 & 0.193 & 0.187 \\ 0.292 & 0.300 & 0.299 & 0.297 \\ 0.230 & 0.229 & 0.233 & 0.229 \\ 0.280 & 0.289 & 0.276 & 0.287 \end{pmatrix}\)
• \(\mathbf{s}_4 = \begin{pmatrix} 0.193 \\ 0.295 \\ 0.230 \\ 0.283 \end{pmatrix}\)

The transition matrix \(\mathbf{T} = \begin{pmatrix} 0.64 & 0.27 \\ 0.36 & 0.73 \end{pmatrix}\) gives the probability that a commuter's next journey is by train (R) or by bus (U), with the modes in the order R, U. Write down the probability that a commuter who now takes the train takes the bus on the next journey.

• Start train: column R. End bus: row U.
• Column R, row U: \(0.36\)

A forest has three kinds of tree, 1, 2 and 3. Each year the trees change with transition matrix \(\mathbf{T} = \begin{pmatrix} 0.66 & 0.12 & 0.17 \\ 0.23 & 0.79 & 0.24 \\ 0.11 & 0.09 & 0.59 \end{pmatrix}\), in that order.

1. (a) Find the long-term probability matrix, with entries to 3 significant figures.

   • \(\mathbf{T}^{100} = \begin{pmatrix} 0.282 & 0.282 & 0.282 \\ 0.527 & 0.527 & 0.527 \\ 0.191 & 0.191 & 0.191 \end{pmatrix}\)
   • Every column is the same: the long-term probabilities do not depend on the start.

2. (b) Hence state the long-term percentage of type 3.

   • Read any column: the entry for type 3 is \(0.191\)
   • As a percentage: 19.1%

A student is on time (O) or late (L) for school each day. If the student is on time today, the probability of being on time tomorrow is 0.92. If the student is late today, the probability of being on time tomorrow is 0.53.

1. (a) Write down the transition matrix.

   • Column 1: \(0.92, 0.08\). Column 2: \(0.53, 0.47\)
   • \(\mathbf{T} = \begin{pmatrix} 0.92 & 0.53 \\ 0.08 & 0.47 \end{pmatrix}\)

2. (b) The student was late today. Find the probability that the student is on time on the fourth day after today.

   • \(\mathbf{T}^4 = \begin{pmatrix} 0.872 & 0.849 \\ 0.128 & 0.151 \end{pmatrix}\)
   • Start in state 2 (column 2), end in state 1 (row 1): \(0.849\)

3. (c) Find the exact steady-state vector, and state the long-term probability that the student is on time.

   • \(\mathbf{T}\mathbf{s} = \mathbf{s}\) with \(\mathbf{s} = (s_1, s_2)\)
   • \(−0.08 s_1 + 0.53 s_2 = 0\)
   • \(s_1 + s_2 = 1\)
   • Solve these equations on the calculator: \(\mathbf{s} = (\frac{53}{61}, \frac{8}{61})\)
   • The long-term probability of the first state is \(\frac{53}{61}\).

\(\mathbf{T} = \begin{pmatrix} p & 0.28 \\ 1 − p & 0.72 \end{pmatrix}\) is a transition matrix with steady-state vector \(\mathbf{s} = \begin{pmatrix} 0.4 \\ 0.6 \end{pmatrix}\).

1. (a) Use \(\mathbf{T}\mathbf{s} = \mathbf{s}\) to write down an equation for p.

   • The first row of \(\mathbf{T}\mathbf{s} = \mathbf{s}\) gives \(p \times 0.4 + 0.28 \times 0.6 = 0.4\)

2. (b) Solve it to find p.

   • \(p = 0.58\)
   • Check: the second row gives the same value of p.