Powers of the transition matrix and state vectors Problem set A Worked solutions

Each matrix has the states in the order A, B. Use your calculator.

1. (a) \(\mathbf{T} = \begin{pmatrix} 0.31 & 0.54 \\ 0.69 & 0.46 \end{pmatrix}\). Find the probability of being in state B after 3 steps, given that you start in A.

   • \(\mathbf{T}^3 = \begin{pmatrix} 0.432 & 0.444 \\ 0.568 & 0.556 \end{pmatrix}\)
   • Start A: column A. End B: row B.
   • \(P = 0.568\)

2. (b) \(\mathbf{T} = \begin{pmatrix} 0.74 & 0.41 \\ 0.26 & 0.59 \end{pmatrix}\). Find the probability of being in state A after 5 steps, given that you start in B.

   • \(\mathbf{T}^5 = \begin{pmatrix} 0.613 & 0.610 \\ 0.387 & 0.390 \end{pmatrix}\)
   • Start B: column B. End A: row A.
   • \(P = 0.610\)

\(\mathbf{T} = \begin{pmatrix} 0.48 & 0.17 & 0.28 \\ 0.31 & 0.62 & 0.12 \\ 0.21 & 0.21 & 0.6 \end{pmatrix}\), with the states in the order A, B, C. Find the probability of being in state C after 4 steps, given that you start in A.

• \(\mathbf{T}^4 = \begin{pmatrix} 0.301 & 0.292 & 0.311 \\ 0.363 & 0.372 & 0.330 \\ 0.336 & 0.336 & 0.359 \end{pmatrix}\)
• Start A: column A. End C: row C.
• \(P = 0.336\)

\(\mathbf{T} = \begin{pmatrix} 0.63 & 0.09 & 0 & 0 \\ 0.19 & 0.52 & 0.24 & 0 \\ 0.11 & 0.28 & 0.58 & 0 \\ 0.07 & 0.11 & 0.18 & 1 \end{pmatrix}\), with the states in the order A, B, C, D. State D is absorbing.

1. (a) Find the probability of being in state D after 5 steps, given that you start in A.

   • \(\mathbf{T}^5 = \begin{pmatrix} 0.148 & 0.0756 & 0.0468 & 0 \\ 0.217 & 0.201 & 0.176 & 0 \\ 0.222 & 0.226 & 0.208 & 0 \\ 0.413 & 0.497 & 0.570 & 1.00 \end{pmatrix}\)
   • Start A: column A. End D: row D.
   • \(P = 0.413\)

2. (b) Find the probability of being in state B after 3 steps, given that you start in C.

   • \(\mathbf{T}^3 = \begin{pmatrix} 0.283 & 0.0971 & 0.0374 & 0 \\ 0.251 & 0.280 & 0.238 & 0 \\ 0.222 & 0.295 & 0.310 & 0 \\ 0.244 & 0.327 & 0.414 & 1.00 \end{pmatrix}\)
   • Start C: column C. End B: row B.
   • \(P = 0.238\)

\(\mathbf{T} = \begin{pmatrix} 0.77 & 0.32 \\ 0.23 & 0.68 \end{pmatrix}\) and \(\mathbf{s}_0 = \begin{pmatrix} 0.7 \\ 0.3 \end{pmatrix}\). Find \(\mathbf{s}_3\).

• \(\mathbf{s}_3 = \mathbf{T}^3 \mathbf{s}_0\)
• \(\mathbf{T}^3 = \begin{pmatrix} 0.620 & 0.529 \\ 0.380 & 0.471 \end{pmatrix}\)
• \(\mathbf{s}_3 = \begin{pmatrix} 0.593 \\ 0.407 \end{pmatrix}\)
• The entries add to 1.

\(\mathbf{T} = \begin{pmatrix} 0.74 & 0.07 & 0.09 \\ 0.14 & 0.87 & 0.13 \\ 0.12 & 0.06 & 0.78 \end{pmatrix}\) and \(\mathbf{s}_0 = \begin{pmatrix} 0.4 \\ 0 \\ 0.6 \end{pmatrix}\). Find \(\mathbf{s}_4\).

• \(\mathbf{s}_4 = \mathbf{T}^4 \mathbf{s}_0\)
• \(\mathbf{T}^4 = \begin{pmatrix} 0.379 & 0.174 & 0.201 \\ 0.367 & 0.653 & 0.353 \\ 0.254 & 0.173 & 0.445 \end{pmatrix}\)
• \(\mathbf{s}_4 = \begin{pmatrix} 0.272 \\ 0.359 \\ 0.369 \end{pmatrix}\)
• The entries add to 1.

For each transition matrix, say whether the chain is regular. If it is, find the smallest power of the matrix with every entry positive. If it is not, say why.

1. (a) \(\mathbf{T} = \begin{pmatrix} 0 & 0.6 \\ 1 & 0.4 \end{pmatrix}\), with two states.

   • \(\mathbf{T}\) has a zero, so find \(\mathbf{T}^2\).
   • \(\mathbf{T}^2 = \begin{pmatrix} 0.6 & 0.24 \\ 0.4 & 0.76 \end{pmatrix}\)
   • Every entry is positive: regular, with smallest power 2.

2. (b) \(\mathbf{T} = \begin{pmatrix} 1 & 0.2 & 0 \\ 0 & 0.5 & 0 \\ 0 & 0.3 & 1 \end{pmatrix}\), with three states.

   • Column 1 is \((1, 0, 0)\) and column 3 is \((0, 0, 1)\): two absorbing states.
   • An absorbing column keeps its zeros in every power of \(\mathbf{T}\).
   • So no power has every entry positive: not regular.

3. (c) \(\mathbf{T} = \begin{pmatrix} 0 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{pmatrix}\), with three states.

   • \(\mathbf{T}^2 = \begin{pmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 1 & 0 & 0 \end{pmatrix}\)
   • \(\mathbf{T}^3 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}\), the identity matrix.
   • The powers repeat, always with zeros: not regular.

Three delivery companies, Atlas, Bolt and Core, have market shares of 45%, 35% and 20%. Each year customers move between the companies with transition matrix \(\mathbf{T} = \begin{pmatrix} 0.81 & 0.04 & 0.13 \\ 0.07 & 0.88 & 0.17 \\ 0.12 & 0.08 & 0.7 \end{pmatrix}\), in that order. Find the projected market share of each company after 4 years.

• \(\mathbf{s}_0 = \begin{pmatrix} 0.45 \\ 0.35 \\ 0.2 \end{pmatrix}\)
• \(\mathbf{s}_4 = \mathbf{T}^4 \mathbf{s}_0 = \begin{pmatrix} 0.329 \\ 0.429 \\ 0.243 \end{pmatrix}\)
• Atlas 32.9%, Bolt 42.9%, Core 24.3%

A city has three bike stations, A, B and C. At the end of each day a bike moves between stations with transition matrix \(\mathbf{T} = \begin{pmatrix} 0.66 & 0.21 & 0.09 \\ 0.22 & 0.52 & 0.35 \\ 0.12 & 0.27 & 0.56 \end{pmatrix}\), in that order. A bike is at station C on Monday evening. Find the probability that it is at station A on Thursday evening.

• Monday to Thursday is 3 steps.
• \(\mathbf{T}^3 = \begin{pmatrix} 0.407 & 0.294 & 0.242 \\ 0.337 & 0.385 & 0.394 \\ 0.256 & 0.321 & 0.364 \end{pmatrix}\)
• Start C: column C. End A: row A. \(P = 0.242\)

A gym has 1200 members. At the start, 800 use the morning session (M) and 400 the evening session (E). Each month the members change sessions with transition matrix \(\mathbf{T} = \begin{pmatrix} 0.88 & 0.34 \\ 0.12 & 0.66 \end{pmatrix}\), in the order M, E. Find the number of members in each session after 3 months.

• \(\mathbf{s}_0 = \begin{pmatrix} 800 \\ 400 \end{pmatrix}\)
• \(\mathbf{s}_3 = \mathbf{T}^3 \mathbf{s}_0 = \begin{pmatrix} 873 \\ 327 \end{pmatrix}\)
• morning 873, evening 327. The total is still 1200.

The weather in a town is Sunny (S) or Rainy (R) each day. The transition matrix is \(\mathbf{T} = \begin{pmatrix} 0.8 & 0.4 \\ 0.2 & 0.6 \end{pmatrix}\), with the states in the order S, R. Monday is sunny.

1. (a) Find the probability that Tuesday is sunny and Wednesday and Thursday are both rainy.

   • Route: S to S, then S to R, then R to R.
   • \(P(S \to S) \times P(S \to R) \times P(R \to R) = 0.8 \times 0.2 \times 0.6\)
   • \(= 0.096\)

2. (b) Find the probability that Thursday is rainy, whatever the weather on Tuesday and Wednesday.

   • Three steps, any route: find \(\mathbf{T}^3\).
   • \(\mathbf{T}^3 = \begin{pmatrix} 0.688 & 0.624 \\ 0.312 & 0.376 \end{pmatrix}\)
   • Start S: column S. End R: row R. \(P = 0.312\)

Three shops A, B and C have shares of 60%, 30% and 10%. Each month the shares change with transition matrix \(\mathbf{T} = \begin{pmatrix} 0.77 & 0.11 & 0.06 \\ 0.16 & 0.81 & 0.14 \\ 0.07 & 0.08 & 0.8 \end{pmatrix}\), in that order. Find the first month in which shop C has a share of more than 25%.

• Find \(\mathbf{s}_n = \mathbf{T}^n \mathbf{s}_0\) for n = 1, 2, 3, … and look at the third entry.
• n = 6: 24.8%, which is not enough.
• n = 7: 25.5%, which is more than 25%.
• The first month is 7.

\(\mathbf{T} = \begin{pmatrix} 0.46 & 0.13 & 0.24 \\ 0.33 & 0.58 & 0.27 \\ 0.21 & 0.29 & 0.49 \end{pmatrix}\), with the states in the order A, B, C.

1. (a) Find the probability of going from A to C in exactly two steps by adding the probabilities of the three routes.

   • Via A: \(P(A \to A) \times P(A \to C) = 0.46 \times 0.21 = 0.0966\)
   • Via B: \(P(A \to B) \times P(B \to C) = 0.33 \times 0.29 = 0.0957\)
   • Via C: \(P(A \to C) \times P(C \to C) = 0.21 \times 0.49 = 0.1029\)
   • Add: \(0.2952\)

2. (b) Check your answer by finding \(\mathbf{T}^2\) on your calculator.

   • \(\mathbf{T}^2 = \begin{pmatrix} 0.3049 & 0.2048 & 0.2631 \\ 0.3999 & 0.4576 & 0.3681 \\ 0.2952 & 0.3376 & 0.3688 \end{pmatrix}\)
   • Column A, row C: \(0.2952\), the same.