Powers of the transition matrix and state vectors Problem set B Worked solutions

Each matrix has the states in the order A, B. Use your calculator.

1. (a) \(\mathbf{T} = \begin{pmatrix} 0.62 & 0.17 \\ 0.38 & 0.83 \end{pmatrix}\). Find the probability of being in state B after 4 steps, given that you start in A.

   • \(\mathbf{T}^4 = \begin{pmatrix} 0.337 & 0.296 \\ 0.663 & 0.704 \end{pmatrix}\)
   • Start A: column A. End B: row B.
   • \(P = 0.663\)

2. (b) \(\mathbf{T} = \begin{pmatrix} 0.13 & 0.52 \\ 0.87 & 0.48 \end{pmatrix}\). Find the probability of being in state A after 3 steps, given that you start in B.

   • \(\mathbf{T}^3 = \begin{pmatrix} 0.337 & 0.396 \\ 0.663 & 0.604 \end{pmatrix}\)
   • Start B: column B. End A: row A.
   • \(P = 0.396\)

\(\mathbf{T} = \begin{pmatrix} 0.57 & 0.24 & 0.09 \\ 0.12 & 0.49 & 0.33 \\ 0.31 & 0.27 & 0.58 \end{pmatrix}\), with the states in the order A, B, C. Find the probability of being in state A after 5 steps, given that you start in C.

• \(\mathbf{T}^5 = \begin{pmatrix} 0.271 & 0.269 & 0.262 \\ 0.321 & 0.325 & 0.331 \\ 0.408 & 0.406 & 0.407 \end{pmatrix}\)
• Start C: column C. End A: row A.
• \(P = 0.262\)

\(\mathbf{T} = \begin{pmatrix} 1 & 0.17 & 0.08 & 0.06 \\ 0 & 0.43 & 0.14 & 0.22 \\ 0 & 0.27 & 0.52 & 0.12 \\ 0 & 0.13 & 0.26 & 0.6 \end{pmatrix}\), with the states in the order A, B, C, D. State A is absorbing.

1. (a) Find the probability of being in state A after 6 steps, given that you start in D.

   • \(\mathbf{T}^6 = \begin{pmatrix} 1.00 & 0.523 & 0.439 & 0.430 \\ 0 & 0.133 & 0.157 & 0.160 \\ 0 & 0.156 & 0.181 & 0.182 \\ 0 & 0.188 & 0.223 & 0.227 \end{pmatrix}\)
   • Start D: column D. End A: row A.
   • \(P = 0.430\)

2. (b) Find the probability of being in state D after 3 steps, given that you start in B.

   • \(\mathbf{T}^3 = \begin{pmatrix} 1.00 & 0.349 & 0.239 & 0.225 \\ 0 & 0.191 & 0.197 & 0.224 \\ 0 & 0.234 & 0.265 & 0.217 \\ 0 & 0.226 & 0.299 & 0.334 \end{pmatrix}\)
   • Start B: column B. End D: row D.
   • \(P = 0.226\)

\(\mathbf{T} = \begin{pmatrix} 0.39 & 0.72 \\ 0.61 & 0.28 \end{pmatrix}\) and \(\mathbf{s}_0 = \begin{pmatrix} 0.2 \\ 0.8 \end{pmatrix}\). Find \(\mathbf{s}_4\).

• \(\mathbf{s}_4 = \mathbf{T}^4 \mathbf{s}_0\)
• \(\mathbf{T}^4 = \begin{pmatrix} 0.547 & 0.535 \\ 0.453 & 0.465 \end{pmatrix}\)
• \(\mathbf{s}_4 = \begin{pmatrix} 0.537 \\ 0.463 \end{pmatrix}\)
• The entries add to 1.

\(\mathbf{T} = \begin{pmatrix} 0.83 & 0.06 & 0.11 \\ 0.04 & 0.79 & 0.18 \\ 0.13 & 0.15 & 0.71 \end{pmatrix}\) and \(\mathbf{s}_0 = \begin{pmatrix} 0.25 \\ 0.35 \\ 0.4 \end{pmatrix}\). Find \(\mathbf{s}_3\).

• \(\mathbf{s}_3 = \mathbf{T}^3 \mathbf{s}_0\)
• \(\mathbf{T}^3 = \begin{pmatrix} 0.614 & 0.159 & 0.226 \\ 0.135 & 0.563 & 0.322 \\ 0.251 & 0.278 & 0.452 \end{pmatrix}\)
• \(\mathbf{s}_3 = \begin{pmatrix} 0.300 \\ 0.360 \\ 0.341 \end{pmatrix}\)
• The entries add to 1.

For each transition matrix, say whether the chain is regular. If it is, find the smallest power of the matrix with every entry positive. If it is not, say why.

1. (a) \(\mathbf{T} = \begin{pmatrix} 0.7 & 1 \\ 0.3 & 0 \end{pmatrix}\), with two states.

   • \(\mathbf{T}\) has a zero, so find \(\mathbf{T}^2\).
   • \(\mathbf{T}^2 = \begin{pmatrix} 0.79 & 0.7 \\ 0.21 & 0.3 \end{pmatrix}\)
   • Every entry is positive: regular, with smallest power 2.

2. (b) \(\mathbf{T} = \begin{pmatrix} 0.6 & 0 & 0 \\ 0.1 & 1 & 0.3 \\ 0.3 & 0 & 0.7 \end{pmatrix}\), with three states.

   • Column 2 is \((0, 1, 0)\): the second state is absorbing.
   • An absorbing column keeps its zeros in every power of \(\mathbf{T}\).
   • So no power has every entry positive: not regular.

3. (c) \(\mathbf{T} = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}\), with two states.

   • \(\mathbf{T}^2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}\), the identity matrix.
   • \(\mathbf{T}^3 = \mathbf{T}\), and the powers keep repeating, always with zeros.
   • Not regular.

Three mobile networks, Nova, Orbit and Pulse, have market shares of 25%, 50% and 25%. Each quarter customers move between the networks with transition matrix \(\mathbf{T} = \begin{pmatrix} 0.78 & 0.06 & 0.14 \\ 0.12 & 0.88 & 0.08 \\ 0.1 & 0.06 & 0.78 \end{pmatrix}\), in that order. Find the projected market share of each network after 5 quarters.

• \(\mathbf{s}_0 = \begin{pmatrix} 0.25 \\ 0.5 \\ 0.25 \end{pmatrix}\)
• \(\mathbf{s}_5 = \mathbf{T}^5 \mathbf{s}_0 = \begin{pmatrix} 0.279 \\ 0.469 \\ 0.252 \end{pmatrix}\)
• Nova 27.9%, Orbit 46.9%, Pulse 25.2%

A lift stops at three floors, A, B and C, at the end of each trip. The next floor depends on the present floor, with transition matrix \(\mathbf{T} = \begin{pmatrix} 0.57 & 0.18 & 0.14 \\ 0.31 & 0.63 & 0.26 \\ 0.12 & 0.19 & 0.6 \end{pmatrix}\), in that order. The lift is at floor A. Find the probability that it is at floor C after 4 more trips.

• 4 trips is 4 steps.
• \(\mathbf{T}^4 = \begin{pmatrix} 0.298 & 0.272 & 0.262 \\ 0.435 & 0.443 & 0.422 \\ 0.267 & 0.285 & 0.316 \end{pmatrix}\)
• Start A: column A. End C: row C. \(P = 0.267\)

A shop has 1200 loyalty customers. At the start, 300 use the app (P) and 900 use the card (C). Each week the customers change method with transition matrix \(\mathbf{T} = \begin{pmatrix} 0.82 & 0.43 \\ 0.18 & 0.57 \end{pmatrix}\), in the order P, C. Find the number of customers using each method after 4 weeks.

• \(\mathbf{s}_0 = \begin{pmatrix} 300 \\ 900 \end{pmatrix}\)
• \(\mathbf{s}_4 = \mathbf{T}^4 \mathbf{s}_0 = \begin{pmatrix} 833 \\ 367 \end{pmatrix}\)
• app 833, card 367. The total is still 1200.

A morning bus is On time (O) or Late (L) each day. The transition matrix is \(\mathbf{T} = \begin{pmatrix} 0.7 & 0.45 \\ 0.3 & 0.55 \end{pmatrix}\), with the states in the order O, L. The bus is on time on Monday.

1. (a) Find the probability that Tuesday and Wednesday are both late and Thursday is on time.

   • Route: O to L, then L to L, then L to O.
   • \(P(O \to L) \times P(L \to L) \times P(L \to O) = 0.3 \times 0.55 \times 0.45\)
   • \(= 0.07425\), which is 0.0743 to 3 significant figures.

2. (b) Find the probability that Thursday is on time, whatever happens on Tuesday and Wednesday.

   • Three steps, any route: find \(\mathbf{T}^3\).
   • \(\mathbf{T}^3 = \begin{pmatrix} 0.60625 & 0.590625 \\ 0.39375 & 0.409375 \end{pmatrix}\)
   • Start O: column O. End O: row O. \(P = 0.606\)

Three bus routes P, Q and R carry 70%, 20% and 10% of the passengers. Each year the shares change with transition matrix \(\mathbf{T} = \begin{pmatrix} 0.72 & 0.12 & 0.06 \\ 0.21 & 0.78 & 0.11 \\ 0.07 & 0.1 & 0.83 \end{pmatrix}\), in that order. Find the first year in which route R carries more than 30% of the passengers.

• Find \(\mathbf{s}_n = \mathbf{T}^n \mathbf{s}_0\) for n = 1, 2, 3, … and look at the third entry.
• n = 6: 29.3%, which is not enough.
• n = 7: 30.6%, which is more than 30%.
• The first year is 7.

\(\mathbf{T} = \begin{pmatrix} 0.37 & 0.28 & 0.12 \\ 0.41 & 0.32 & 0.52 \\ 0.22 & 0.4 & 0.36 \end{pmatrix}\), with the states in the order A, B, C.

1. (a) Find the probability of going from B to A in exactly two steps by adding the probabilities of the three routes.

   • Via A: \(P(B \to A) \times P(A \to A) = 0.28 \times 0.37 = 0.1036\)
   • Via B: \(P(B \to B) \times P(B \to A) = 0.32 \times 0.28 = 0.0896\)
   • Via C: \(P(B \to C) \times P(C \to A) = 0.4 \times 0.12 = 0.048\)
   • Add: \(0.2412\)

2. (b) Check your answer by finding \(\mathbf{T}^2\) on your calculator.

   • \(\mathbf{T}^2 = \begin{pmatrix} 0.2781 & 0.2412 & 0.2332 \\ 0.3973 & 0.4252 & 0.4028 \\ 0.3246 & 0.3336 & 0.364 \end{pmatrix}\)
   • Column B, row A: \(0.2412\), the same.