7.07 Transition matrices

Suppose in a certain region on any given day, the chance of the weather changing depends only on what the weather was doing on the previous day. To keep things simple, we will say that the weather has two states-either wet or dry.

Let's say that the probability of being dry on any day is conditional upon whether or not it was dry the day before. Specifically let's assume that the probability of being dry on any day n+1, given it was dry on day n, is 0.3.

Using standard conditional probability notation we can write that Pr\left(D_{n+1}| D_n\right)=0.3.

Since dry is the complement of wet, we can say that Pr\left(W_{n+1}| D_n\right)=0.7.

Further, suppose we assume that the probability of being wet on any day n+1, given it was wet on day n, is 0.6. Thus we can similarly write that Pr\left(W_{n+1}| W_n\right)=0.6 and Pr\left(W_{n+1}| D_n\right)=0.4.

The situation is illustrated in what is known as a state diagram as shown.

We can summarise the state diagram as a transition matrix T shown here:

T= \begin{bmatrix} 0.3 & 0.4\\ 0.7 & 0.6 \end{bmatrix}

Of course, we have to remember the order in which we placed these conditional probabilities. For example the entry 0.7 refers to Pr\left(W_{n+1}| D_n\right) etc. Our convention shows the complementary probabilities are written in columns, but there are other conventions around.

Check the sense of the entire matrix with this table:

Using matrices like this can make the process of finding the probabilities of future states very easy.

Examples

Example 1

The following two states, A and B, and their transition probabilities are displayed in the diagram below.

Construct the transition matrix T that represents the transitional probabilities between each state.

Worked Solution

Create a strategy

Use the following pattern of the transition matrix.

\qquad \quad A \qquad \quad \,\, B\\ T= \begin{matrix} A\\ B \end{matrix} \begin{bmatrix} A\to A & B\to A\\ A\to B & B\to B \end{bmatrix}

Apply the idea

\displaystyle T

\displaystyle =

\displaystyle \qquad A \qquad B\\ \begin{matrix} A\\ B \end{matrix} \begin{bmatrix} 0.12 & 0.91\\ 0.88 & 0.09 \end{bmatrix}

Write the corresponding values

Watch question walkthrough

Suppose we begin the process that the initial day was a wet day. We look to the second column of our matrix as it denotes the probabilities given it is wet today. We can write a state matrix as S_0=\begin{pmatrix}0\\1\end{pmatrix}showing that on our initial day S_0, there was a probability of 1 that the day was wet.

To determine probabilities for the next state S_1, we simply multiply Tby S_0 using matrix multiplication as follows:

T \times S_0 = \begin{bmatrix} 0.3 & 0.4 \\ 0.7 & 0.6 \end{bmatrix} \begin{pmatrix} 0 \\ 1 \end{pmatrix} = \begin{pmatrix} 0.4 \\ 0.6 \end{pmatrix}

What this means is that there is a 40\% chance that the next day is dry, and a 60\% chance that the next day is wet. This second state could be denoted as S_1= \begin{pmatrix} 0.4 \\ 0.6 \end{pmatrix} .

What about the state after this? S_2 can be found by reapplying T to S_1, so that:

S_2 = TS_1 = \begin{bmatrix} 0.3 & 0.4 \\ 0.7 & 0.6 \end{bmatrix} \begin{pmatrix} 0.4 \\ 0.6 \end{pmatrix} = \begin{pmatrix} 0.36 \\ 0.64 \end{pmatrix}

This means that the second day has a 36\% chance of being dry and 64\% chance of being wet, given the previous day had a 40\% chance of being dry and a 60\% chance of being wet.

We could continue in this manner, progressively working out probabilities for each new day given the probabilities of the previous day-however, there is a clever way to short cut the process.

Recall, that S_1=TS_0 and that S_2=TS_1.

But that means that S_2=T\left(S_1\right)=T\left(TS_0\right)=T^2S_0.

Continuing the pattern we see that S_3=T\left(S_2\right)=T\left(T^2S_0\right)=T^3S_0.

In fact, we can generalise the pattern to reveal that:

S_n=T^nS_0

In other words, rather than progress through a series of tedious calculations to finding the probabilities for the nth day, we can simply use technology to determine T^n, and then multiply that by S_0 to find S_n.

For example, we calculate S_{10} as:

S_{10} = T_{10}S_0 = \begin{bmatrix} 0.3 & 0.4 \\ 0.7 & 0.6 \end{bmatrix}^{10} \begin{pmatrix} 0\\ 1 \end{pmatrix}= \begin{pmatrix} 0.363636\\ 0.636364 \end{pmatrix}

Examples

Example 2

In a certain town there are two insurance companies where customers can insure their homes annually. After an extensive investigation, it was found that 80\% of home-owners who used company A in any year will use that company again in the next year. The other 20\% switch to company B. Further 60\% of those who use company B will stay with company B, and the others switch to company A.

a

If both companies originally had 500 customers, what will be the case after 3 years?

Worked Solution

Create a strategy

We can use the following state diagram:

Apply the idea

\displaystyle S_{3}	\displaystyle =	\displaystyle T^{3} \times S_{0}	Write the formula
	\displaystyle =	\displaystyle \begin{bmatrix} 0.8 & 0.6 \\ 0.2 & 0.4 \end{bmatrix}^{3} \begin{pmatrix} 500\\ 500 \end{pmatrix}	Substitute the value of T and S_{0}
	\displaystyle =	\displaystyle \begin{pmatrix} 748\\ 252 \end{pmatrix}	Evaluate

So after 3 years 748 homeowners will obtain their insurance from company A and 252 homeowners will obtain their from company B.

b

What will be the situation after 10 years?

Worked Solution

Create a strategy

We can use the pattern: S_n=T^n\times S_0 where, n = \text{number of years}, T= \text{transition matrix}, and S_{0}=\text{number of customers}.

Apply the idea

\displaystyle S_{10}	\displaystyle =	\displaystyle T^{10} \times S_{0}	Write the pattern
	\displaystyle =	\displaystyle \begin{bmatrix} 0.8 & 0.6 \\ 0.2 & 0.4 \end{bmatrix}^{10} \begin{pmatrix} 500\\ 500 \end{pmatrix}	Substitute the value of T and S_{0}
	\displaystyle =	\displaystyle \begin{pmatrix} 750\\ 250 \end{pmatrix}	Evaluate

So after 10 years 750 homeowners will obtain their insurance from company A and 250 homeowners will obtain their from company B.

c

Predict the steady state matrix?

Worked Solution

Create a strategy

Compare the results in part (a) and part (b).

Apply the idea

Comparing the results for after 3 and 10 years, we can see that the solutions have not changed much with company A getting 75\% of ultimate market share and therefore company B gets 25\%. We believe this because over a 7 year period, the number of homeowners accessing each company hardly changed and so has reached a point where it will no longer change.

d

Suppose instead that company A retains p\% of its customers and loses the rest to company B, and company B retains q\% of its customers and loses the rest to company A. Then it can be shown that the long term percentage market share for company A and company B is \dfrac{q}{1-p+q}\% and \dfrac{1-p}{1-p+q}\% respectively. Verify this result for our example.

Worked Solution

Create a strategy

We can use the given formulas: \dfrac{q}{1-p+q}\% and \dfrac{1-p}{1-p+q}\% where, p=0.8 and q=0.6.

Apply the idea

\displaystyle \dfrac{q}{1-p+q}	\displaystyle =	\displaystyle \dfrac{0.6}{1-0.8+0.6}	Substitute p=0.8 and q=0.6
	\displaystyle =	\displaystyle \dfrac{0.6}{0.8}	Evaluate the denominator
	\displaystyle =	\displaystyle 0.75	Evaluate
\displaystyle \dfrac{1-p}{1-p+q}	\displaystyle =	\displaystyle \dfrac{1-0.8}{1-0.8+0.6}	Substitute p=0.8 and q=0.6
	\displaystyle =	\displaystyle \dfrac{0.2}{0.8}	Evaluate the denominator
	\displaystyle =	\displaystyle 0.25	Evaluate

Notice with our example, that there is not that much difference between the probabilities of S_3 and S_{10}. In fact using technology S_{100} as the same as S_{10}.

Provided T contains no zeros, it can be proved that as n becomes larger and larger, S_n=T^nS_0 converges to what is referred to as a steady state.

Referring to our example, what this means is that the long term probabilities of dry and rainy days for this region are given by something close to \dfrac{36}{99}=\dfrac{4}{11} and \dfrac{63}{99}=\dfrac{7}{11} respectively.

In fact we can find the precise probability values for the steady state by solving the matrix equation T\begin{pmatrix} x \\ y \end{pmatrix}=\begin{pmatrix} x \\ y \end{pmatrix}. The reason for this is that at some stage, with just one more multiplication by T, a state matrix will will not change. The elements x and y will simply stay as they are.

The long term state matrix is usually referred to as S_\infty=\begin{pmatrix} x \\ y \end{pmatrix} and it is quite easy to find the precise values of x an y.

Using a little matrix algebra we have:

We can therefore say, using multiplication on the first row and column, that -0.7x+0.4y=0 and since we know that the two probabilities we seek are complimentary, we can also write x+y=1.

These two equations can be solved simultaneously to reveal that x andy are \dfrac{4}{11} and \dfrac{7}{11} respectively.

Examples

A special property of a transition matrix is that: |T|^n=|T^n|.

Examples