6.10 Transition matrices

Suppose in a certain region on any given day, the chance of the weather changing depends only on what the weather was doing on the previous day. To keep things simple, we will say that the weather has two states - either wet or dry.

Let's say that the probability of being dry on any day is conditional upon whether or not it was dry the day before. Specifically let's assume that the probability of being dry on any day n+1, given it was dry on day n, is 0.3.

Using standard conditional probability notation we can write that Pr\left(D_{n+1}|D_n\right)=0.3. Since dry is the complement of wet, we can say that Pr\left(W_{n+1}|D_n\right)=0.7.

Further, suppose we assume that the probability of being wet on any day n+1, given it was wet on day n, is 0.6. Thus we can similarly write that Pr\left(W_{n+1}| W_n\right)=0.6 and Pr\left(W_{n+1}| D_n\right)=0.4.

The situation is illustrated in what is known as a state diagram.

We can summarise the state diagram as a transition matrix T shown here:T= \begin{bmatrix} 0.3 & 0.4\\ 0.7&0.6 \end{bmatrix}

Of course, we have to remember the order in which we placed these conditional probabilities. For example the entry 0.7 refers to Pr\left(W_{n+1}| D_n\right) etc. Our convention shows the complementary probabilities are written in columns, but there are other conventions around.

Check the sense of the entire matrix with this table:

Using matrices like this can make the process of finding the probabilities of future states very easy.

Suppose we begin the process that the initial day was a wet day. We look to the second column of our matrix as it denotes the probabilities given it is wet today. We can write a state matrix as S_0=\begin{pmatrix} 0\\1 \end{pmatrix}showing that on our initial day S_0, there was a probability of 1 that the day was wet.

To determine probabilities for the next state S_1, we simply multiply T by S_0 using matrix multiplication as follows:T \times S_0= \begin{bmatrix} 0.3 & 0.4\\ 0.7&0.6 \end{bmatrix}\begin{pmatrix} 0\\1 \end{pmatrix} =\begin{pmatrix} 0.4\\ 0.6\end{pmatrix}

What this means is that there is a 40\% chance that the next day is dry, and a 60\% chance that the next day is wet. This second state could be denoted as S_{1}=\begin{pmatrix} 0.4\\ 0.6\end{pmatrix}.

What about the state after this? S_{2} can be found by reapplying T to S_{1}, so that:S_2=TS_1= \begin{bmatrix} 0.3 & 0.4\\ 0.7&0.6 \end{bmatrix}\begin{pmatrix} 0.4\\0.6 \end{pmatrix} =\begin{pmatrix} 0.36\\ 0.64\end{pmatrix}

This means that the second day has a 36\% chance of being dry and 64\% chance of being wet, given the previous day had a 40\% chance of being dry and a 60\% chance of being wet.

We could continue in this manner, progressively working out probabilities for each new day given the probabilities of the previous day-however, there is a clever way to short cut the process.

Recall, that S_1=TS_0 and that S_2=TS_1.

But that means that S_2=T\left(S_1\right)=T\left(TS_0\right)=T^2S_0.

Continuing the pattern we see that S_3=T\left(S_2\right)=T\left(T^2S_0\right)=T^3S_0.

In fact, we can generalise the pattern to reveal that: S_n=T^nS_0

In other words, rather than progress through a series of tedious calculations to finding the probabilities for the nth day, we can simply use technology to determine T^n, and then multiply that by S_0 to find S_n.

For example, we calculate S_{10} as:S_{10}=T^{10} S_0= \begin{bmatrix} 0.3 & 0.4\\ 0.7&0.6 \end{bmatrix}^{10}\begin{pmatrix} 0\\1 \end{pmatrix} =\begin{pmatrix} 0.363636\\ 0.636364\end{pmatrix}

Examples

Example 1

The following two states, A and B, and their transition probabilities are displayed in the diagram below.

Construct the transition matrix T that represents the transitional probabilities between each state.

Worked Solution

Create a strategy

Use the fact that the the arrows in the diagram indicate the direction of transition, and the probability marked adjacent is the probability for such a transition. Place these probabilities in the appropriate entries of matrix T.

Apply the idea

Constructing the transition matrix T that represents the transitional probabilities between each state is given by:T=\begin{array}{cc} & \begin{array}{ccc} A & B \end{array} \\ \begin{array}{c} A \\ B \end{array} & \left[ \begin{array}{ccc} 0.12 & 0.91 \\ 0.88 & 0.09 \end{array}\right] \end{array}

Watch question walkthrough

Example 2

A website uploads two blog posts each day, one about social issues (S) and the other about environmental concerns (E).

a

Of the people who read a blog every day, 52\% of those that read about social issues on one day will read about social issues the next day. Also, 83\% of those that read about environmental concerns will also read about environmental concerns the next day.

Which of the following diagrams best represents this information?

A

B

C

D

Worked Solution

Create a strategy

Consider the direction of each arrow and the transition probabilities of each state to choose the best diagram that represents the above information.

Apply the idea

If 52\% of those that read blog S will read S the next day, then 100\%-52\%=48\% of those that read S must read E the next day.

Similarly, if 83\% of those that read blog E will read E the next day, then the remaining 100\%-83\%=17\% of those that read E will read S the next day.

The diagram that represents this transition probabilities between each state is given by:

So, Option A is the diagram that represents the given information.

Watch question walkthrough

b

Construct the transition matrix T that represents the transitional probabilities between each state.

Worked Solution

Create a strategy

Use the fact that the the arrows in the diagram indicate the direction of transition, and the probability marked adjacent is the probability for such a transition. Place these probabilities in the appropriate entries of matrix T.

Apply the idea

Constructing the transition matrix T that represents the transitional probabilities between each state is given by:T=\begin{array}{c} & \begin{array}{ccc} E && S \end{array} \\ & \left[ \begin{array}{ccc} 0.83 & 0.48 \\ 0.17 & 0.52 \end{array}\right] \begin{array}{c} E \\ S \end{array}\end{array}

Watch question walkthrough

c

On a certain day, the website records that 800 people read blog (S) while 550 people read (E). Use this information to predict the number of readers that will read blog (E) in 3 days time. Round your answer to the nearest whole number.

Worked Solution

Create a strategy

Use the formula on future state given by S_n=T^n \times S_0, where:

• S_{n} is a column vector whose entries are the number of people that read blog E and S on n days time.

• T^{n} is the transition matrix to the nth power.

• S_{0} is a column vector whose entries are the number of people that read blog E and S on this certain day

Use a calculator or technology to solve for future state,

Apply the idea

Since we want to predict the number of readers that will read blog (E) in 3 days time, then it means that n=3.

\displaystyle S_{n}	\displaystyle =	\displaystyle T^{n} \times S_0	Write down the formula
\displaystyle S_{3}	\displaystyle =	\displaystyle \begin{bmatrix} 0.83 & 0.48\\ 0.17&0.52 \end{bmatrix}^{3}\begin{pmatrix} 800\\550 \end{pmatrix}	Substitute 3 on n and write the transition matrix
\displaystyle S_{3}	\displaystyle =	\displaystyle \begin{pmatrix} 988\\ 362 \end{pmatrix}	Evaluate

So, the predicted number of readers that will read blog (E) in 3 days time is 988.

Watch question walkthrough

Notice with our example above, that there is not that much difference between the probabilities of S_3 and S_{10}. In fact using technology S_{100} as the same as S_{10}.

Provided T contains no zeros, it can be proved that as n becomes larger and larger, S_n=T^nS_0 converges to what is referred to as a steady state.

Referring to our example, what this means is that the long term probabilities of dry and rainy days for this region are given by something close to \dfrac{36}{99}=\dfrac{4}{11}and \dfrac{63}{99}=\dfrac{7}{11} respectively.

Three grocery stores A,B and C each have an initial customer base of 400 people. Every week however, each store finds that they lose a certain percentage of their base to other stores. The percentage of people moving from one store to another (or staying loyal to a store) are summarised in the following state diagram:

So for example, 60\% of people who use store A in any one week stay with store A the following week. However 10\% shift to store B and 30\% shift to store C. The same idea applies to the other stores.

We can write the transition matrix down as T= \begin{bmatrix} 0.60&0.20&0.15\\ 0.10&0.50&0.15\\ 0.30&0.30&0.70 \end{bmatrix} and the initial state matrix as S_{0}= \begin{pmatrix} 400\\ 400\\ 400 \end{pmatrix} .

We can determine a future state, say week 3 as S_3=T^3\times S_0, given by:

\begin{bmatrix} 0.60&0.20&0.15\\ 0.10&0.50&0.15\\ 0.30&0.30&0.70 \end{bmatrix}^3 \begin{pmatrix} 400\\ 400\\ 400 \end{pmatrix} = \begin{bmatrix} 0.349&0.285&0.261\\ 0.183&0.247&0.207\\ 0.468&0.468&0.532 \end{bmatrix} \begin{pmatrix} 400\\ 400\\ 400 \end{pmatrix} = \begin{pmatrix} 358\\ 254.8\\ 587.2\\ \end{pmatrix}

The calculations were made using technology.

Here is the state matrix determined for week 6:

S_6=T^6S_0= \begin{bmatrix} 0.60&0.20&0.15\\ 0.10&0.50&0.15\\ 0.30&0.30&0.70 \end{bmatrix}^6 \begin{pmatrix} 400\\ 400\\ 400 \end{pmatrix} = \begin{pmatrix} 350.819\\ 250\\ 599.181\\ \end{pmatrix}

There is some evidence of convergence here between week 3 and week 6.

We might think about trying a much larger power in an attempt to find the steady state matrix a little quicker. Technology or CAS are indispensable in this regard.

If we arbitrarily try week 20 we see that:

S_{20}=T^{20}S_0= \begin{bmatrix} 0.60&0.20&0.15\\ 0.10&0.50&0.15\\ 0.30&0.30&0.70 \end{bmatrix}^{20} \begin{pmatrix} 400\\ 400\\ 400 \end{pmatrix} = \begin{pmatrix} 350\\ 250\\ 600 \end{pmatrix}

week 21 shows the same state matrix as week 20, so the numbers 350, 250 and 600 represent the long term customer base for each of stores A, B and C respectively.

Examples