6.10 Transition matrices

Transition matrices using the weather

Suppose in a certain region on any given day, the chance of the weather changing depends only on what the weather was doing on the previous day. To keep things simple, we will say that the weather has two states–either wet or dry.

Let's say that the probability of being dry on any day is conditional upon whether or not it was dry the day before. Specifically let's assume that the probability of being dry on any day $n+1$n+1, given it was dry on day $n$n, is $0.3$0.3.

Using standard conditional probability notation we can write that $Pr\left(D_{n+1}\mid D_n\right)=0.3$Pr(Dn+1​∣Dn​)=0.3. Since dry is the complement of wet, we can say that $Pr\left(W_{n+1}\mid D_n\right)=0.7$Pr(Wn+1​∣Dn​)=0.7.

Further, suppose we assume that the probability of being wet on any day $n+1$n+1, given it was wet on day $n$n, is $0.6$0.6. Thus we can similarly write that $Pr\left(W_{n+1}\mid W_n\right)=0.6$Pr(Wn+1​∣Wn​)=0.6 and $Pr\left(W_{n+1}\mid D_n\right)=0.4$Pr(Wn+1​∣Dn​)=0.4.

The situation is illustrated in what is known as a state diagram, shown here:

We can summarise the state diagram as a transition matrix $T$T shown here:

Of course, we have to remember the order in which we placed these conditional probabilities. For example the entry $0.7$0.7 refers to $Pr\left(W_{n+1}\mid D_n\right)$Pr(Wn+1​∣Dn​) etc. Our convention shows the complementary probabilities are written in columns, but there are other conventions around.

Check the sense of the entire matrix with this table:

	Given it is dry today...	Given it is wet today...
Probability of being dry tomorrow?	$0.3$0.3	$0.4$0.4
Probability of being wet tomorrow?	$0.7$0.7	$0.6$0.6

Using matrices like this can make the process of finding the probabilities of future states very easy.

 

Finding future states

Suppose we begin the process that the initial day was a wet day. We look to the second column of our matrix as it denotes the probabilities given it is wet today. We can write a state matrix as showing that on our initial day $S_0$S0​, there was a probability of $1$1 that the day was wet.

To determine probabilities for the next state $S_1$S1​, we simply multiply $T$T by $S_0$S0​ using matrix multiplication as follows:

What this means is that there is a $40%$40% chance that the next day is dry, and a $60%$60% chance that the next day is wet. This second state could be denoted as .

What about the state after this? $S_2$S2​ can be found by reapplying $T$T to $S_1$S1​, so that:

This means that the second day has a $36%$36% chance of being dry and $64%$64% chance of being wet, given the previous day had a $40%$40% chance of being dry and a $60%$60% chance of being wet.

We could continue in this manner, progressively working out probabilities for each new day given the probabilities of the previous day–however, there is a clever way to short cut the process.

 

A nice short-cut

Recall, that $S_1=TS_0$S1​=TS0​ and that $S_2=TS_1$S2​=TS1​.

But that means that $S_2=T\left(S_1\right)=T\left(TS_0\right)=T^2S_0$S2​=T(S1​)=T(TS0​)=T2S0​.

Continuing the pattern we see that $S_3=T\left(S_2\right)=T\left(T^2S_0\right)=T^3S_0$S3​=T(S2​)=T(T2S0​)=T3S0​.

In fact, we can generalise the pattern to reveal that:

$S_n=T^nS_0$Sn​=TnS0​

In other words, rather than progress through a series of tedious calculations to finding the probabilities for the $n$nth day, we can simply use technology to determine $T^n$Tn, and then multiply that by $S_0$S0​ to find $S_n$Sn​.

For example, we calculate $S_{10}$S10​ as:

Worked example

Example 1

In a certain town there are two insurance companies where customers can insure their homes annually. After an extensive investigation, it was found that $80%$80% of home-owners who used company $A$A in any year will use that company again in the next year. The other $20%$20% switch to company $B$B. Further $60%$60% of those who use company $B$B will stay with company $B$B, and the others switch to company $A$A.

Consider the following questions:

(a) If both companies originally had $500$500 customers, what will be the case after $3$3 years?

(b) What will be the situation after $10$10 years?

(c) Predict the steady state matrix?

(d) Suppose instead that company $A$A retains $p%$p% of its customers and loses the rest to company $B$B, and company $B$B retains $q%$q% of its customers and loses the rest to company $A$A. Then it can be shown that the long term percentage market share for company $A$A and company $B$B is $\frac{q}{1-p+q}%$q1−p+q​% and $\frac{1-p}{1-p+q}%$1−p1−p+q​% respectively. Verify this result for our example.

 

(a)

Think: We can represent the situation using a state diagram as follows:

 

Do: The transition matrix becomes with .

Since we have $S_n=T^n\times S_0$Sn​=Tn×S0​, then:

Hence after $3$3 years $748$748 homeowners will obtain their insurance from company $A$A and $252$252 homeowners will obtain their from company $B$B.

(b)

Think: We now want to find $T^{10}$T10 using technology.

Do: After $10$10 years we have:

Hence after $10$10 years $750$750 homeowners will obtain their insurance from company $A$A and $250$250 homeowners will obtain their from company $B$B.

(c)

Think: When comparing the results for after $3$3 and $10$10 years, we can see that the solutions have not changed much with company $A$A getting $75/%$75/% of ultimate market share and therefore company $B$B gets $25/%$25/%. We believe this because over a $7$7 year period, the number of homeowners accessing each company hardly changed and so has reached a point where it will no longer change.

(d)

Using the formula provided, we have for $p=0.8$p=0.8 and $q=0.6$q=0.6:

$\frac{q}{1-p+q}$q1−p+q​	$=$=	$\frac{0.6}{1-0.8+0.6}$0.61−0.8+0.6​
	$=$=	$\frac{6}{8}$68​
	$=$=	$0.75$0.75
$\frac{1-p}{1-p+q}$1−p1−p+q​	$=$=	$\frac{1-0.8}{1-0.8+0.6}$1−0.81−0.8+0.6​
	$=$=	$\frac{2}{8}$28​
	$=$=	$0.25$0.25

This accords with our answer for question $3$3.

 

In the long run, a 'steady state'

Notice with our example, that there is not that much difference between the probabilities of $S_3$S3​ and $S_{10}$S10​. In fact using technology $S_{100}$S100​ as the same as $S_{10}$S10​.

Provided $T$T contains no zeros, it can be proved that as $n$n becomes larger and larger, $S_n=T^nS_0$Sn​=TnS0​ converges to what is referred to as a steady state.

Referring to our example, what this means is that the long term probabilities of dry and rainy days for this region are given by something close to $\frac{36}{99}=\frac{4}{11}$3699​=411​ and $\frac{63}{99}=\frac{7}{11}$6399​=711​ respectively.

 

 

Practice questions

Question 1

Question 2

 

Exploration

Three grocery stores $A,B$A,B and $C$C each have an initial customer base of $400$400 people. Every week however, each store finds that they lose a certain percentage of their base to other stores. The percentage of people moving from one store to another (or staying loyal to a store) are summarised in the following state diagram:

So for example, $60%$60% of people who use store $A$A in any one week stay with store $A$A the following week. However $10%$10% shift to store $B$B and $30%$30% shift to store $C$C. The same idea applies to the other stores.

We can write the transition matrix down as and the initial state matrix as .

We can determine a future state, say week $3$3 as $S_3=T^3\times S_0$S3​=T3×S0​, given by:

The calculations were made using technology.

Here is the state matrix determined for week $6$6:

There is some evidence of convergence here between week $3$3 and week $6$6.

We might think about trying a much larger power in an attempt to find the steady state matrix a little quicker. Technology or CAS are indispensable in this regard.

If we arbitrarily try week $20$20 we see that:

week $21$21 shows the same state matrix as week $20$20, so the numbers $350$350, $250$250 and $600$600 represent the long term customer base for each of stores $A$A, $B$B and $C$C respectively.

 

Practice questions

Question 3