EXT: Law of total probability

The Law of Total Probability allows us to calculate the likelihood of an event whose occurrence is influenced by which of several other disjoint events occurs. It involves the idea of conditional probability and one of the axioms of probability.

An example will clarify the concept.

Example

Consider a sample space consisting of four possible weather conditions: we assume tomorrow will be fine or overcast (but not both) and the temperature will reach $25^{\circ}C$25∘C  or not. We do not know in advance whether the day will be overcast or fine but we would like to know how likely it is that the temperature will reach $25^{\circ}C$25∘C anyway.

We know that if the day is overcast the probability that the temperature level will be reached is $0.2$0.2 and if the day is fine, the probability is $0.7$0.7. We also know from meteorological records that the probability of an overcast day at this time of the year is $0.4$0.4.  

We observe that the events Overcast and Fine are disjoint: if one occurs, the other does not. We argue that the total probability of reaching $25^{\circ}C$25∘C must be the sum of the probabilities of reaching this temperature given the occurrence of each of the separate events Overcast and Fine. The following diagram illustrates the situation.

Intuitively, considering the areas of the relevant parts of the diagram, we might realise that the required calculation is $0.2\times0.4+0.7\times0.6$0.2×0.4+0.7×0.6. Thus, the total probability is $0.5$0.5 in this case.

 

 

More generally, suppose $B_1,B_2,B_3,...$B1​,B2​,B3​,... is a partition of a sample space $S$S. (The events $B_i$Bi​have no common outcomes.) Let $A$A be an event that does have outcomes in common with some or all of the $B_i$Bi​.

We speak of the conditional probability that $A$A occurs given that one of the $B_i$Bi​ occurs and for this we use the notation $P\left(A|B_i\right)$P(A|Bi​) for each $B_i$Bi​.

(This is the probability that $A$A occurs given that $B_i$Bi​ occurs. Events $B_i$Bi​ and $A$A both occur if they have a common outcome. We use the intersection notation $A\cap B_i$A∩Bi​ to mean events $A$A and $B_i$Bi​ both occur.)

It seems reasonable to define the conditional probability $P\left(A|B_i\right)$P(A|Bi​) by $P\left(A|B_i\right)=\frac{P\left(A\cap B_i\right)}{P\left(B_i\right)}$P(A|Bi​)=P(A∩Bi​)P(Bi​)​ and so, 

$P\left(A\cap B_i\right)=P\left(A|B_i\right).P\left(B_i\right)$P(A∩Bi​)=P(A|Bi​).P(Bi​)

One of the axioms that we assume for probability tells us under what condition probabilities may be added. If $B_1,B_2,B_3,...$B1​,B2​,B3​,... are disjoint events, then

$P\left(B_1\cup B_2\cup B_3\cup...\right)=P\left(B_1\right)+P\left(B_2\right)+P\left(B_3\right)+...$P(B1​∪B2​∪B3​∪...)=P(B1​)+P(B2​)+P(B3​)+...

Now, for a sample space $S$S,  $A=A\cap S=A\cap\left(B_1\cup B_2\cup B_3\cup...\right)$A=A∩S=A∩(B1​∪B2​∪B3​∪...) where the events  $B_i$Bi​ partition the sample space. (The union $A\cup B$A∪B is the set of outcomes that belong to $A$A or $B$B.)

By a distributive law for unions and intersections, we have 

$A=\left(A\cap B_1\right)\cup\left(A\cap B_2\right)\cup\left(A\cap B_3\right)\cup...$A=(A∩B1​)∪(A∩B2​)∪(A∩B3​)∪...

and since the right-hand side of this equation is again a union of disjoint sets, we see that

$P\left(A\right)=P\left(A\cap B_1\right)+P\left(A\cap B_2\right)+P\left(A\cap B_3\right)+...$P(A)=P(A∩B1​)+P(A∩B2​)+P(A∩B3​)+...

Then, using the conditional probability notation, this is 

$P\left(A\right)=P\left(A|B_1\right).P\left(B_1\right)+P\left(A|B_2\right).P\left(B_2\right)+P\left(A|B_3\right).P\left(B_3\right)+...$P(A)=P(A|B1​).P(B1​)+P(A|B2​).P(B2​)+P(A|B3​).P(B3​)+...

and this corresponds perfectly to the calculation that was done in the example above.

 

In summary, we may use the idea of total probability to calculate the probability of an event $A$A when the conditional probabilities $P\left(A|B_i\right)$P(A|Bi​) are known and the events $B_i$Bi​ partition the sample space.

Worked Examples:

question 1

question 2