It happens from time to time that the noble art of statistics is used to mislead.
This can occur due to poor experimental design or a flawed survey procedure leading to biased results. It can also occur when perfectly good statistical results are manipulated in the reporting to give a false impression.
Bias in experimental design is avoided by randomisation in the selection of samples as is discussed in other chapters. The same care to avoid sample bias is needed in survey sampling. In addition, the design of the questions to be asked in the survey needs to be done carefully to avoid response bias.
In the reporting of statistical evidence, there are many ways in which the truth can be obscured and a false impression conveyed. Misleading graphical representations are discussed in another chapter. It is important to be aware that a graph may mislead by having the scale on an axis not begin at zero, or by having scale steps of unequal size. Icons, pie-chart sectors, bar graph elements and other visual representations can be drawn deceptively.
The text of a report based on statistical evidence may mislead by omitting some of the data. This may occur when outliers are removed from the data without proper investigation but merely because they do not accord with the investigator's prior belief. Survey responses can be presented without the precise survey questions or with other contextual information suppressed, thus inviting a wrong inference.
Worked Examples
Question 1
Lachlan asks $120$120 Year 12 students at his school how much time they spend on homework per night. $78$78 Year 12 students say they do more than $3$3 hours. At a meeting of the student council Lachlan reports "$65%$65% of students at this school do too much homework.
Which one of the following explains why this is misleading?
The survey does not represent the population of the school.
AThe question should have been multiple choice.
BThe question was biased.
CThe sample size was too small.
D
Question 2