NZ Level 7 (NZC) Level 2 (NCEA)
topic badge
Misrepresentation of Results
Lesson

It happens from time to time that the noble art of statistics is used to mislead.

This can occur due to poor experimental design or a flawed survey procedure leading to biassed results. It can also occur when perfectly good statistical results are manipulated in the reporting to give a false impression.

Bias in experimental design is avoided by randomisation in the selection of samples as is discussed in other chapters. The same care to avoid sample bias is needed in survey sampling. In addition, the design of the questions to be asked in the survey needs to be done carefully to avoid response bias. 

In the reporting of statistical evidence, there are many ways in which the truth can be obscured and a false impression conveyed. Misleading graphical representations are discussed in another chapter. It is important to be aware that a graph may mislead by having the scale on an axis not begin at zero, or by having scale steps of unequal size. Icons, pie-chart sectors, bar graph elements and other visual representations can be drawn deceptively.

The text of a report based on statistical evidence may mislead by omitting some of the data. This may occur when outliers are removed from the data without proper investigation but merely because they do not accord with the investigator's prior belief. Survey responses can be presented without the precise survey questions or with other contextual information suppressed, thus inviting a wrong inference.

Worked Examples

Question 1

Lachlan asks $120$120 Year 12 students at his school how much time they spend on homework per night. $78$78 Year 12 students say they do more than $3$3 hours. At a meeting of the student council Lachlan reports "$65%$65% of students at this school do too much homework.

  1. Which of the following explain why this is misleading?

    The survey does not represent the population of the school.

    A

    The question should have been multiple choice.

    B

    The question was biased.

    C

    The sample size was too small.

    D

    The survey does not represent the population of the school.

    A

    The question should have been multiple choice.

    B

    The question was biased.

    C

    The sample size was too small.

    D

Question 2

An online company asks all its customers to complete a survey and $58%$58% respond. In the survey there was a question which asked “How happy are you with our service?”. Respondents could tick either “Very Happy” or “Somewhat Happy”.

In their annual newsletter the company wrote an article reporting “$100%$100% of customers are happy with our service”.

  1. Which of the following explain why this is misleading? Choose all appropriate answers.

    Only $58%$58% responded so these results are not indicative of all customers.

    A

    Not enough customers were asked to take part in the survey.

    B

    The scale for the response did not provide enough options for the reply.

    C

    Only $58%$58% responded so these results are not indicative of all customers.

    A

    Not enough customers were asked to take part in the survey.

    B

    The scale for the response did not provide enough options for the reply.

    C

 

Outcomes

S7-1

Carry out investigations of phenomena, using the statistical enquiry cycle: A conducting surveys that require random sampling techniques, conducting experiments, and using existing data sets B evaluating the choice of measures for variables and the sampling and data collection methods used C using relevant contextual knowledge, exploratory data analysis, and statistical inference.

S7-3

S7-3 Evaluate statistically based reports: A interpreting risk and relative risk B identifying sampling and possible non-sampling errors in surveys, including polls

91265

Conduct an experiment to investigate a situation using statistical methods

91266

Evaluate a statistically based report

What is Mathspace

About Mathspace