12.04 Misrepresentation of results

Statistics are usually included in media to support facts, reinforce arguments or provide additional information to the viewer. However, we must not forget that they are a powerful tool of persuasion, and must be interpreted with caution, as statistics can be deliberately manipulated or skewed by the author to shape the opinions of viewers. Data may be manipulated in the following common ways:

• Cherry-picking data - discarding results that are unfavourable.
• Wrong measure used - we have seen that different the measures of centre and spread may give different impressions of the data if outliers are present.
• Biased surveys - the phrasing of survey questions and the interviewer can often influence the responses given.
• Sampling technique - a sample should be representative of the population of interest. A poor sampling technique can introduce bias to the results. Practices likely to produce bias include very small samples or samples not selected randomly, such as from volunteer phone polls.
• Over-generalisation - when asserting a conclusion from a particular group to a wider population when the initial group is not representative of the larger population.
• Overstating significance of findings - sometimes it is not clearly stated how large the sample was or how large the margin of error is. If the error margin is large then then result may be significantly off the stated figure.
• Correlation and causation - if a relationship is observed between two variables A and B, it is tempting to jump to the conclusion that A causes B. However, it may be that B causes A, the two may in part cause each other, there may be a third factor that causes both or the relationship may be purely by chance.
• Comparing apples and oranges - when comparing two data sets that cannot be meaningfully compared against each other. (Example)
• Misinterpreting statistics or probability - probabilities, in particular, are commonly misunderstood and misrepresented in media court cases. (see prosecutor's fallacy)

Misrepresentation of statistics often happens unintentionally by a source who is reporting on a subject for which they are not an expert or are not familiar with the statistics quoted. However, misrepresentation may also be intentional to lead viewers to a particular conclusion. Critical evaluation of figures given in articles is very important in our information rich community. This video has some tips for looking deeper than the attention grabbing headlines.

Another way to commonly misrepresent data is using an inappropriate visual representation. Graphs are an important tool to convey statistics, however, a poorly constructed graph may lead a reader to an incorrect conclusion. Review the common ways misleading graphs can be constructed.

Practice questions

Question 1

Question 2

Question 3

Question 4