Statistics are usually included in media to support facts, reinforce arguments or provide additional information to the viewer. However, we must not forget that they are a powerful tool of persuasion, and must be interpreted with caution, as statistics can be deliberately manipulated or skewed by the author to shape the opinions of viewers. The data behind the graph may be manipulated in the following common ways:
- Cherry-picking data - discarding results that are unfavourable.
- Wrong measure used - we have seen that different the measures of centre and spread may give different impressions of the data if outliers are present.
- Biased surveys - the phrasing of survey questions and the interviewer can often influence the responses given.
- Sampling technique - a sample should be representative of the population of interest. A poor sampling technique can introduce bias to the results. Practices likely to produce bias include very small samples or samples not selected randomly, such as from volunteer phone polls.
- Over-generalisation - when asserting a conclusion from a particular group to a wider population when the initial group is not representative of the larger population.
- Overstating significance of findings - sometimes it is not clearly stated how large the sample was or how large the margin of error is. If the error margin is large then then result may be significantly off the stated figure.
- Correlation and causation - if a relationship is observed between two variables A and B, it is tempting to jump to the conclusion that A causes B. However, it may be that B causes A, the two may in part cause each other, there may be a third factor that causes both or the relationship may be purely by chance.
- Comparing apples and oranges - when comparing two data sets that cannot be meaningfully compared against each other. (Example)
- Misinterpreting statistics or probability - probabilities, in particular, are commonly misunderstood and misrepresented in the media a court cases. (see prosecutor's fallacy)
Misrepresentation of statistics often happens unintentionally by a source who is reporting on a subject for which they are not an expert or are not familiar with the statistics quoted. However, misrepresentation may also be intentional to lead viewers to a particular conclusion. Critical evaluation of figures given in articles is very important in our information rich community. This video has some tips for looking deeper than the attention grabbing headlines.
Discussion
Read the articles below and answer the following questions for each article:
- Identify the statistics that were presented in the article.
- Write up a summary of these statistics.
- Do you think the claims made in the article were consistent with the statistics provided?
- Do you think any claims were overstated or statistics misused? Do you need further information to ascertain this?
- What source(s) were the statistics drawn from? Are those sources independent?
- Look up the source(s) online. What other interesting statistics would you include if you had the chance to edit the article?
- Are there graphs you could include to make the statistics more engaging or easier to interpret?
Article 1: We may be getting smarter but more animals are under threat
Article 2: Swans have facts to back allowance
Article 3: More than two million Australians in poverty
Article 4: Jobless? Figure it out, if you can
Article 5: Select your own from ABC's Story Lab