Investigation: Statistics in the media

Statistics in the media

When we read an article, or the government releases a statement or a friend is convincing you of an argument, we will often use statistics to support our argument. And to be even more convincing we might use a visual display like a pie graph or a bar chart to show the information.

When someone is making an argument or trying to make their point they may be making deliberate choices to make the data look a certain way in order to prove their point. Sometimes this is done accidentally and other times it is done deliberately to mislead.
 

There are many ways someone can twist data to make our point:

• Cherry-picking data - selecting the results that are favourable.
• Biased surveys and sampling - the phrasing of survey questions and the interviewer can often influence the responses given, as well as how they have selected the people being surveyed.
• Over-generalisation - when asserting a conclusion from a particular group to a wider population when the initial group is not representative of the larger population.
• Overstating significance of findings - sometimes it is not clearly stated how large the sample was or how large the margin of error is. If the error margin is large then the true result may be significantly different from the stated figure.
• Correlation and causation - if a relationship is observed between two variables A and B, it is tempting to jump to the conclusion that A causes B. However, it may be that B causes A, the two may in part cause each other, there may be a third factor that causes both or the relationship may not exist at all (and the observation was made be purely by chance).
• Bad comparisons - sometimes two data sets cannot be meaningfully compared against each other. For example, if native speakers of English are compared against non-native speakers in an English reading comprehension task, we cannot conclude that the native speakers are "more intelligent" when they inevitably do better.
• Misinterpreting statistics or probability - probabilities, in particular, are commonly misunderstood and misrepresented in the media. For example, stories of lottery winners are commonplace in society, which serves to obscure the astronomically small chances (typically less than 1 in 300 million or more) that you would win if you bought a ticket.

 

Open up your favourite news or entertainment website and see if you can find some examples of these tactics. Present them to your class and see who can find the most outrageous misuse of statistics!

 

Misleading graphs

Let's take a look at this chart and its caption:

"Children are committing crimes because movies aren't interesting anymore."

We can use two pieces of data that seem to be changing together, like movie attendance and juvenile crime in a city, and start making up stories.

But in truth we don't know if one is being caused by the other. It's possible that the increased crime rate led to the movie cinemas shutting down. It is even more likely that there is a third factor, like increasing unemployment, might explain both trends. Alternatively, these two things might be totally unrelated!

If you look through enough data you'll find something to tell a story about, and using statistics and charts can make the story you tell seem even more convincing. But being convincing with statistics is not the same as discovering the truth with statistics, and we all need to be careful about what conclusions we draw and what we consider as a "fact" when someone tells a story.

Let's try another one:

"Petrol prices still getting cheaper"

Analysing data isn't just about how you present the data you look at, but also which data points within a dataset that you choose. From the chart the caption seems true! But if we zoom out over the last 20 years another story emerges - they seem to be increasing in general. 

Petrol prices over a longer time period

 

Convincing visuals

Look at the following images and their titles.

"Chinese people love cats more than Brazilians!"	"Results of this year's school election, Dante wins easily"

 

What are they trying to convince you of?

The first one is showing us how much more China likes cats than Brazil. The second one is convincing us that Dante easily beat Miguel in the school election.

Are either of their statements true? Do you notice anything strange they have done to to display the data?

Let's start with the second image, the school election, first.

Looking closely, how much bigger is the vote difference? Only 3 votes!

Dante has definitely won the election, but did he "destroy" Miguel? The y-axis in the original graph starts at 150. Let's resize it to start at 0, and give the chart a new title.

 

Which of these graphs more accurately tells the story of the school election?  In this case the second graph shows how close it really was.

Now let's consider the two cats. That second cat appears much bigger! 

The underlying data tells us that there are around twice as many cats in China than Brazil, which means the cat on the right is twice as tall. However, by using a 2D image instead of a bar, the second cat is actually four times bigger!

We have used basic columns to represent the same data in the second graph below. Suddenly the difference doesn't seem so big. 

The heading also suggests that China loves cats more, but China has a much bigger population compared to Brazil. The third graph shows the number of cats in each country per 100000 people.

Two charts can use the same data and still produce different graphs with the opposite conclusion. We could caption the third graph "Brazilian people love cats more than Chinese people!", and maybe that would be convincing to you.

Ultimately we are only measuring the number of cats, not how much people love their pets. The best caption for the third image would be something like "There are three times as many cats per person in Brazil than in China", which delivers the information without loaded language.

 

Check this article

Read this article carefully and examine the chart that accompanies it. What problems can you find? What questions might you want to ask the people who wrote it?

"Recent findings have found that approximately half of all soccer players are below the average height of people in A-league soccer competition. Parents that wish to have their children compete in basketball competitively should not allow their children to play soccer."

 

Create your own

The table below shows the average marks for students in Year 8 in their final year maths test, their height, their sex, the number of DVD sales in Australia that year, and whether the principal of the school was over 175 cm tall.

Imagine you are a writer for an online news site that needs more views. You are asked to write an article using this data that will grab people’s attention - even if it isn’t statistically relevant, or completely misleading.

Using the tactics found above, write a misleading headline and display the data in a misleading chart to make your headline work.

Year	Sex	Average maths mark (split by sex)	Average maths mark (combined)	Average height	Average height (combined)	DVD sales (millions of dollars)	Was the principal over 175 cm tall?
2013	Female	68	73	159	162	28.4	YES
	Male	78		165
2014	Female	72	74	158	161	14.9	NO
	Male	76		164
2015	Female	81	75	159	161	11.4	NO
	Male	69		163
2016	Female	83	77	159	163	9.9	YES
	Male	71		167
2017	Female	84	80	160	164	8.4	YES
	Male	76		168
2018	Female	80	81	159	161	7.3	NO
	Male	82		163

Write the first paragraph of your news story to go with it!

Prompts:

You do not have to use all the data, just choose the bits that help your story. Look at just  two or three columns and see if there is a certain story you can tell by using the data in the right (or wrong) way.

Here are some possible headlines you could make if you get stuck:

• Better test results in boys getting taller
• Less time watching DVDs results in higher test scores
• Taller principals motivate students to grow taller