topic badge
New Zealand
Level 7 - NCEA Level 2

Sources of Bias

Lesson

gazette.com by Asay

Being biased means that you are being one-sided.  It would mean you are not viewing things from the neutral perspective. Bias can come in many forms.  Maybe you prefer your friends when picking a sports team, or you give one of you family pets your scraps before another.  Bias can be intentional or unintentional.  We are going to focus here on the sources of bias when collecting or analysing data. 

To avoid bias, attention must be paid firstly to the method of selecting the sample and secondly, in the case of a survey, to the nature of the questions asked.

Sample selection

A sample needs to represent the population from which it is drawn. Subjects should be selected randomly in such a way that each member of the population is equally likely to be selected. 

A sample can fail to accurately reflect the population if certain groups are over- or under-represented in it. This can happen when subjects are selected because of some convenience factor - they have a listed telephone number or live in an easily accessible location or happen to be available when the survey is being done, for example.

Over- or under-representation can also happen when subjects volunteer to be in the survey. Self-selected respondents tend to be those with stronger than usual opinions or with a particular characteristic that may distinguish them from the general population and lead to their response type being over-represented. 

Surveys that are voluntary and in which there are many non-responses will generally under-represent the kinds of responses that might have been given by the non-responding subjects.

Randomisation is the key to reducing bias in the selection of a sample. The technique for obtaining a simple random sample is discussed elsewhere. One refinement of this is known as a stratified random sample. In this, if it is known that the population contains factors in certain proportions that are likely to influence the result, then the sample can be drawn in such a way that it consists of strata with the factors in the same proportions as in the population. For example, if it is known that 10% of the population is left-handed and it is suspected that this may influence the responses to the survey, then a researcher could ensure that the sample also contains random selections of both left- and right-handed subjects in the correct proportion.

Survey questions

Bias in a survey can also occur as a result of the nature of the questions asked. This is called response bias. Often, subjects are presented with a choice of several options by which to answer a question. Survey questions can tend to elicit responses unfairly favouring one option over another.

This can happen when the wording of the question implicitly suggests the kind of response the questioner wishes to obtain by including emotive language or by including options that are more socially acceptable than the others. Also, there may be more negative choices available than positive among the options, or the reverse, making it more likely that one response direction will occur more often just by chance.

Worked examples

QUESTION 1

"Do you prefer this rad shirt or the ordinary one on the shelves at the moment?"

  1. Is the question biased or fair?

    Biased

    A

    Fair

    B
  2. Identify the type of bias.

    It is an emotive question.

    A

    It uses specific terminology.

    B

    It is a leading question.

    C

QUESTION 2

Which of these questions are fair?

  1. Do you feel that the TV news is an inaccurate portrayal of life’s problems?

    A

    Don't you think this newspaper is biased?

    B

    Do you prefer the look and feel of thick lush carpeting in your living room?

    C

    Do you take these extra strength multi-vitamins to supplement your diet?

    D

    None of them

    E

QUESTION 3

A TV station wants to know what the most popular type of music is, so they ask listeners to contact them and vote for their favourite type of music.

  1. Is the sample chosen biased or fair?

    Biased

    A

    Fair

    B
  2. Identify the type of bias involved.

    The sample is from self-selecting participants, i.e. only those that made an effort to respond.

    A

    The sample is not representative of the target population.

    B

Outcomes

S7-1

Carry out investigations of phenomena, using the statistical enquiry cycle: A conducting surveys that require random sampling techniques, conducting experiments, and using existing data sets B evaluating the choice of measures for variables and the sampling and data collection methods used C using relevant contextual knowledge, exploratory data analysis, and statistical inference.

S7-3

S7-3 Evaluate statistically based reports: A interpreting risk and relative risk B identifying sampling and possible non-sampling errors in surveys, including polls

91265

Conduct an experiment to investigate a situation using statistical methods

91266

Evaluate a statistically based report

What is Mathspace

About Mathspace