Language and Use of Statistics

NZ Level 7 (NZC) Level 2 (NCEA)

Sources of Errors

Lesson

In any measurement procedure, there is inevitably some error. This is true when an experiment is being done to measure some physical quantity, and it is true in the case of surveys designed to quantify people's voting intentions or brand preferences.

We distinguish two kinds of error: systematic error and random error.

Systematic error occurs when there is something wrong with the measuring apparatus or with the procedure being used. An instrument may not have been correctly calibrated, for example, or a scale may have been consistently misread. In a survey, bias may have been introduced by poor design of the survey questions or by the manner in which the sample of the population to be surveyed was selected.

Once detected, systematic error can usually be reduced by improvements in the experimental procedure and by taking steps to eliminate bias due to the survey design and to the sampling method.

Random error can be thought of as the remaining unexplained error after known sources of systematic error have been removed.

When several careful measurements of the same physical quantity are made, slightly different results may be reported at each trial due to small but uncontrolled fluctuations in the experimental conditions and by the impossibility of reading a measurement scale beyond some level of precision. The reported result of such a measurement is given as the mean of all the observed results together with an indication of how far the observations vary about the mean. The amount of variation is usually given by a statistic called the variance or by its square root, the standard deviation.

The accuracy of the result of a series of repeated measurements is increased by increasing the number of trials of the experiment, thereby reducing the standard deviation of the measurement data.

In a sample survey, it is to be expected that repeated sampling will give a somewhat different result for each sample. Small samples are more variable than larger ones. Techniques exist for determining how large a sample should be in order to be confident that the true value of the quantity being estimated is reasonably close to the value observed in the sample.

Polling conducted before an election, for example, often gives the percentage of voters expected to vote for a particular candidate and also a margin of error which is to do with the accuracy of the sampling process and, in particular, with the size of the sample of the population that was surveyed.

It is possible to devise classroom experiments to demonstrate the increasing variability that occurs with samples of decreasing size. One way to do this is as follows:

Categorical data

Assume a total class size of 30 people. Ask everyone in the class individually which of two comparable products they prefer. Calculate the percentage of people in the class who prefer product A. Now partition the class randomly into two groups of 15. Again, calculate the percentage of people in each group who prefer brand A. Continue this process by randomly partitioning the class into groups of 10 then 6, 5 and 3.

For each group size, compare the difference between the largest and smallest calculated percentage preferring product A.

In a computer simulation of this experiment, it was clear that smaller differences between the largest and smallest percentage of subjects in each group preferring product A were associated with larger group sizes.

Tina wants to know which new smart phone she should buy and decides to base her decision on other people’s opinions. She decides to interview some people at the Retirement Home where her grandparents live one Sunday afternoon.

Is this an example of Sampling Error or Measurement Error?

Sampling error

AMeasurement error

BSampling error

AMeasurement error

BWhat type of Sampling Error is this?

The sample is too small

AThe sample is not random

BThe sample does not adequately represent the population.

CThe sample is too small

AThe sample is not random

BThe sample does not adequately represent the population.

C

Adults attending a local cinema were asked the following question:

“How many times did you see a movie at this cinema last year?”

Is this an example of Sampling Error or Measurement Error?

Sampling error

AMeasurement error

BSampling error

AMeasurement error

BWhat type of Measurement Error is this?

The scale provided is inadequate.

AThis is an assumption based question.

BPoor and/or leading question wording.

CThe scale provided is inadequate.

AThis is an assumption based question.

BPoor and/or leading question wording.

CWhat is the main reason why this question is poor?

Poor and/or leading question wording

AThe people being surveyed need more information.

BRelies too heavily on respondent memory.

CPoor and/or leading question wording

AThe people being surveyed need more information.

BRelies too heavily on respondent memory.

C

Carry out investigations of phenomena, using the statistical enquiry cycle: A conducting surveys that require random sampling techniques, conducting experiments, and using existing data sets B evaluating the choice of measures for variables and the sampling and data collection methods used C using relevant contextual knowledge, exploratory data analysis, and statistical inference.

S7-3 Evaluate statistically based reports: A interpreting risk and relative risk B identifying sampling and possible non-sampling errors in surveys, including polls

Conduct an experiment to investigate a situation using statistical methods

Evaluate a statistically based report