Sources of Errors

In any measurement procedure, there is inevitably some error. This is true when an experiment is being done to measure some physical quantity, and it is true in the case of surveys designed to quantify people's voting intentions or brand preferences.

We distinguish two kinds of error: systematic error and random error. 

Systematic Error

Systematic error occurs when there is something wrong with the measuring apparatus or with the procedure being used. An instrument may not have been correctly calibrated, for example, or a scale may have been consistently misread. In a survey, bias may have been introduced by poor design of the survey questions or by the manner in which the sample of the population to be surveyed was selected.

Once detected, systematic error can usually be reduced by improvements in the experimental procedure and by taking steps to eliminate bias due to the survey design and to the sampling method.

Random Error

Random error can be thought of as the remaining unexplained error after known sources of systematic error have been removed.

When several careful measurements of the same physical quantity are made, slightly different results may be reported at each trial due to small but uncontrolled fluctuations in the experimental conditions and by the impossibility of reading a measurement scale beyond some level of precision. The reported result of such a measurement is given as the mean of all the observed results together with an indication of how far the observations vary about the mean. The amount of variation is usually given by a statistic called the variance or by its square root, the standard deviation.

The accuracy of the result of a series of repeated measurements is increased by increasing the number of trials of the experiment, thereby reducing the standard deviation of the measurement data.

 

In a sample survey, it is to be expected that repeated sampling will give a somewhat different result for each sample. Small samples are more variable than larger ones. Techniques exist for determining how large a sample should be in order to be confident that the true value of the quantity being estimated is reasonably close to the value observed in the sample. 

Polling conducted before an election, for example, often gives the percentage of voters expected to vote for a particular candidate and also a margin of error which is to do with the accuracy of the sampling process and, in particular, with the size of the sample of the population that was surveyed.

Example

It is possible to devise classroom experiments to demonstrate the increasing variability that occurs with samples of decreasing size. One way to do this is as follows:

Categorical data

Assume a total class size of 30 people. Ask everyone in the class individually which of two comparable products they prefer. Calculate the percentage of people in the class who prefer product A. Now partition the class randomly into two groups of 15. Again, calculate the percentage of people in each group who prefer brand A. Continue this process by randomly partitioning the class into groups of 10 then 6, 5 and 3.

For each group size, compare the difference between the largest and smallest calculated percentage preferring product A. 

In a computer simulation of this experiment, it was clear that smaller differences between the largest and smallest percentage of subjects in each group preferring product A were associated with larger group sizes.

Worked Examples

Question 1

Question 2