1.07 Sample standard deviation

Standard deviation is a measure commonly used to describe how widely spread the observations in a set of data are. It is a measure of spread, like the  range and interquartile range  , however the standard deviation is related to the mean value rather than the median value.

The sample standard deviation, denoted s, is a number that represents how far on average each score is away from the mean.

The formula to find standard deviation appears quite complex and involves a lot of calculations. Fortunately, most calculators can determine the standard deviation for us:

s=\sqrt{\dfrac{1}{n-1}\sum_{i=1}^n\left(\overline{x}-x_i \right)^2}

Some observations about standard deviation:

• The standard deviation represents a distance from the mean, so it is non-negative.

• If most of the data are clustered around the mean, then the standard deviation is small (close to 0).

• If there are many data points scattered away from the mean value, the standard deviation will be large.

• The standard deviation is one number, which is representative of the entire set of data.

When we wish to know certain statistics to do with large numbers of subjects, it may be too expensive or time consuming to collect data on every subject. For this reason, it is often useful to estimate the population statistics on the basis of random samples drawn from the whole population.

The formula given above is used to calculate the standard deviation for a sample of data. If we instead know data for each subject in the population, the formula is slightly different.

Suppose the same mathematics test was administered to 10\,000 year 11 students. Education authorities are interested in how well the test distinguished between different levels of ability among the students. They consider that a better test would have a wider spread of scores than an inferior one.

A random sample of 50 student results is drawn from the 10\,000. It is standard practice to use digital technology in some form to calculate the mean and the standard deviation of a set of scores. In doing so, it would be important in this case to choose the sample standard deviation option rather than the population standard deviation.

If the mean of the sample is 67\% and the sum of the squared differences from the mean is 1035, the sample standard deviation is s=\sqrt{\frac{1}{50-1}\times 1035} \approx 4.6

Assuming that the 10\,000 scores had an approximately normal distribution, we expect the majority of scores to lie within 1 standard deviation from the mean and nearly all of them to lie within 3 standard deviations from the mean.

This suggests that the test had enough easy questions so that most students could achieve scores above 53\%, which is 3 times the standard deviation below the mean. On the other hand, three times the standard deviations above the mean is 81\%, so there may have been some very hard questions as well that only a couple of students got right.

Examples