Australian Curriculum 12 General Mathematics - 2020 Edition
topic badge
2.01 Bivariate data and line of best fit
Lesson

The least-squares regression line

In the previous chapter, we learnt to describe the relationship between numerical variables.  If there is a relationship, we can recognise when it is linear or non-linear, strong or weak, and positive or negative. 

We can draw a line, by eye, that best fits the points in a scatter plot.  The images below show some examples of a line of best fit.  The line of best fit does not necessarily go through the data points.  It is positioned so that it minimises the overall distance from the points.

Constructing a line of best fit in this way can be very useful because the line summarises all of the data points in a way that allows us to predict the value of the response (dependent) variable that corresponds to a given value of the explanatory (independent) variable, or vice-versa.  However, it takes a certain amount of judgement to choose the position and angle of the line. Constructing the line by eye is not very reliable, especially if the scatter plot does not show a strong relationship.

Fortunately, there are mathematical calculations that we can use to determine a line of best fit with more consistency.  In this lesson, we will learn to construct a least-squares regression line

The best way to understand it is through a demonstration.  Try moving the points to see how the least-squares regression line follows the data.

 

Calculating the least-squares regression line

The least-squares regression line is a straight line, so it can be represented by a linear function, in gradient-intercept form:

Least squares regression line notation

$y=mx+c$y=mx+c

$y$y is the predicted value of the response (dependent) variable

$x$x is our explanatory (independent) variable

$m$m is the gradient of our line

$c$c is the vertical intercept (or $y$y-intercept) of our line.

 

It is common practice to use the symbol $\hat{y}$^y (which is pronounced "y-hat"), instead of $y$y, as the variable for the predicted value of the response variable.

 

Unless we are working with a very small data set, the calculations to determine the least-squares regression line are too tedious to do by hand.  In this course, we will use technology for these calculations.

Select the brand of calculator you use below to work through an example of using a calculator to generate a scatter plot and find the equation of the least squares regression line.

 

Casio Classpad

How to use the CASIO Classpad to complete the following tasks regarding scatterplots and linear regression.

The average number of pages read to a child each day and the child’s growing vocabulary are measured. Consider the data set given below:

Pages read per day ($x$x) $25$25 $27$27 $29$29 $3$3 $13$13 $31$31 $18$18 $29$29 $29$29 $5$5
Total vocabulary ($y$y) $402$402 $440$440 $467$467 $76$76 $220$220 $487$487 $295$295 $457$457 $460$460 $106$106
  1. Use your calculator to generate a scatterplot of the data.

  2. Using your calculator, find an equation for the least squares regression line of $y$y on $x$x.

    Give your answer in the form $y=ax+b$y=ax+b. Give all values to two decimal places.

TI Nspire

How to use the TI Nspire to complete the following tasks regarding scatterplots and linear regression.

The average number of pages read to a child each day and the child’s growing vocabulary are measured. Consider the data set given below:

Pages read per day ($x$x) $25$25 $27$27 $29$29 $3$3 $13$13 $31$31 $18$18 $29$29 $29$29 $5$5
Total vocabulary ($y$y) $402$402 $440$440 $467$467 $76$76 $220$220 $487$487 $295$295 $457$457 $460$460 $106$106
  1. Use your calculator to generate a scatterplot of the data.

  2. Using your calculator, find an equation for the least squares regression line of $y$y on $x$x.

    Give your answer in the form $y=ax+b$y=ax+b. Give all values to two decimal places.

 

Practice question

Question 1

Use technology to find the line of best fit for the data below. Write the equation with the coefficient and constant term to the nearest two decimal places.

$x$x $24$24 $37$37 $19$19 $31$31 $32$32 $22$22 $14$14 $30$30 $23$23 $40$40
$y$y $-7$7 $-8$8 $-3$3 $-6$6 $-9$9 $-8$8 $-2$2 $-8$8 $-8$8 $-12$12

Interpreting the line of best fit

Now that we can determine the equation of the least-squares regression line, we are able to make important observations about the original data.  When we are analysing data, it is important that we consider the context.

In the calculator examples above, $y$y represents the vocabulary (the response variable) and $x$x represents the number of pages read per day (the explanatory variable).

We found the least squares regression line in the form $y=mx+c$y=mx+c, to be $y=14.87x+30.26$y=14.87x+30.26.

The $m$m value displayed is the gradient of the least-squares regression line.  Since this is a positive number, it indicates that there is a positive relationship between the variables.  In the context of this example, this tells us that for each additional page of reading per day, a child's vocabulary increases by approximately $15$15 words.

The $c$c value provided is the vertical intercept of the line of best fit and, in the context of this example, tells us that a child that does no reading would have a vocabulary of approximately $30$30 words. 

 

Interpreting the least-squares regression line

Given a least squares squares regression line of the form $y=mx+c$y=mx+c

The $m$m value shows the gradient:

  • if the gradient is positive, when the explanatory increases by $1$1 unit, the response variable increases by $m$m units. 
  • if the gradient is negative, when the explanatory increases by $1$1 unit, the response variable decreases by $m$m units. 

The $c$c value shows the vertical intercept (also known as the $y-$yintercept):

  • when the explanatory variable is $0$0, the value of the response variable is $c$c.

When we interpret the vertical intercept, we need to consider if it makes sense for the explanatory variable to be zero and the response variable to have a value indicated by $c$c.

 

Practice questions

Question 2

A least squares regression line is given by $y=3.59x+6.72$y=3.59x+6.72.

  1. State the gradient of the line.

  2. Which of the following is true?

    The gradient of the line indicates that the bivariate data set has a positive correlation.

    A

    The gradient of the line indicates that the bivariate data set has a negative correlation.

    B

    The gradient of the line indicates that the bivariate data set has a positive correlation.

    A

    The gradient of the line indicates that the bivariate data set has a negative correlation.

    B
  3. Which of the following is true?

    If $x$x increases by $1$1 unit, then $y$y increases by $3.59$3.59 units.

    A

    If $x$x increases by $1$1 unit, then $y$y decreases by $3.59$3.59 units.

    B

    If $x$x increases by $1$1 unit, then $y$y decreases by $6.72$6.72 units.

    C

    If $x$x increases by $1$1 unit, then $y$y increases by $6.72$6.72 units.

    D

    If $x$x increases by $1$1 unit, then $y$y increases by $3.59$3.59 units.

    A

    If $x$x increases by $1$1 unit, then $y$y decreases by $3.59$3.59 units.

    B

    If $x$x increases by $1$1 unit, then $y$y decreases by $6.72$6.72 units.

    C

    If $x$x increases by $1$1 unit, then $y$y increases by $6.72$6.72 units.

    D
  4. State the value of the $y$y-intercept.

Question 3

The price of various second-hand Mitsubishi Lancers are shown below.

Age $1$1 $2$2 $0$0 $5$5 $7$7 $4$4 $3$3 $4$4 $8$8 $2$2
Value (dollars) $16000$16000 $13000$13000 $21990$21990 $10000$10000 $8600$8600 $12500$12500 $11000$11000 $11000$11000 $4500$4500 $14500$14500
  1. Find the equation of the Least Squares Regression Line for the price ($y$y) in terms of age ($x$x).

    Round all values to the nearest integer.

  2. State the value of the $y$y-intercept.

  3. The value of the $y$y-intercept indicates that when a Mitsubishi Lancer is brand new, its value is, on average, $\editable{}$ dollars.

  4. Does the interpretation in the previous part make sense in this context?

    Yes, when the explanatory variable has a value of zero, this is still within the data range and the value of the dependent variable makes sense.

    A

    No, when the explanatory variable has a value of zero, this is outside the data range and the value of the dependent variable does not make sense.

    B

    Yes, when the explanatory variable has a value of zero, this is still within the data range and the value of the dependent variable makes sense.

    A

    No, when the explanatory variable has a value of zero, this is outside the data range and the value of the dependent variable does not make sense.

    B
  5. State the gradient of the line.

  6. Which of the following is true?

    If the explanatory variable increases by $1$1 unit, then the dependent variable increases by $1694$1694 units.

    A

    If the explanatory variable increases by $1$1 unit, then the dependent variable increases by $18407$18407 units.

    B

    If the explanatory variable increases by $1$1 unit, then the dependent variable decreases by $18407$18407 units.

    C

    If the explanatory variable increases by $1$1 unit, then the dependent variable decreases by $1694$1694 units.

    D

    If the explanatory variable increases by $1$1 unit, then the dependent variable increases by $1694$1694 units.

    A

    If the explanatory variable increases by $1$1 unit, then the dependent variable increases by $18407$18407 units.

    B

    If the explanatory variable increases by $1$1 unit, then the dependent variable decreases by $18407$18407 units.

    C

    If the explanatory variable increases by $1$1 unit, then the dependent variable decreases by $1694$1694 units.

    D

 

Correlation coefficient ($r$r)

The correlation coefficient was introduced in the previous chapter as a measure that tells us the strength of a relationship between two variables.  It is denoted by the letter $r$r.  The sign of $r$r also tells us the direction of the relationship.

Some key aspects of the correlation coefficient are summarised below, followed by examples that show how we can calculate the correlation coefficient with our CAS calculator.

 

Correlation coefficient
  • A perfect positive correlation has a value of $1$1. That means that if we graphed the variables on a scatter plot, it would show that all the data points lie exactly on a straight line with a positive gradient.
  • A perfect negative correlation has a value of $-1$1 in which case all data points lie exactly on a straight line with a negative gradient.
  • A value of $0$0 indicates that there is no relationship between the variables.
  • To be more descriptive about the relationship between variables, we can further divide up the line to indicate other values with descriptions like "weak", "moderate" and "strong", in the figure below.  

The strength of correlation depends on the size of the r value, so we can ignore the positive or negative sign:

  • if the size is less than 0.5, then we have a weak correlation; 
  • if the size of the correlation is greater than 0.8, then we have a strong correlation. 
  • if the size is between 0.5 and 0.8, then we a moderate correlation.

 

Select the brand of calculator you use below to work through an example of using a calculator to find the correlation coefficient.

Casio Classpad

How to use the CASIO Classpad to complete the following task to find the correlation coefficient.

A café records the temperature and number of hot chocolates sold on the same day of the week for 10 weeks. Consider the data set given below:

Temperature  ($x\ ^\circ C$x °C) $14$14 $25$25 $18$18 $19$19 $27$27 $24$24 $16$16 $12$12 $26$26 $22$22
Hot chocolates sold ($y$y) $35$35 $8$8 $20$20 $22$22 $5$5 $23$23 $32$32 $34$34 $8$8 $14$14
  1. Calculate the value of the correlation coefficient ($r$r).

    Give your answer to three decimal places.

TI Nspire

How to use the TI Nspire to complete the following task to find the correlation coefficient.

A café records the temperature and number of hot chocolates sold on the same day of the week for 10 weeks. Consider the data set given below:

Temperature  ($x\ ^\circ C$x °C) $14$14 $25$25 $18$18 $19$19 $27$27 $24$24 $16$16 $12$12 $26$26 $22$22
Hot chocolates sold ($y$y) $35$35 $8$8 $20$20 $22$22 $5$5 $23$23 $32$32 $34$34 $8$8 $14$14
  1. Calculate the value of the correlation coefficient ($r$r).

    Give your answer to three decimal places.

 

Practice question

Question 4

Given the following data:

x $1$1 $4$4 $7$7 $10$10 $13$13 $16$16 $19$19
y $4$4 $4.25$4.25 $4.55$4.55 $4.4$4.4 $4.45$4.45 $4.75$4.75 $4.2$4.2
  1. Calculate the correlation coefficient and give your answer to two decimal places.

  2. Choose the best description of this correlation.

    Moderate negative

    A

    Strong positive

    B

    Weak negative

    C

    Moderate positive

    D

    Strong negative

    E

    Weak positive

    F

    Moderate negative

    A

    Strong positive

    B

    Weak negative

    C

    Moderate positive

    D

    Strong negative

    E

    Weak positive

    F

Coefficient of determination ($r^2$r2)

When we discuss the coefficient of determination or the value of $r^2$r2, we can already tell that it must be related to the value of the correlation coefficient ($r$r) and something to do with measuring the relationship between two variables. 

Coefficient of determination

$r^2$r2 tells us the proportion of the response variable ($y$y) that can be explained by the variation in the explanatory variable ($x$x).

For example, if $r^2=0.92$r2=0.92 then we can say that $92%$92% of the variation in the response variable is explained by the variation in the explanatory variable.

Alternatively, we can say that $8%$8% of the variation in the response variable is not explained by the variation in the explanatory variable.

Unlike the correlation coefficient, the coefficient of determination is always a positive value, so it does not indicate if there is a positive or negative relationship between variables.

The closer $r^2$r2 is to $1$1, the more the variation in the response variable is explained by the variation in the explanatory variable.

If we already have the value of $r$r, we can square the value to get $r^2$r2.

So if $r=0.8$r=0.8, then $r^2=0.64$r2=0.64

If $r=-0.9$r=0.9, then $r^2=0.81$r2=0.81

If we don't already have the value of $r$r, our calculators will calculate it for us. In fact, the $r^2$r2 value is given on the same screen as the $r$r value.

Important - correlation is not causation! 

Just as with $r$r values when two variables have a strong relationship, that is $r^2$r2 is close to $1$1, we cannot say that one variable causes change in the other variable.

 

Select the brand of calculator you use below to work through an example of using a calculator to find the coefficient of determination.

 

Casio Classpad

How to use the CASIO Classpad to complete the following task to find the coefficient of determination.

A café records the temperature and number of hot chocolates sold on the same day of the week for $10$10 weeks. Consider the data set given below:

Temperature ($x\ ^\circ$x °C) $14$14 $25$25 $18$18 $19$19 $27$27 $24$24 $16$16 $12$12 $26$26 $22$22
Hot chocolates sold ($y$y) $35$35 $8$8 $20$20 $22$22 $5$5 $23$23 $32$32 $34$34 $8$8 $14$14
  1. Calculate the value of the coefficient of determination ($r^2$r2).

    Give your answer to three decimal places.

TI Nspire

How to use the TI Nspire to complete the following task to find the coefficient of determination.

A café records the temperature and number of hot chocolates sold on the same day of the week for $10$10 weeks. Consider the data set given below:

Temperature ($x\ ^\circ$x °C) $14$14 $25$25 $18$18 $19$19 $27$27 $24$24 $16$16 $12$12 $26$26 $22$22
Hot chocolates sold ($y$y) $35$35 $8$8 $20$20 $22$22 $5$5 $23$23 $32$32 $34$34 $8$8 $14$14
  1. Calculate the value of the coefficient of determination ($r^2$r2).

    Give your answer to three decimal places.

 

Practice questions

Question 5

A scientist investigated the link between the number of cancer cells killed by a certain drug and the strength of the drug used. The results were recorded and the coefficient of determination $r^2$r2 was found to be $0.92$0.92.

  1. Which of the following is true?

    Select all that apply.

    There is a strong relationship between the strength of the drug used and the cancer cells killed.

    A

    The number of cancer cells killed causes the strength of the drug used.

    B

    We cannot infer a causal relationship between strength of the drug used and the cancer cells killed.

    C

    The strength of the drug used causes the cancer cells to be killed.

    D

    There is a weak relationship between the strength of the drug used and the cancer cells killed.

    E

    There is a strong relationship between the strength of the drug used and the cancer cells killed.

    A

    The number of cancer cells killed causes the strength of the drug used.

    B

    We cannot infer a causal relationship between strength of the drug used and the cancer cells killed.

    C

    The strength of the drug used causes the cancer cells to be killed.

    D

    There is a weak relationship between the strength of the drug used and the cancer cells killed.

    E

Question 6

A linear association between two data sets is such that the coefficient of determination $r^2$r2 is $0.80$0.80.

  1. Calculate the correlation coefficient if the relationship is negative.

    Give your answer to four decimal places.

  2. Hence choose the option that best describes the strength of the relationship.

    Weak

    A

    Moderate

    B

    Strong

    C

    Weak

    A

    Moderate

    B

    Strong

    C

Question 7

A linear association between two data sets is such that the correlation coefficient is $-0.72$0.72.

What proportion of the variation can be explained by the linear relationship?

Give your answer to the nearest percent.

Question 8

The heights (in cm) and the weights (in kg) of $8$8 primary school children is shown on the scattergraph below.

Loading Graph...

  1. Calculate the value of the coefficient of determination.

    Give your answer to two decimal places.

  2. Hence or otherwise calculate the value of the correlation coefficient.

    Give your answer to two decimal places.

  3. What percentage of the variation in weight is accounted for by the height of the child?

    Give your answer to the nearest whole percent.

  4. Consider these two comments on the claim “The weight of a child is primarily influenced by their height.”

    Which do you think is most correct?

    This claim is valid and is supported by the strong relationship between the two variables.

    A

    While this claim is supported by a strong relationship between the two variables, we cannot state causality as there may be other factors influencing the outcome.

    B

    This claim is valid and is supported by the strong relationship between the two variables.

    A

    While this claim is supported by a strong relationship between the two variables, we cannot state causality as there may be other factors influencing the outcome.

    B

Outcomes

ACMGM054

calculate and interpret the correlation coefficient (r) to quantify the strength of a linear association

ACMGM057

model a linear relationship by fitting a least-squares line to the data

ACMGM059

interpret the intercept and slope of the fitted line

ACMGM060

use the coefficient of determination to assess the strength of a linear association in terms of the explained variation

What is Mathspace

About Mathspace