Menu
Sign up Sign in

AI HL · AHL 8

Statistics 2

AHL 8.2

Sum of squared residuals and R²

AHL 8.2

Sum of squared residuals and R²

Target: I can find a residual and the sum of squared residuals \(SS_{res}\) by hand from a model and a table of data. (+3 more)

By the end of this section

Key points

Residuals and SS_res

  • A residual is the data value minus the value the model predicts: \(y-\hat y\). Above the curve is positive, below is negative.
  • \(SS_{res}=\sum(y-\hat y)^2\). Square first, then add. A smaller \(SS_{res}\) means a closer fit.
  • \(SS_{res}\) grows with the number of points, so use it to compare models fitted to the same data, not to compare different data sets.

The coefficient of determination

  • \(R^2\) lies between 0 and 1. A value of 1 means the model passes through every point.
  • \(R^2\) compares models and data sets of different sizes. About 0.7 or more is usually a good fit.
  • \(R^2\) is the proportion of the variation in y that the model accounts for: \(R^2=0.872\) means about 87.2%.
  • For a linear model, \(R^2=r^2\). The sign of r is the direction. \(R^2\) is never negative.
  • Adding a term to a polynomial never lowers \(R^2\), so a cubic fits at least as well as a quadratic. Ask whether the extra bend makes sense in the context.

Predicting from a model

  • Use the stored regression function to predict. Rounded coefficients can change the third significant figure.
  • Inside the range of the data, a prediction is interpolation and is usually reliable if \(R^2\) is high.
  • Outside the range it is extrapolation. A polynomial can turn and give impossible values, and a high \(R^2\) says nothing about where there are no data.
  • To comment, say that the value is outside the range, and describe what the model does there (it falls, it goes negative) and why that cannot be true in the context.

On the calculator

R² from a linear regression

  • Run linear regression on the two lists.
  • Read r and R² from the results. For a linear model \(R^2=r^2\).
  • Read the model's gradient and intercept from the same screen.

Screenshot and video to come

R² from a quadratic or cubic regression

  • Run quadratic or cubic regression on the two lists.
  • Read the coefficients and \(R^2\) from the results.
  • Store the regression function so you can graph the curve and predict values.

Screenshot and video to come

Residuals and SS_res on the calculator

  • Put the model in as a function and evaluate it at each x.
  • In a list, calculate the data value minus the model value.
  • Square each residual and add them to find \(SS_{res}\).

Screenshot and video to come

Explore: which model fits?

The dots are data. Choose a model: the orange curve is the fitted model and the teal segments are the residuals. Compare SSres and R² for all five, then look at what each model does beyond the data.

ModelSSresR²

For the exponential and the power model, R² is the R² of the straight line through (x, ln y) or (ln x, ln y), as the calculator gives it. SSres is always measured on y.

Choose a model.

  1. Choose "Crop yield" and go through the five models. Which has the highest R²? Which would you trust, and why?
  2. The cubic never has a lower R² than the quadratic. Does that mean the cubic is the better model? What would you want to know about the context?
  3. Tick "Show beyond the data" for the quadratic and the cubic. What do they do on the right? Could the real quantity do that?
  4. For the caffeine data, compare the exponential model with the quadratic. Which one makes sense a long time after the drink?

Explore: what "least squares" means

Drag the two orange handles to place a line among the points. Each residual is a vertical segment, and its square is drawn beside it. Make the squares add up to as little as you can.

SSres =

Sum of the squares

Your line
Least squares line
Smallest possible sum

Drag a handle.

  1. Move the line so that one square gets smaller. What happens to the squares of the other points? Can you make every square small at once?
  2. Why are the residuals squared before they are added? What would go wrong if you added the residuals themselves?
  3. Press "Show the least squares line". Move one handle a little in either direction. Why does the sum go up both ways?
  4. Choose "Odd point". Where does the least squares line go, and why does the odd point have so much pull on it?

Worked examples

Problems like the ones in class. Pick one, try it, then open the working.
Residuals and SS_res by hand

A coach company models the fuel used, F litres, on a journey of d km by \(F=2.1+0.34d\). The table shows six recorded journeys. Find the residual for each journey (data value minus model value), then find \(SS_{res}\).

d (km)102035507090F (litres)5.98.714.618.62633.4
Working
  1. Model values

    \(\hat F=2.1+0.34d\): 5.5, 8.9, 14, 19.1, 25.9, 32.7

    Put each d into the model. \(\hat F\) is the value the model predicts.

  2. 102030405060708090100510152025303540d (km)F (litres)
    Residuals

    \(F-\hat F\): 0.4, −0.2, 0.6, −0.5, 0.1, 0.7

    Data minus model. Positive means the point is above the line.

  3. Square each residual

    0.16, 0.04, 0.36, 0.25, 0.01, 0.49

    Squares are never negative, so residuals above and below the line cannot cancel.

  4. Add

    \(SS_{res}=\) \(0.16 + 0.04 + 0.36 + 0.25 + 0.01 + 0.49 = 1.31\)

    A smaller \(SS_{res}\) means the model is closer to the data.

Video walkthrough coming
R² for a linear model

The table gives the value, £y, of a used laptop of age x years. Fit a linear model on the calculator and find the coefficient of determination \(R^2\) to 3 significant figures.

x (years)12345678y (£)1210920874749680436385255
Working
  1. Linear regression

    From the calculator: linear regression: a = −128, b = 1260

    The model is \(y=-128x+1260\).

  2. Read r

    r = −0.985

    The calculator gives the Pearson correlation coefficient r. It is negative because y falls as x rises.

  3. Square it

    \(R^2=r^2=(−0.985)^2\) = 0.971

    For a linear model \(R^2=r^2\). The calculator also shows \(R^2\) directly.

  4. Meaning

    About 97.1% of the variation in y is explained by the linear model

    \(R^2\) is between 0 and 1. Close to 1 is a good fit.

Video walkthrough coming
Comparing a linear and a quadratic model

An agricultural scientist records the crop yield, y tonnes per hectare, when 10x kg of fertiliser per hectare is used. Find \(R^2\) for a linear model and for a quadratic model. Which fits better?

x (10 kg)0369121518y (t/ha)2.75.26.477.17.15.5
Working
  1. Linear model

    From the calculator: linear regression: \(R^2\) = 0.391

    A low value: the points rise and then fall, so a line does not fit.

  2. 2468101214161820123456789x (10 kg)y (t/ha)
    Quadratic model

    From the calculator: quadratic regression: a = −0.0364, b = 0.808, c = 2.84, \(R^2\) = 0.982

    The calculator shows \(R^2\) with the coefficients.

  3. Compare

    0.982 > 0.391

    The quadratic (teal curve) is much closer to the points than the line (orange). The larger \(R^2\) is the better fit, and the shape agrees with the context.

Video walkthrough coming
R² for a cubic model

The flow, y m³/s, of a river is recorded each day for eight days after a storm (x = 0 to 7). Fit a cubic model and state \(R^2\) to 3 significant figures.

x (day)01234567y (m³/s)5.311.511.79.56.96.56.611.7
Working
  1. Cubic regression

    From the calculator: cubic regression: a = 0.287, b = −3.01, c = 7.89, d = 5.64

    Model: \(y = 0.287x^3 - 3.01x^2 + 7.89x + 5.64\)

  2. 12345678246810121416x (day)y (m³/s)
    Read R²

    \(R^2\) = 0.966

    Read it from the same results screen. About 96.6% of the variation in y is explained.

  3. Compare

    Quadratic \(R^2\) = 0.000549

    The data fall, then rise: two bends. A quadratic cannot do that, so the cubic is far better.

Video walkthrough coming
R² from the correlation coefficient

A researcher fits a linear model to data on the outdoor temperature and the amount of gas used to heat a house. Pearson's product-moment correlation coefficient is \(r=-0.934\). Find \(R^2\) and say what it means.

Working
  1. Square r

    \(R^2=r^2=(-0.934)^2\) = 0.872

    This works for a linear model only.

  2. Interpret

    About 87.2% of the variation in gas used is explained by the linear model

    The negative sign of r only shows the direction. \(R^2\) shows how much of the variation the model explains.

Video walkthrough coming
A cubic model for planetary data

The table gives the distance x AU of each of the eight planets of a distant star from the star, and the orbital period T years. Find a regression model \(T=ax^3+bx^2+cx+d\), giving a, b, c and d to 3 significant figures, and state \(R^2\).

x (AU)0.250.450.91.42.23.86.19T (years)0.09880.2380.6751.312.585.8511.921.3
Working
  1. Cubic regression

    From the calculator: cubic regression: a = −0.00767, b = 0.255, c = 0.703, d = −0.127

    a is small because x³ is large. Keep the 3 significant figures: the leading zeros do not count.

  2. Write the model

    \(T = -0.00767x^3 + 0.255x^2 + 0.703x - 0.127\)

    Quote each coefficient to 3 significant figures.

  3. Read R²

    \(R^2\) = 1.00

    A very good fit inside the data range.

Video walkthrough coming
Predicting beyond the data

Use the cubic model from the last example to predict the orbital period of a planet at 30 AU from its star. Comment on the reliability of your prediction.

Working
  1. Evaluate

    T(30) = 43.7 years

    Use the stored regression function, not the rounded coefficients.

  2. Where is x = 30?

    Data range: 0.25 to 9 AU. 30 is outside it.

    A prediction inside the range is interpolation. Outside it is extrapolation, and it is much less reliable.

  3. 510152025303540−1010203040506070x (AU)T (years)P
    Look at the curve

    The model reaches a maximum at x = 23.5 AU, then falls

    Beyond that point the model says a planet further out takes less time to orbit. That cannot be true.

  4. Comment

    Not reliable

    x = 30 is more than 3 times the largest x in the data, and the cubic turns downwards after 23.5 AU. A high \(R^2\) inside the data does not make the model reliable outside it.

Video walkthrough coming