Menu
Sign up Sign in

AI HL · AHL 8

Statistics 2

0 of 56 skills done

0
AHL 8.10

Unbiased estimates of the mean and variance; validity and reliability

AHL 8.10

Unbiased estimates of the mean and variance; validity and reliability

Target: I can find unbiased estimates of the population mean and variance from a list of data, using the calculator. (+4 more)

By the end of this section

Key points

Estimating from a sample

  • A sample is used to estimate the population. The sample mean \(\bar x\) is an unbiased estimate of \(\mu\): over many samples it averages to the true value.
  • The variance of a sample, \(s_n^2\) (dividing by n), tends to be too small. The unbiased estimate is \(s_{n-1}^2=\dfrac{n}{n-1}s_n^2\).
  • Why n − 1: the data lie closer to their own mean \(\bar x\) than to the true mean \(\mu\), so \(\sum(x-\bar x)^2\) is a little too small. Dividing by \(n-1\) instead of n corrects it exactly on average: \(E(s_{n-1}^2)=\sigma^2\).
  • The definition: \(s_{n-1}^2=\dfrac{\sum(x-\bar x)^2}{n-1}=\dfrac{1}{n-1}\left(\sum x^2-\dfrac{(\sum x)^2}{n}\right)\). Use the second form when you are given \(\sum x\) and \(\sum x^2\).
  • The calculator's two standard deviations: \(\sigma_x\) (divides by n) is \(s_n\), and \(s_x\) (divides by \(n-1\)) is \(s_{n-1}\). To estimate the population, use \(s_x\), and square it for the variance.
  • Common error: \(s_{n-1}\) is the square root of an unbiased estimate of \(\sigma^2\), but it is not an unbiased estimate of \(\sigma\).
  • Take the standard deviation that divides by \(n-1\), then square it for the variance. Never square the one that divides by n.

Converting and finding n

  • From \(s_n\) to \(s_{n-1}\): \(s_{n-1}=\sqrt{\dfrac{n}{n-1}}\,s_n\). The unbiased value is always a little larger.
  • Given the ratio \(\dfrac{s_{n-1}}{s_n}=r\): square it, \(\dfrac{n}{n-1}=r^2\), then solve for n. n is a whole number.

Sampling and extreme values

  • A random sample gives every member of the population the same chance of being chosen. Samples that people choose to answer, or that come from one place, are biased.
  • Name the direction of a bias: say whether the estimate is likely to be too big or too small, and why.
  • Remove an extreme value only for a reason you can state, such as an impossible value or a recording error. Say so, then estimate from the rest.

Validity and reliability

  • Reliability: the results are consistent. Tests: test-retest (same test twice), parallel forms (two versions), internal consistency (questions on the same thing agree).
  • Validity: the question or test measures what it is meant to measure. Content validity: do the questions cover and measure the right thing? Criterion-related validity: do the results agree with an outside measure?
  • Reasons a question lacks content validity: leading, vague words, two questions in one, asks for opinion instead of a measurement.
  • Anonymous answers are more honest, which improves validity.

On the calculator

Unbiased estimates with 1-Variable Stats

  • Enter the data in a list and run 1-Variable Stats.
  • Read \(\bar x\) and the sample standard deviation \(s_x\), which divides by \(n-1\). That is \(s_{n-1}\). Ignore \(\sigma_x\), which divides by n.
  • Square \(s_{n-1}\) for the unbiased variance. Check what your calculator calls each one.

Screenshot and video to come

Worked examples

Problems like the ones in class. Pick one, try it, then open the working.
Unbiased estimates from a list

Seven randomly chosen days, the waiting time (minutes) for a bus was 8, 13, 9, 15, 11, 12, 18. Find unbiased estimates of the mean and the variance of the waiting time on all days.

Working
  1. Enter the data

    From the calculator: 1-Variable Stats on the list of seven times

    Put the data in a list first.

  2. Read the mean

    \(\bar x = 12.3\)

    The sample mean is an unbiased estimate of \(\mu\).

  3. Read the standard deviation

    \(s_{n-1} = 3.450\) (the calculator's sample SD), \(\sigma_x=3.194\) (divides by n)

    Take the one that divides by n − 1. Check which name your calculator uses.

  4. Square it

    \(s_{n-1}^2 = 3.450^2 = 11.9\)

    Check: \(\frac{7}{6}\times 3.194^2=11.9\).

Video walkthrough coming
From s_n to s_{n-1}

The masses of 15 eggs have standard deviation 3.2 g, calculated by dividing by n. Find the unbiased estimate of the population standard deviation, \(s_{n-1}\).

Working
  1. The rule

    \(s_{n-1}^2=\dfrac{n}{n-1}s_n^2\)

    Square both sides first if you like: the variance is what is scaled.

  2. Substitute

    \(s_{n-1}=\sqrt{\dfrac{15}{14}}\times 3.2\)

    n = 15.

  3. Evaluate

    \(s_{n-1}=3.31\) g

    Always a little bigger than \(s_n\).

Video walkthrough coming
Find n from the ratio

For a sample, \(s_{n-1}\) is 1.05409 times bigger than \(s_n\). Find the sample size n.

Working
  1. Write the ratio

    \(\dfrac{s_{n-1}}{s_n}=\sqrt{\dfrac{n}{n-1}}=1.05409\)

    Divide the two standard deviations.

  2. Square

    \(\dfrac{n}{n-1}=1.05409^2=1.1111\)

    Get rid of the square root.

  3. Solve for n

    \(n=1.1111(n-1)\) gives \(n=10.00\), so \(n=10\)

    n is a whole number.

Video walkthrough coming
A survey with an extreme value

A health officer leaves a questionnaire at a gym reception: 'How many hours do you exercise each week?' Seven were posted back: 4, 6, 5, 3, 7, 5, 120. (a) Explain why 120 can reasonably be removed. (b) For the other six, find unbiased estimates of the mean and variance. (c) Comment on the sampling method. (d) Is the mean more likely to be an over- or under-estimate?

Working
  1. Remove the extreme value

    120 hours is 17 hours a day, every day

    It is not a possible answer: probably a mistake. Say why before you remove it.

  2. Estimates

    From the calculator: \(\bar x = 5.00\), \(s_{n-1}=1.414\), so \(s_{n-1}^2=2.00\)

    Six values left, so divide by 5.

  3. Sampling method

    Only gym members who chose to post it back were asked

    This is a biased sample: not random, and it leaves out people who do not exercise.

  4. Direction of the bias

    Overestimate

    People who go to a gym exercise more than most, and keen people are more likely to reply.

Video walkthrough coming
Reliability and validity

A teacher gives eight students the same anxiety questionnaire twice, two weeks apart. Scores: first sitting 14, 18, 11, 20, 16, 9, 13, 17; second sitting 15, 17, 12, 19, 18, 10, 12, 16. (a) Name the reliability test and find the correlation coefficient r. (b) Is the questionnaire reliable? (c) One question reads 'Do you agree that exams are unfair and stressful?' Say why it lacks content validity. (d) How would the teacher test criterion-related validity?

Working
  1. Name the test

    Test-retest reliability

    Same test, same people, two occasions. Reliable means the results are consistent.

  2. Find r

    From the calculator: linear regression of second on first: \(r=0.944\)

    r close to 1 means the two sittings agree, so the test is reliable.

  3. Content validity

    The question is leading and asks about two things at once

    Validity means the test measures what it claims to. A leading question pushes the answer, and 'unfair and stressful' is two ideas.

  4. Criterion-related validity

    Compare the scores with an outside measure of anxiety, such as a counsellor's rating

    Does the score agree with something that measures the same thing? That is criterion-related validity.

Video walkthrough coming
The definition of the unbiased variance

A random sample of 5 observations is 6, 9, 4, 11, 5. Find the unbiased estimate of the population variance from the definition, and check it against the calculator's two standard deviations.

Working
  1. The mean

    \(\bar x=\dfrac{6+9+4+11+5}{5}=7\)

    The sample mean is the unbiased estimate of \(\mu\).

  2. Sum of squared deviations

    \(\sum(x-\bar x)^2=(-1)^2+2^2+(-3)^2+4^2+(-2)^2=34\)

    Deviations from \(\bar x\): −1, 2, −3, 4, −2.

  3. Divide by n − 1

    \(s_{n-1}^2=\dfrac{34}{5-1}=8.5\)

    Dividing by 4, not 5. Dividing by 5 gives 6.8, which is too small.

  4. Check with the sums

    \(\sum x=35\), \(\sum x^2=279\): \(\dfrac{279-35^2/5}{4}=\dfrac{34}{4}=8.5\)

    The same value, without finding any deviations.

  5. The calculator

    From the calculator: 1-Variable Stats gives \(\sigma_x=2.61\) and \(s_x=2.92\)

    \(s_x^2=8.5\) (squaring the unrounded \(s_x=2.9155\); 2.92 squared gives 8.53 only because of rounding). The estimate of \(\sigma^2\) comes from \(s_x\), not \(\sigma_x\).

Video walkthrough coming
Which standard deviation?

A teacher times 10 randomly chosen chess moves with a calculator. 1-Variable Stats shows \(\sigma_x=4.20\) s and \(s_x=4.43\) s. Which value should she use to estimate the standard deviation of all moves? Find the unbiased estimate of the variance, and explain why \(\sigma_x\) is not used.

Working
  1. Choose

    Use \(s_x=4.43\) (this is \(s_{n-1}\))

    \(s_x\) divides by \(n-1\). \(\sigma_x\) divides by n and describes the sample only.

  2. Square

    \(s_{n-1}^2=4.43^2=19.6\)

    Check: \(\dfrac{10}{9}\times4.20^2=19.6\).

  3. Why not \(\sigma_x\)

    \(\sigma_x^2=17.6<19.6\)

    The data lie closer to \(\bar x\) than to \(\mu\), so dividing by n gives a value that is too small on average. Dividing by \(n-1\) corrects it.

Video walkthrough coming