Unbiased Estimates

An estimator is a random variable that is used to estimate a population parameter

An estimator is called unbiased if its expected value is equal to the population parameter

An estimate from an unbiased estimator is called an unbiased estimate
This means that the mean of the unbiased estimates will get closer to the population parameter as more samples are taken

The sample mean is an unbiased estimate for the population mean
- $\bar{x} = \frac{\sum x}{n}$
The sample variance is not an unbiased estimate for the population variance
- $s_{n}^{2} = \frac{\sum {(x - \bar{x})}^{2}}{n} = \frac{\sum x^{2}}{n} - {(\bar{x})}^{2}$
- On average the sample variance will underestimate the population variance
- As the sample size increases the sample variance gets closer to the unbiased estimate

A sample of n data values (x₁, x₂, ... etc) can be used to find unbiased estimates for the mean and variance of the population
An unbiased estimate for the mean μ of a population can be calculated using
- $\bar{x} = \frac{\sum x}{n}$
An unbiased estimate for the variance σ² of a population can be calculated using
- $s_{n - 1}^{2} = \frac{n}{n - 1} s_{n}^{2}$
- This is given in the formula booklet
- This can also be written as
  - Notice that dividing by $n$ gives a biased estimate but dividing by $n - 1$ gives an unbiased estimate

Different calculators can use different notations for $s_{n - 1}^{2}$
- $σ_{n - 1}^{2}$ , $s_{}^{2}$ , ${\hat{s}}_{}^{2}$ are notations you might see
- You may also see the square roots of these

Unfortunately s_n_-1is not an unbiased estimate for the standard deviation of the population
It is better to work with the unbiased variance rather than standard deviation
There is not a formula for an unbiased estimate for the standard deviation that works for all populations

You do not need to learn this proof
- It is simply here to help with your understanding
Suppose the population of X has mean μ and variance σ²
Take a sample of n observations
- X_1,X_2,..., X_n
- E(X_i) = μ
Using the formula for a linear combination of n independent variables:

\begin{array}{rcl} E (\bar{X}) & = & E (\frac{X_{1} + X_{2} + \dots + X_{n}}{n}) \\ = & \frac{E (X_{1}) + E (X_{2}) + \dots + E (X_{n})}{n} \\ = & \frac{μ + μ + \dots + μ}{n} \\ = & \frac{n μ}{n} \\ = & μ \end{array}

As $\begin{array}{rcl} E (\bar{X}) & = & μ \end{array}$ this shows the formula will produce an unbiased estimate for the population mean

You do not need to learn this proof
- It is simply here to help with your understanding
Suppose the population of X has mean μ and variance σ²
Take a sample of n observations
- X_1,X_2,..., X_n
- E(X_i) = μ
- Var(X_i) = σ²
Using the formula for a linear combination of n independent variables:

\begin{array}{rcl} Var (\bar{X}) & = & Var (\frac{X_{1} + X_{2} + \dots + X_{n}}{n}) \\ = & \frac{Var (X_{1}) + Var (X_{2}) + \dots + Var (X_{n})}{n^{2}} \\ = & \frac{σ^{2} + σ^{2} + \dots + σ^{2}}{n^{2}} \\ = & \frac{n σ^{2}}{n^{2}} \\ = & \frac{σ^{2}}{n} \end{array}

It can be shown that $E ({\bar{X}}^{2}) = μ^{2} + \frac{σ^{2}}{n}$
- This comes from rearranging $Var (\bar{X}) = E ({\bar{X}}^{2}) - {[E (\bar{X})]}^{2}$
It can be shown that $E (X^{2}) = E ({X_{i}}^{2}) = μ^{2} + σ^{2}$
- This comes from rearranging $Var (X) = E (X^{2}) - {[E (X)]}^{2}$
Using the formula for a linear combination of n independent variables:

\begin{array}{rcl} E (S_{n}^{2}) & = & E (\frac{\sum X_{i}^{2}}{n} - {\bar{X}}^{2}) \\ = & \frac{\sum E (X_{i}^{2})}{n} - E ({\bar{X}}^{2}) \\ = & \frac{\sum (μ^{2} + σ^{2})}{n} - (μ^{2} + \frac{σ^{2}}{n}) \\ = & \frac{n (μ^{2} + σ^{2})}{n} - (μ^{2} + \frac{σ^{2}}{n}) \\ = & μ^{2} + σ^{2} - (μ^{2} + \frac{σ^{2}}{n}) \\ = & σ^{2} - \frac{σ^{2}}{n} \\ = & \frac{n σ^{2} - σ^{2}}{n} \\ = & \frac{n - 1}{n} σ ² \end{array}

As $E (S_{n}^{2}) \neq σ^{2}$ this shows that the sample variance is not unbiased
- You need to multiply by $\frac{n}{n - 1}$
- $E (S_{n - 1}^{2}) = σ^{2}$

Check the wording of the exam question carefully to determine which of the following you are given:
- The population variance: $σ^{2}$
- The sample variance: $s_{n}^{2}$
- An unbiased estimate for the population variance: $s_{n - 1}^{2}$

The times, $X$ minutes, spent on daily revision of a random sample of 50 IB students from the UK are summarised as follows.

n = 50

\sum x = 6174

s_{n}^{2} = 1384.3

Calculate unbiased estimates of the population mean and variance of the times spent on daily revision by IB students in the UK.

4-6-2-ib-ai-hl-unbiased-estimates-we-solution

Unbiased Estimates (DP IB Maths: AI HL)