Unbiased Estimators

An estimator is a rule built from sample data and used to guess an unknown population parameter such as μ\mu, pp or σ2\sigma^2. Different random samples give different numbers, so an estimator θ^\hat\theta is itself a random variable with a sampling distribution. It is unbiased for θ\theta when the mean of that distribution is the parameter itself:

Approximate sampling distributions of two competing estimators of the same parameter

Approximate sampling distributions of two competing estimators of the same parameter


E(θ^)=θfor every possible value of θE(\hat\theta) = \theta \quad \text{for every possible value of } \theta


The bias is Bias(θ^)=E(θ^)−θ\text{Bias}(\hat\theta) = E(\hat\theta) - \theta, so an unbiased estimator has bias zero. The phrase "for every possible value of θ\theta" carries weight: an estimator that happens to give the right answer on average at one particular θ\theta but not at others is not unbiased.

Two target diagrams, one for each estimator. Each dot is one of eight equally likely possible values in that estimator’s complete sampling distribution at the shown \theta; the offsets from \theta stay the same for every possible \theta. The crosshair marks the true parameter value. The left panel belongs to Estimator 1 and the right panel to Estimator 2.

Two target diagrams, one for each estimator. Each dot is one of eight equally likely possible values in that estimator’s complete sampling distribution at the shown θ\theta; the offsets from θ\theta stay the same for every possible θ\theta. The crosshair marks the true parameter value. The left panel belongs to Estimator 1 and the right panel to Estimator 2.

Bias is not accuracy, and accuracy is not precision

Bias describes where the sampling distribution is centred. Variance describes how widely it spreads. The two are independent: an estimator can be centred on the target and still scatter widely, or be tightly clustered and consistently off to one side.


Neither property alone tells you how good a single estimate will be, which is why the mean squared error combines them as MSE=Variance+Bias2\text{MSE} = \text{Variance} + \text{Bias}^2.

Linear combinations

Expectation is linear, so for any constants cic_i and observations with E(Xi)=μE(X_i) = \mu,


E ⁣(∑iciXi)=(∑ici)μE\!\left(\sum_i c_i X_i\right) = \left(\sum_i c_i\right)\mu


which equals μ\mu for every μ\mu exactly when the coefficients sum to one. That single condition settles a whole family of questions. For instance μ^=25X1+35X2\hat\mu = \tfrac{2}{5}X_1 + \tfrac{3}{5}X_2 has coefficients summing to 25+35=1\tfrac25 + \tfrac35 = 1, so it is unbiased for μ\mu even though it weights the two observations unequally. Weighting unequally does change the variance, which is a separate question.

Three standard results

  • The sample mean Xˉ\bar X is unbiased for μ\mu, since its coefficients are 1/n1/n each and sum to one.
  • The sample proportion p^=X/n\hat p = X/n is unbiased for pp.
  • For independent identically distributed observations with finite variance, the sample variance S2=1n−1∑(Xi−Xˉ)2S^2 = \dfrac{1}{n-1}\sum (X_i - \bar X)^2 is unbiased for σ2\sigma^2, whereas the version dividing by nn is biased: E ⁣[1n∑(Xi−Xˉ)2]=n−1nσ2E\!\left[\tfrac1n\sum (X_i - \bar X)^2\right] = \dfrac{n-1}{n}\sigma^2. The reason is that the deviations are taken about Xˉ\bar X rather than about μ\mu, and Xˉ\bar X is by construction the value that makes that sum of squares as small as possible, so the sum is systematically too small.

Worked example

Suppose the true variance is σ2=9\sigma^2 = 9 and a student divides by n=5n = 5 rather than n−1n - 1. The bias is


Bias=n−1nσ2−σ2=−σ2n=−95=−1.8\text{Bias} = \frac{n-1}{n}\sigma^2 - \sigma^2 = -\frac{\sigma^2}{n} = -\frac{9}{5} = -1.8


so on average this estimator falls short of σ2\sigma^2 by 1.8 units. Notice that the bias shrinks as nn grows: the estimator is biased at every sample size but becomes asymptotically unbiased.



The figure shows two target diagrams. For each estimator, the eight equally likely dots give its complete sampling distribution. Their offsets from the crosshair stay the same for every possible value of the true parameter θ\theta. The crosshair marks θ\theta. Which estimator is unbiased?

Two target diagrams, one for each estimator. Each dot is one of eight equally likely possible values in that estimator’s complete sampling distribution at the shown \theta; the offsets from \theta stay the same for every possible \theta. The crosshair marks the true parameter value. The left panel belongs to Estimator 1 and the right panel to Estimator 2.

Two target diagrams, one for each estimator. Each dot is one of eight equally likely possible values in that estimator’s complete sampling distribution at the shown θ\theta; the offsets from θ\theta stay the same for every possible θ\theta. The crosshair marks the true parameter value. The left panel belongs to Estimator 1 and the right panel to Estimator 2.

Approximate sampling distributions of two competing estimators of the same parameter

Approximate sampling distributions of two competing estimators of the same parameter

Choose an answer

Create a free account to get started

Unlock full course notes by signing up

  • Video lessons taught by top educators
  • Practice questions with guided solutions
  • Written review notes for reference
  • Expert answers anytime from course tutors

Let X1,X2,X3X_1, X_2, X_3 be independent random variables drawn from a population with mean μ\mu. Which of the following estimators of μ\mu is unbiased?

Choose an answer

Create a free account to get started

Unlock full course notes by signing up

  • Video lessons taught by top educators
  • Practice questions with guided solutions
  • Written review notes for reference
  • Expert answers anytime from course tutors

Let X1,X2,X3X_1, X_2, X_3 be independent observations with E(Xi)=μE(X_i) = \mu for each ii. Consider the estimator μ^=aX1+2aX2+3aX3\hat\mu = aX_1 + 2aX_2 + 3aX_3, where aa is a constant. Find the value of aa that makes μ^\hat\mu an unbiased estimator of μ\mu. Give your answer as a decimal to four places.

Create a free account to get started

Unlock full course notes by signing up

  • Video lessons taught by top educators
  • Practice questions with guided solutions
  • Written review notes for reference
  • Expert answers anytime from course tutors