Unbiased estimation of standard deviation

From Wikipedia, the free encyclopedia

The standard deviation is often estimated from a random sample drawn from the population. The most common measure used is the sample standard deviation, which is defined by

$s = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (x_i - \overline{x})^2}\,,$

where $\{x_1,x_2,\ldots,x_n\}$ is the sample (formally, realizations from a random variable X) and $\overline{x}$ is the sample mean.

The reason for this definition is that s² is an unbiased estimator for the variance σ² of the underlying population, if that variance exists and the sample values are drawn independently with replacement. However, s estimates the population standard deviation σ with negative bias; that is, s tends to underestimate σ.

An intuitive explanation why the square root of the sample variance is a biased estimator of the standard deviation is that the standard deviation is a nonlinear function of the variance, so the property of being unbiased does not carry over. This is because the operations of taking the mean and applying a function in general do not commute unless that function is linear, and, by definition, an estimate is unbiased when its mean equals to the quantity being estimated.

[edit] Bias correction

When the random variable is normally distributed, a minor correction exists to eliminate the bias. To derive the correction, note that for normally distributed X, Cochran's theorem implies that $\sqrt{n-1}s/\sigma$ has a chi distribution with $n - 1$ degrees of freedom. Consequently,

$\operatorname{E}[s] = c_4\sigma$

where $c 4$ is a constant that depends on the sample size n as follows:

$c_4=\sqrt{\frac{2}{n-1}}\frac{\Gamma\left(\frac{n}{2}\right)}{\Gamma\left(\frac{n-1}{2}\right)}$

and $\Gamma(\cdot)$ is the gamma function.

Thus an unbiased estimator of σ is had by dividing s by $c 4$ . Tables giving the value of $c 4$ for selected values of n may be found in most textbooks on statistical quality control. As n grows large it approaches 1, and even for smaller values the correction is minor. For example, for $n = 10$ the value of $c 4$ is about 0.9727. It is important to keep in mind this correction only produces an unbiased estimator for normally distributed X. When this condition is satisfied, another result about s involving $c 4$ is that the standard deviation of s is $\sigma\sqrt{1-c_4^2}$ .