

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer

When calculating a sample variance a factor of (N-1) appears instead of N (see ). Does anybody have an intuitive way of explaining this to students who need to use this fact but maybe haven't taken a statistics course?

share|cite|improve this question
This is essentially a duplicate of Sample Standard Deviation vs. Population Standard Deviation. – Mike Spivey Sep 1 '11 at 19:04
But the answer there just says that there is a bias and here's how you correct it. Is there a way to explain the correction intuitively (yes that's a little vague). – Boris Sep 1 '11 at 19:07
There is an attempt at an intuitive explanation in the second part of that answer. Does that help? – Mike Spivey Sep 1 '11 at 19:08
The concrete example I cite in my answer below can be understood without even using algebra. – Michael Hardy Sep 1 '11 at 20:02
The fact that correction for bias can sometimes be a very bad thing to do seems not widely understood among non-statisticians, including some who teach statistics. In this example of estimating variance, by the usual mean-squared error criterion, and assuming the sample is from a normally distributed population, the unbiased estimator is only slightly worse than the biased estimator, so it's not a good example to illustrate that point. But the idea that it is the unbiased estimator that is worse may fail to be as widely appreciated as it could be. – Michael Hardy Sep 1 '11 at 20:11

The Wikipedia article linked to above has a section (written by me) titled "The source of the bias". It explains it via a concrete example.

But note also that correcting for bias, when it can be done, is not always a good idea. I wrote this paper about that:

(My article begins near the bottom of the first page of this reprint.)

share|cite|improve this answer
+1 for the interesting paper. – Mike Spivey Sep 1 '11 at 21:10
Thank you. (....extra line because the software does not allow short comments....) – Michael Hardy Sep 1 '11 at 21:21
I use dollars $$$$ – The Chaz 2.0 Sep 1 '11 at 21:28
(to circumvent the minimum character requirement) And while I haven't seen anyone else use this, I'm not opposed to sharing it with such a generous (mathematical) contributor! – The Chaz 2.0 Sep 1 '11 at 21:29
@TheChaz: Unicode zero width spaces work much better, and don't have the formatting issues like in your comment above. – Ilmari Karonen Sep 16 '13 at 14:27

To me, the main idea is that the sample mean is not the distribution (or population) mean. The sample mean is "closer" to the sample data than the distribution mean, so the variance computed is smaller.

Suppose the distribution mean is $m_d$. The sum of $n$ variates (the sample) is $n m_s$, where $m_s$ is the sample mean. Recall that the mean and variance of a sum of variates are the sum of the means and sum of the variances of the variates. That is, the distribution mean of the sum of $n$ variates is $n m_d$ and the distribution variance of the sum of $n$ variates is $n v_d$. In other words, $$ \mathrm{E}[(n m_s-n m_d)^2]=n v_d $$ or equivalently, $$ \mathrm{E}[(m_s-m_d)^2]=\frac{1}{n}v_d $$ Let us compute the expected sample variance as $$ \begin{align} &\mathrm{E}[v_s]\\ &=\mathrm{E}\left[\frac{1}{n}\sum_{k=1}^n(x_k-m_s)^2\right]\\ &=\mathrm{E}\left[\frac{1}{n}\sum_{k=1}^n\left((x_k-m_d)^2+2(x_k-m_d)(m_d-m_s)+(m_d-m_s)^2\right)\right]\\ &=\mathrm{E}\left[\frac{1}{n}\sum_{k=1}^n\left((x_k-m_d)^2+2(m_s-m_d)(m_d-m_s)+(m_d-m_s)^2\right)\right]\\ &=\mathrm{E}\left[\frac{1}{n}\sum_{k=1}^n\left((x_k-m_d)^2-(m_d-m_s)^2\right)\right]\\ &=v_d-\frac{1}{n}v_d\\ &=\frac{n{-}1}{n}v_d \end{align} $$ Thus, $$ v_d=\frac{n}{n{-}1}\mathrm{E}[v_s] $$ This is why, to estimate the distribution variance, we multiply the sample variance by $\frac{n}{n{-}1}$. Thus, it appears as if we are dividing by $n{-}1$ instead of $n$.

share|cite|improve this answer
+1, though for an intuitive explanation, you could have stopped after the first paragraph. – Henry Sep 1 '11 at 22:51
@Henry: yes, I know. Since the derivation follows the idea of my intuitive explanation, I thought I would attach it for anyone who is interested. – robjohn Sep 2 '11 at 3:27
sorry to bump, but how did you justify replacing $x_k-m_d$ with $m_s-m_d$? I mean it makes sense their expectations are equal, but I was a little confused. – oldrinb Dec 27 '14 at 20:45
@oldrinb: I think that you are asking about the equation $$\mathrm{E}\left[\frac1n\sum_{k=1}^nx_k\right] =\mathrm{E}\left[\frac1n\sum_{k=1}^nm_s\right]$$ That follows because $$\frac1n\sum_{k=1}^nm_s =m_s =\frac1n\sum_{k=1}^nx_k$$ – robjohn Dec 28 '14 at 14:01
You use $m_s$ for what is commonly denoted by $\bar{X}$ (the sample mean), right? It's defined as a sum of $n$ random variables $X_i$ divided by $n$. – user4205580 Jan 9 at 22:25

The sum $\sum (x_i - a)^2$ is minimized when $a$ is the average of the $x_i$'s. The proof is a simple exercise in algebra (write sum as quadratic in $a$) or calculus (differentiate to get $\sum (x_i - a) = 0 $).

Therefore, $\sum (x_i - \overline{x})^2 \leq \sum (x_i - \mu)^2$. The same inequality holds with averages in place of sums.

The average of the squared $(x_i - \mu)$'s is an unbiased estimate of the variance, since each term has that variance as its expected value. Replacing $\mu$ by $\overline{x}$ produces a smaller estimate of the variance, and the expected value of that estimate is lower than the expected value of the unbiased estimator with $\mu$. The latter is the variance, so the expression with $1/N$ will (on average) underestimate the variance.

To summarize: the use of $\overline{x}$ as a surrogate for $\mu$ causes a downward bias in estimating variance by $\sum (x_i - \overline{x})^2 / N$.

share|cite|improve this answer

The question refers to "explaining this to students who need to use this fact but maybe haven't taken a statistics course". If they're more advanced than those who will understand the example that I mentioned that does not require algebra beyond expanding $(a+b)^2$, maybe a couple of other points of view are worth looking at.

We can write $$ \begin{bmatrix}x_1 \\ \vdots \\ x_n\end{bmatrix} = \begin{bmatrix}\overline{x} \\ \vdots \\ \overline{x} \end{bmatrix} + \begin{bmatrix} x_1 - \overline{x} \\ \vdots \\ x_n - \overline{x}\end{bmatrix}, $$ and notice that the two vectors being added are the orthogonal projections of the sum onto spaces of dimensions $1$ and $n-1$. The expected value of the first summand is $\mu$ times a column of $1$s, and the expected value of the second summand is $0$. So rotate the coordinate system so that this becomes $$ \begin{bmatrix}u_1 \\ \vdots \\ u_n\end{bmatrix} = \begin{bmatrix} u_1 \\ 0 \\ 0 \\ \vdots \\ 0 \end{bmatrix} + \begin{bmatrix} 0 \\ u_2 \\ u_3 \\ \vdots \\ u_n \end{bmatrix}. $$ The expected value of the first entry in the first summand is $\mu\sqrt{n}$. The expected value of every entry in the second summand is $0$. The expected value of the square of the norm of the second vector is $n-1$ times the expected value of the square of any of its entries. That's where the $n-1$ comes from. Notice that $$ \underbrace{\sum_{i=1}^n (x_i - \overline{x})^2}_{n\text{ terms}} = \underbrace{\sum_{i=2}^n u_i^2}_{n-1\text{ terms}}. $$

If students do know some probability theory, the above can also explain why $\sum_{i=1}^n (X_i - \overline{X})^2/\sigma^2$ has a chi-square distribution with $n-1$ degrees of freedom when there are certain assumptions about normal distribution and about independence. (I use capital $X$ this time since it's a random variable.) It can also explain why $\overline{X}$ is actually independent of that chi-square random variable.

Another thing that is sometimes useful in thinking about this topic is the algebraic identity $$ \sum_{i=1}^n (x_i - \mu)^2 = n(\overline{x} - \mu)^2 + \sum_{i=1}^n (x_i - \overline{x})^2 \text{ where } \overline{x} = \frac{x_1+\cdots+x_n}n. $$ Clearly this implies that $$ \sum_{i=1}^n (x_i - \mu)^2 \ge \sum_{i=1}^n (x_i - \overline{x})^2 $$ with equality if and only if $\overline{x}=\mu$. This is of course the same thing as what was used in the concrete example in the Wikipedia article linked in my earlier answer, but stated in a way that will be understood by students who know more algebra than the expansion of $(a+b)^2$.

share|cite|improve this answer
Very nice explanation, the LA approach gives a different perspective. – jeremy radcliff Jun 24 at 1:38
@jeremyradcliff : Thank you. $\qquad$ – Michael Hardy Jun 24 at 16:40

The main reason is that (unknown to many people!) Bessel's correction only gives an unbiased estimator for the variance of samples with replacement.

The result of this sampling with replacement is that you get "more of the same" numbers into the sample than are in the original population. Therefore your variance in the sample is lower on average compared to the original population.

This bias is corrected by dividing by a smaller number ($n-1$ instead of $n$) so that the variance gets bigger, i.e. unbiased again.

share|cite|improve this answer

I just finished a purely visual explanation of the intuition of bessels at creatorguides.

Basically it happens because the sample mean can differ from the population mean, so the proof uses the binomial theorem to show this by creating the population covariance squares out of the sample covariance squares like the section in the wikipedia article.

The wikipedia article explains why it's n-1 and not n once you understand the gist of the need for it.

share|cite|improve this answer

Here's a really simple example:

Say that you have a list of the heights of all the players in a sports league. Call that your "population" and calculate the variance. Now pick a sample of three players at random and calculate the variance of your sample. Chances are that your sample won't pick up the most extreme values, thereby causing the variance of the sample to be lower than the variance of the overall population.

In order to correct for that, you need to adjust your sample variance upward. That's why you use 1/(n-1) instead of 1/n. At small values of n (such as 3) the difference between these two will be pretty big. As your sample size increases, the difference gets smaller. This is because your sample is more likely to capture the extreme values in the population and therefore your sample variance will approach the population variance.

share|cite|improve this answer
This is a misleading explanation. If you pick a small sample, you expect to get a small spread, but you'll divide by the size of the sample in the end anyway, so that's not a reason for the resulting sample variance to be smaller. Indeed, your argument would still apply if you computed the variance with respect to the true population mean, but in this case the expected sample variance is equal to the population variance. – Rahul Nov 21 '14 at 16:46

Your Answer


