Variance
Statistics & ProbabilityVariance measures the average squared distance of data values from the mean, indicating how spread out the data is.
Formula
s^2 = \dfrac{\sum (x_i - \bar{x})^2}{n-1}
Definition
Variance is a number that measures how much the values in a dataset vary from the average. A bigger variance means the values are more spread out.
Example
Scores of $80$, $80$, $80$, $80$ have a variance of $0$ (no spread). Scores of $60$, $70$, $90$, $100$ have a large variance because they are far from the average of $80$.
Key Insight
Variance and standard deviation both measure spread, but standard deviation is in the original units (like inches or points), making it easier to interpret. Variance is in squared units.
Definition
The sample variance $s^2 = \sum (x_i-\bar{x})^2/(n-1)$ is the average squared deviation from the sample mean. It is always non-negative and equals zero only when all values are identical. Taking the square root gives the standard deviation, which is in the original units.
Example
Data: $2$, $4$, $4$, $4$, $5$, $5$, $7$, $9$. Mean $= 5$. Squared deviations: $9$, $1$, $1$, $1$, $0$, $0$, $4$, $16$. Sum $= 32$. Variance $s^2 = 32/7 = 4.57$. Standard deviation $s = \sqrt{4.57} = 2.14$.
Key Insight
Squaring the deviations serves two purposes: it eliminates negative signs (so deviations do not cancel) and it penalizes large deviations more heavily than small ones.
Definition
The population variance $\sigma^2 = E[(X-\mu)^2] = E[X^2] - \mu^2$ is a fundamental property of a distribution. The sample variance $s^2 = \frac{1}{n-1}\sum(x_i-\bar{x})^2$ is unbiased: $E[s^2] = \sigma^2$. In ANOVA, total variance is decomposed into between-group and within-group components, enabling F-tests.
Example
The law of total variance: $\text{Var}(Y) = E[\text{Var}(Y|X)] + \text{Var}(E[Y|X])$ decomposes variance into an average within-group component and a between-group component. This identity underlies random effects models and the analysis of variance framework.
Key Insight
The Cramer-Rao lower bound states that no unbiased estimator of $\theta$ can have variance less than $1/I(\theta)$, where $I(\theta)$ is the Fisher information. For normal data, $s^2$ achieves this bound, making it the minimum variance unbiased estimator (MVUE) of $\sigma^2$.