Bias (Statistics)
Bias in statistics is a systematic error that causes results to consistently deviate from the true population value.
Definition
Bias means a survey or study is unfair because it consistently favors certain results; a biased sample does not truly represent the whole population. It is a systematic error in sampling or measurement that causes results to consistently differ from the true value, with common sources including sampling bias (non-random selection), response bias (dishonest answers), and question-wording bias. Formally, the bias of an estimator $T$ for parameter $\theta$ is defined as $\text{Bias}(T) = E[T] - \theta$; an unbiased estimator has $\text{Bias} = 0$, and the mean squared error decomposes as $\text{MSE} = \text{Variance} + \text{Bias}^2$, revealing the bias-variance tradeoff: reducing bias may increase variance and vice versa.
Example
Asking only students in the math club whether they like math will give a biased result because math club members probably like math more than average students. A phone survey conducted only during daytime hours will underrepresent working adults who are unavailable, producing a sample biased toward retired or unemployed respondents. The maximum likelihood estimator of population variance $\sigma^2$ uses $n$ in the denominator, yielding a biased estimator that underestimates by a factor of $(n-1)/n$; replacing $n$ with $n-1$ gives the unbiased sample variance $s^2$.
Key Insight
Bias is like a scale that always reads $2$ pounds too heavy: no matter how many times you weigh something, the answer is always wrong in the same direction, and bias cannot be fixed by increasing sample size (a biased sample of 1 million people is still biased). Randomization is the primary defense. In regularized regression (e.g., ridge regression), bias is deliberately introduced to reduce variance and improve prediction on new data, exemplifying the bias-variance tradeoff in practice.