Parameter (Statistics)
Statistics & ProbabilityA parameter is a numerical value that describes a characteristic of an entire population.
Definition
A parameter is a number that describes something about a whole population. Because we usually cannot measure everyone, parameters are often unknown and must be estimated.
Example
The average height of every adult in the United States is a parameter. We do not know the exact value because we cannot measure everyone.
Key Insight
Parameters are the true values we wish we knew. Statistics (from samples) are our best guesses at those true values.
Definition
A parameter is a numerical characteristic of a population, such as the population mean ($\mu$) or population standard deviation ($\sigma$). Parameters are fixed but typically unknown. Greek letters are used for parameters; Roman letters for sample statistics.
Example
If we define the population as all high school seniors in a state, then the true average SAT score of all of them is a parameter ($\mu$). The average SAT score of a random sample of $200$ seniors is the corresponding statistic ($\bar{x}$).
Key Insight
The goal of statistical inference is to use sample statistics to make well-reasoned claims about unknown population parameters.
Definition
A parameter $\theta$ is an element of the parameter space that indexes the family of distributions $\{F(x; \theta)\}$. Point estimation, interval estimation, and hypothesis testing are the three main inferential tasks related to parameters. In Bayesian inference, $\theta$ is itself treated as a random variable with a prior distribution.
Example
For a Bernoulli population, $\theta = p$ (the probability of success). MLE yields $\hat{p} = \bar{x}$ (the sample proportion), which is unbiased and achieves the Cramer-Rao lower bound.
Key Insight
The Cramer-Rao inequality gives a theoretical lower bound on the variance of any unbiased estimator of $\theta$, providing a benchmark for efficiency.