Parameter (Statistics)
A parameter is a numerical value that describes a characteristic of an entire population.
Definition
A parameter is a number that describes something about a whole population, such as the population mean ($\mu$) or population standard deviation ($\sigma$). Because we usually cannot measure everyone, parameters are often unknown and must be estimated; Greek letters are used for parameters while Roman letters denote sample statistics. Formally, a parameter $\theta$ is an element of the parameter space that indexes the family of distributions $\{F(x; \theta)\}$, and point estimation, interval estimation, and hypothesis testing are the three main inferential tasks related to it. In Bayesian inference, $\theta$ is itself treated as a random variable with a prior distribution.
Example
The average height of every adult in the United States is a parameter we do not know exactly because we cannot measure everyone. If we define the population as all high school seniors in a state, the true average SAT score of all of them is a parameter ($\mu$), while the average SAT score of a random sample of $200$ seniors is the corresponding statistic ($\bar{x}$). For a Bernoulli population, $\theta = p$ (the probability of success), and MLE yields $\hat{p} = \bar{x}$ (the sample proportion), which is unbiased and achieves the Cramer-Rao lower bound.
Key Insight
Parameters are the true values we wish we knew; statistics computed from samples are our best guesses at those true values, and the goal of statistical inference is to use them to make well-reasoned claims about unknown population parameters. The Cramer-Rao inequality gives a theoretical lower bound on the variance of any unbiased estimator of $\theta$, providing a benchmark for efficiency.