Simulation (Probability)
Statistics & ProbabilityA simulation uses random processes to model and estimate the probability of events that are complex to calculate analytically.
Definition
A simulation is when you use a random process (like flipping coins, rolling dice, or using a computer) to model a real situation and estimate probabilities.
Example
To estimate the probability of getting at least $3$ heads in $5$ flips, you flip a coin $5$ times, record the result, and repeat $100$ times. The fraction of trials with $3$ or more heads estimates the probability.
Key Insight
Simulations are especially useful when the math is too complicated. You can always get an approximate answer by running the experiment many times.
Definition
A probability simulation uses a random process to approximate the probability of events. Random number generators, dice, spinners, or random number tables can serve as simulation tools. By running many trials, the experimental probability from the simulation estimates the true theoretical probability.
Example
Simulating birthday problem: $23$ people are in a room. What is the probability at least $2$ share a birthday? Rather than computing analytically, assign random birthdays to $23$ people and check for matches. Repeating $10{,}000$ times gives an experimental probability close to the theoretical $0.507$.
Key Insight
Simulations scale to complex problems that would be analytically intractable. The more trials you run, the closer the simulation result is to the true probability (law of large numbers).
Definition
Monte Carlo simulation is a computational method that uses random sampling to estimate mathematical quantities, including probabilities, integrals, and optimization solutions. The key idea: if $X$ is a random variable with $E[f(X)] = \mu$, then $(1/n)\sum f(X_i) \to \mu$ by the LLN, providing an estimator with standard error $\sigma_f/\sqrt{n}$.
Example
Estimating $\pi$ via Monte Carlo: generate $n$ random points $(x,y)$ uniformly in $[0,1]^2$. Count hits where $x^2+y^2 \le 1$ (inside the quarter circle). $\pi/4$ is estimated by the fraction of hits. For $n=10{,}000$, the estimate has standard error approximately $0.01$, so $\pi$ is estimated to about $2$ decimal places.
Key Insight
Markov chain Monte Carlo (MCMC) methods (Metropolis-Hastings, Gibbs sampling) extend simulation to high-dimensional distributions where direct sampling is impossible. MCMC constructs a Markov chain whose stationary distribution is the target distribution, enabling Bayesian posterior sampling and integration.