Line of Best Fit
The line of best fit is the straight line that most closely follows the pattern of data points on a scatter plot, minimizing overall error.
Formula
\hat{y} = b_0 + b_1 x
Definition
The line of best fit is the straight line drawn through a scatter plot that comes as close as possible to all the data points, summarizing the overall trend of the data. Formally, it is the least-squares regression line $\hat{y} = b_0 + b_1 x$ that minimizes the sum of squared residuals $\sum (y_i - \hat{y}_i)^2$, with slope $b_1 = r(s_y/s_x)$ and intercept $b_0 = \bar{y} - b_1\bar{x}$; the line always passes through $(\bar{x}, \bar{y})$. It is derived by solving the normal equations $(X^TX)\beta = X^TY$; the fitted values lie on the line and the residuals are orthogonal to all predictor vectors, a geometric property of projection onto the column space of $X$.
Example
On a scatter plot of hours studied vs. test scores, the line of best fit goes through the middle of the point cloud, showing the general upward trend, though it does not have to pass through any single data point. For data with $\bar{x}=5$, $\bar{y}=80$, $s_x=2$, $s_y=10$, $r=0.8$: $b_1 = 0.8(10/2) = 4$, $b_0 = 80 - 4(5) = 60$, giving line $\hat{y} = 60 + 4x$ and a prediction of $\hat{y} = 88$ for $x=7$. Geometrically, OLS projects the response vector $Y$ onto the column space of $X$: the fitted values $\hat{Y} = HY$ where $H = X(X^TX)^{-1}X^T$ is the hat (projection) matrix, and the residuals $e = Y - \hat{Y} = (I-H)Y$ lie in the orthogonal complement of the column space of $X$.
Key Insight
The line of best fit is positioned to minimize the overall distance between itself and all the points, and it is only valid for interpolation (predicting within the range of x-values used to build it); extrapolating far beyond the data range is unreliable because the linear relationship may not hold. The hat matrix $H$ has diagonal entries $h_{ii}$ (leverages) that measure how much influence each observation has on its own fitted value; high-leverage points with large residuals are the most influential and are diagnosed with Cook's distance.