Library/CFA (Chartered Financial Analyst)/Quantitative Methods — CFA Program Curriculum, 2026 • LEVEL I • VOLUME 1/Learning Module 10 Simple Linear Regression

Learning Module 10 Simple Linear Regression

50 questions available

Take a quiz Listen to a podcast

Estimation and Core Formulas5 min

Simple linear regression (SLR) models the relation between a dependent variable Y and a single independent variable X as Y = b0 + b1 X + epsilon, where b0 is the intercept, b1 the slope, and epsilon the error term. Ordinary least squares (OLS) fits the line by selecting estimates b_hat0 and b_hat1 that minimize the sum of squared residuals SSE = sum (Yi - Yhat_i)^2. The slope estimator equals the covariance of X and Y divided by the variance of X: b_hat1 = sum (Yi - Ybar)(Xi - Xbar) / sum (Xi - Xbar)^2; the intercept is b_hat0 = Ybar - b_hat1 Xbar. Residuals e_i = Yi - Yhat_i sum to zero by construction. The sample pairwise correlation r between X and Y relates to slope; in SLR, R^2 = r^2 and equals SSR / SST, where SST = sum (Yi - Ybar)^2, SSR = sum (Yhat_i - Ybar)^2, and SSE = sum (Yi - Yhat_i)^2 so that SST = SSR + SSE. Goodness-of-fit measures: coefficient of determination R^2; F-statistic = MSR / MSE with MSR = SSR / k and MSE = SSE / (n - k - 1); and standard error of the estimate se = sqrt(MSE). Hypothesis testing on slope and intercept uses t-statistics: t = (b_hat1 - B1) / SE(b_hat1) and t_intercept = (b_hat0 - B0) / SE(b_hat0), with degrees of freedom n - 2 in SLR. SE(b_hat1) = se / sqrt( sum (Xi - Xbar)^2 ). Testing whether correlation r equals zero uses the same t-statistic as test of slope = 0, t = r sqrt((n - 2)/(1 - r^2)). SLR assumes: (1) linearity (true relation linear in parameters and residuals random), (2) homoskedasticity (constant variance E(epsilon_i^2) = sigma_e^2 across i), (3) independence (errors uncorrelated across observations), and (4) normality of residuals (for finite-sample inference). Residual plots (residual vs X, residual vs time) and histograms/normal probability plots are diagnostic tools to detect nonlinearity, heteroskedasticity (pattern or fan shape), autocorrelation (patterns over time or seasonality), and non-normality or outliers. Heteroskedasticity and autocorrelation invalidate standard error estimates and test statistics; remedies include variable transformation, weighted least squares, adding regime indicators, or time-series models. An analysis of variance (ANOVA) table for regression displays SSR, SSE, SST with degrees of freedom and mean squares; the F-test in ANOVA tests whether the model explains more variance than unexplained variance (i.e., slope differs from zero). Prediction: predicted Y at Xf is Yhat_f = b_hat0 + b_hat1 Xf. The standard error of prediction sf = se sqrt(1 + 1/n + (Xf - Xbar)^2 / sum (Xi - Xbar)^2). A (1 - alpha) prediction interval is Yhat_f ± t_{alpha/2, n-2} sf. Prediction uncertainty increases with se, decreases with n, and increases as Xf moves away from Xbar. Indicator (dummy) independent variables take values 0/1; in SLR the intercept equals mean Y when dummy = 0 and the slope equals difference in means between groups. Functional forms: when the relationship is nonlinear, transform variables: log-lin (ln Y = b0 + b1 X) yields percentage change in Y for unit change in X; lin-log (Y = b0 + b1 ln X) yields absolute change in Y for percentage change in X; log-log (ln Y = b0 + b1 ln X) means b1 is elasticity: percent change in Y for percent change in X. Use fit statistics (R^2, se, F) and residual patterns to choose transformations. Outliers and data errors can dramatically alter slope, R^2, and se; always inspect data and correct erroneous observations. For inferences, choose significance level alpha, calculate t or F statistics, and/or compute p-values; small p-values indicate coefficients significantly different from hypothesized values. As sample size increases, the critical values shrink and smaller correlations or slopes can become significant; conversely, large samples can make trivial effects statistically significant, so consider economic significance as well as statistical significance.

Key Points

OLS estimates b_hat1 and b_hat0 minimize sum of squared residuals.
SST = SSR + SSE; R^2 = SSR / SST = r^2 in SLR.
t-tests for slope and intercept use n - 2 df; F-test compares MSR to MSE.
Prediction interval uses se and increases with distance of Xf from Xbar.
Transformations (log/lin) change interpretation: log-log gives elasticity.

Assumptions and Diagnostics5 min

SLR assumptions are linearity, homoskedasticity, independence, and normality of residuals. Linearity requires that the conditional mean of Y is linear in X (or in transformed X). Nonlinearity shows as systematic patterns in residual-vs-X plots. Homoskedasticity (constant variance) is violated if residual spread varies with X or over regimes; detect by plotting residuals or using tests (e.g., Breusch-Pagan). Independence is violated when residuals are autocorrelated (time series with seasonality), detectable through residual vs time plots and Durbin-Watson tests. Non-normal residuals are a concern for small samples; use histograms, normal probability plots, or tests (e.g., Shapiro-Wilk). Outliers and leverage points can unduly influence estimates; examine data and correct errors or consider robust methods. When assumptions do not hold, remedies include transformations, adding dummy variables for regimes, weighted least squares, generalized least squares, or time-series approaches (ARIMA, HAC standard errors) for autocorrelation. Interpret coefficients carefully: intercept may be meaningless if X=0 is outside feasible range; slope is marginal change in Y per unit change in X; indicator variable slopes represent group mean differences. Consider both statistical and practical (economic) significance when evaluating results.

Key Points

Examine residual plots to detect violations.
Heteroskedasticity and autocorrelation affect standard errors and tests.
Remedies: transform variables, weighted/GMM/GLS, add dummies, correct data errors.
Outliers can distort slope and R^2; always inspect raw data and corrected values.
Normality matters more in small samples; large samples may rely on CLT for inference.

Hypothesis Tests, ANOVA, and Prediction5 min

Use t-tests to evaluate hypotheses on individual coefficients: two-sided or one-sided. The standard error of the slope uses se and the spread of X: SE(b_hat1) = se / sqrt(sum (Xi - Xbar)^2). The F-statistic in the ANOVA table tests the null that all slope coefficients are zero; in SLR F = t^2 for the slope test. ANOVA displays SSR, SSE, SST with df and mean squares; se = sqrt(MSE). For prediction, compute Yhat = b_hat0 + b_hat1 Xf and the standard error sf = se sqrt(1 + 1/n + (Xf - Xbar)^2 / sum (Xi - Xbar)^2). The prediction interval at level (1 - alpha) is Yhat ± t_{alpha/2, n-2} sf. Forecast uncertainty declines with larger n, smaller se, and Xf near Xbar. p-values are the smallest alpha at which the null is rejected; small p-values indicate stronger evidence against H0. When testing slope = 1 or other specific values, use t = (b_hat1 - hypoth_value)/SE(b_hat1). Using indicator variables, compare differences in group means via slope and pooled variance if needed. Beware of multiple testing and the effect of sample size on statistical significance; interpret effect sizes too.

Key Points

t-statistic for slope uses SE that depends on se and X variance.
F-statistic for model fit compares explained to unexplained variance; in SLR F = t^2.
Prediction intervals incorporate model uncertainty and distance of Xf from Xbar.
p-values quantify smallest alpha at which H0 would be rejected.
Testing nonzero constants (e.g., slope = 1) uses the same t formula with hypothesized value.

Functional Forms and Practical Considerations5 min

When the true relation is nonlinear, transform variables: log-lin (ln Y = b0 + b1 X) means b1 approximates proportional change in Y per unit X; lin-log (Y = b0 + b1 ln X) means b1 is absolute change in Y for a percent change in X (approx b1 * d ln X); log-log (ln Y = b0 + b1 ln X) makes b1 an elasticity (percent change in Y per percent change in X). Compare fits using R^2, se, F-statistic, and residual randomness. Do not compare R^2 across models with different dependent variable transformations without care; compare se or transformed fit metrics. Use domain knowledge when selecting functional form. Use indicator variables to capture regime shifts, announcement effects, or group differences; the slope on dummy is difference in means. Verify data quality—errors or outliers can produce misleadingly high R^2 and distorted slope estimates; correct obvious errors and consider robust regression if outliers are genuine. Finally, large samples can make trivial effects statistically significant; assess economic importance in addition to statistical tests.

Key Points

Select functional form based on residual patterns and fit metrics.
Log transformations change coefficient interpretation (elasticities and percent changes).
Indicator variables let SLR compare group means and capture regime shifts.
Always check and clean data; outliers can materially change results.
Balance statistical significance with economic/practical significance.

Questions

Question 1

In a simple linear regression of Y on X using OLS, which expression gives the estimated slope coefficient b_hat1?

Learning Module 10 Simple Linear Regression

Questions

Other chapters