Library/CFA (Chartered Financial Analyst)/JuiceNotes™ 2024: Quantitative Methods - CFA® Level I/Simple Linear Regression

Simple Linear Regression

50 questions available

Take a quiz Listen to a podcast

Basics of Simple Linear Regression5 min

Regression analysis creates a mathematical model to explain a dependent variable using an independent variable. In simple linear regression, this relationship is a straight line: Y = a + b*X + e. The coefficient 'b' is the slope, representing the change in Y for a one-unit change in X. It is calculated as Cov(X,Y)/Var(X). The intercept 'a' is the value of Y when X is zero. The method minimizes the Sum of Squared Errors (SSE), known as the Ordinary Least Squares (OLS) technique.

Key Points

Dependent Variable (Y) vs. Independent Variable (X).
Slope formula: Covariance(X,Y) / Variance(X).
Intercept formula uses mean X and mean Y.
OLS minimizes squared residuals.

Assumptions of the Model5 min

Valid inference relies on four assumptions: Linearity (parameters are linear), Homoskedasticity (constant variance of error terms), Independence (residuals are uncorrelated), and Normality (residuals are normally distributed). Violations, such as heteroskedasticity or non-normality, can invalidate hypothesis tests and confidence intervals.

Key Points

Linearity in parameters.
Homoskedasticity vs. Heteroskedasticity.
Independence of observations.
Normality of residuals.

ANOVA and Measures of Fit6 min

The ANOVA table breaks down total variation (SST) into explained (SSR) and unexplained (SSE) components. The Coefficient of Determination (R-squared) is SSR/SST and measures the proportion of variance explained. For simple regression, R-squared is the square of the correlation coefficient. The Standard Error of Estimate (SEE) is the square root of the Mean Squared Error (MSE) and gauges the fit's precision.

Key Points

SST = SSR + SSE.
R-squared = SSR / SST = correlation^2.
SEE = Square Root of MSE.
F-statistic = MSR / MSE.

Hypothesis Testing and Prediction6 min

We test if the slope is statistically significant using a t-test with n-2 degrees of freedom. A significant slope implies a linear relationship. The F-test yields the same result for simple regression (F = t^2). The model is used for prediction, where the Standard Error of Forecast accounts for both the uncertainty in the estimated line and the specific error term, resulting in prediction intervals that are wider than confidence intervals.

Key Points

t-statistic = (estimated slope - hypothesized slope) / standard error of slope.
Prediction intervals account for SEE and estimation error.
F-test evaluates overall significance.
Standard error of forecast increases as X deviates from its mean.

Functional Forms and Dummy Variables4 min

When relationships are not linear, variables can be transformed using natural logs. A Log-Lin model has a dependent log variable; Lin-Log has an independent log variable. Log-Log models are used for constant elasticity. Dummy variables are binary (0 or 1) indicators used to incorporate qualitative independent variables into the regression.

Key Points

Log-Lin: ln(Y) = b0 + b1*X.
Lin-Log: Y = b0 + b1*ln(X).
Log-Log: ln(Y) = b0 + b1*ln(X) (Double Log).
Dummy variables represent categories like 'Pass/Fail'.

Questions

Question 1

In a simple linear regression equation Y = b0 + b1*X + epsilon, what does the term 'b1' represent?

Simple Linear Regression

Questions

Other chapters