Lecture 4 · Week 4 · Gujarati ch. 3 · about 50 min

Classical Linear Regression Model Assumptions

The seven assumptions of the classical linear regression model and what each one looks like on a graph, the variances and standard errors of the OLS estimators, the Gauss–Markov theorem (OLS is BLUE), and r² with its properties.

By the end you can

List the seven CLRM assumptions with their formulas and draw homoscedasticity, heteroscedasticity and autocorrelation
Compute the estimated error variance, Var(b2), Var(b1) and the standard errors by hand
State the Gauss–Markov theorem and explain linear, unbiased and best with sampling-distribution graphs
Compute r² and r and list the properties of r²

Key idea · Exam recipe (memorise this)

Seven assumptions: linear in parameters · X fixed (or independent of $u$ ) · $E(u_i \mid X_i) = 0$ · homoscedasticity $Var(u_i \mid X_i) = \sigma^2$ · no autocorrelation $cov(u_i, u_j) = 0$ · $n$ greater than the number of parameters · variability in $X$ .
Error variance: $\hat\sigma^2 = \dfrac{\sum e_i^2}{n-2} = \dfrac{RSS}{n-2}$ .
Variances: $Var(b_2) = \dfrac{\sigma^2}{\sum x_i^2}$ , $Var(b_1) = \dfrac{\sum X_i^2}{n\sum x_i^2}\sigma^2$ , $SE = \sqrt{Var}$ . Put $\hat\sigma^2$ in place of $\sigma^2$ .
Gauss–Markov: under the assumptions, OLS is BLUE: Linear, Unbiased ( $E(b_2) = \beta_2$ ), Best = minimum variance among linear unbiased estimators.
r² $= \dfrac{ESS}{TSS} = 1 - \dfrac{RSS}{TSS} = b_2^2\dfrac{\sum x_i^2}{\sum y_i^2}$ , $0 \le r^2 \le 1$ ; $r = \pm\sqrt{r^2}$ with the sign of $b_2$ .
Graphs: sideways bell curves along the PRF (same width = homoscedastic, widening = heteroscedastic); sampling distributions centred on $\beta_2$ (unbiased) and narrow (best).

How the exam asks it: three core exam templates

Exam questions on CLRM assumptions evaluate: (1) calculating standard errors and precision formulas by hand, (2) applying and proving the Gauss–Markov theorem (BLUE), and (3) computing $r^2$ and correlation $r$ directly from summary deviation sums.

Exam template 1: Hand-calculating error variance and standard errors

Question: Five students report study hours $X = (1, 2, 3, 4, 5)$ and exam scores $Y = (4, 6, 7, 8, 10)$ . The fitted line is $\hat{Y}_i = 2.8 + 1.4X_i$ , with residual sum of squares $RSS = 0.40$ , $\sum x_i^2 = 10$ , and $\sum X_i^2 = 55$ . a) Calculate the estimated error variance $\hat{\sigma}^2$ and standard error of regression $\hat{\sigma}$ .
b) Calculate the variance and standard error of the slope estimator, $Var(b_2)$ and $SE(b_2)$ .
c) Calculate the variance and standard error of the intercept estimator, $Var(b_1)$ and $SE(b_1)$ , and report the estimated regression line with standard errors in parentheses.

Exam template 2: Gauss–Markov theorem and proving unbiasedness

Question: Under the classical linear regression assumptions: a) State what each letter in BLUE stands for.
b) Write the OLS slope as a linear combination $b_2 = \sum k_i Y_i$ and state the properties of the weights $k_i$ .
c) Prove that $E(b_2) = \beta_2$ (unbiasedness), and state which specific assumption is required for this result.

Exam template 3: Computing r-squared and r from deviation sums

Question: An exam provides summary deviation sums from a consumption regression: $\sum x_i y_i = 16{,}800$ , $\sum x_i^2 = 33{,}000$ , and $\sum y_i^2 = 8{,}890$ . a) Compute the OLS slope coefficient $b_2$ .
b) Calculate the coefficient of determination $r^2$ using the deviation formula $r^2 = b_2^2 \dfrac{\sum x_i^2}{\sum y_i^2}$ .
c) Calculate the Pearson correlation coefficient $r$ and determine its sign.

Cheat-sheet & the 4-step method

· Cheat-sheet: CLRM assumptions, variances and BLUE

Concept / Parameter	Formula	When to use it	How to write the exam answer
Disturbance variance	$\hat{\sigma}^2 = \dfrac{RSS}{n-2}$	Measure residual variance	” $\hat{\sigma}^2 = RSS/(n-2)$ with $df = n-2$ .”
Standard error of regression	$\hat{\sigma} = \sqrt{\hat{\sigma}^2}$	Average size of residual	” $\hat{\sigma}$ is S.E. of regression in Y units.”
Slope variance	$Var(b_2) = \dfrac{\sigma^2}{\sum x_i^2}$	Precision of slope	” $SE(b_2) = \hat{\sigma} / \sqrt{\sum x_i^2}$ .”
Intercept variance	$Var(b_1) = \dfrac{\sum X_i^2}{n\sum x_i^2}\sigma^2$	Precision of intercept	” $SE(b_1) = \sqrt{Var(b_1)}$ .”
Covariance	$cov(b_1, b_2) = -\bar{X} Var(b_2)$	Estimator co-movement	“Negative covariance when sample mean $\bar{X} \gt 0$ .”
Gauss-Markov (BLUE)	Linear, Unbiased, Minimum Variance	Justify OLS optimality	“OLS is BLUE among linear unbiased estimators.”
r-squared from sums	$r^2 = b_2^2 \dfrac{\sum x_i^2}{\sum y_i^2}$	Goodness of fit	” $r = \operatorname{sign}(b_2)\sqrt{r^2}$ is Pearson’s $r$ .”

Key idea · The 4-step method for CLRM and precision problems

Step 1: Check the 7 CLRM assumptions.
- Confirm linear parameters, fixed $X$ , $E(u_i \mid X_i) = 0$ , constant variance $\sigma^2$ (homoscedasticity), no autocorrelation ( $cov(u_i, u_j) = 0$ ), $n \gt 2$ , and $\sum x_i^2 \gt 0$ .
Step 2: Calculate error variance $\hat{\sigma}^2$ first.
- Compute $RSS = \sum e_i^2$ . Divide by $n-2$ degrees of freedom to obtain $\hat{\sigma}^2$ , then take square root for $\hat{\sigma}$ .
Step 3: Compute variances and standard errors.
- For slope: $Var(b_2) = \hat{\sigma}^2 / \sum x_i^2 \implies SE(b_2) = \sqrt{Var(b_2)}$ .
- For intercept: $Var(b_1) = \dfrac{\sum X_i^2}{n \sum x_i^2}\hat{\sigma}^2 \implies SE(b_1) = \sqrt{Var(b_1)}$ .
Step 4: Report standard format and interpret BLUE properties.
- Write $\hat{Y}_i = \underset{(SE(b_1))}{b_1} + \underset{(SE(b_2))}{b_2} X_i$ . Note that homoscedasticity is required for BLUE (‘Best’), while only zero mean and fixed $X$ are required for unbiasedness.

Why assumptions?

OLS always gives numbers: $b_1$ and $b_2$ can be computed from any data. But we want more than numbers. We want to know how close $b_1$ and $b_2$ are to the true $\beta_1$ and $\beta_2$ . Since the PRF is $Y_i = \beta_1 + \beta_2 X_i + u_i$ , $Y_i$ depends on $u_i$ . So unless we say how the $u_i$ are generated, we cannot say anything about how good $b_1$ and $b_2$ are. The classical linear regression model (CLRM) is the PRF plus seven assumptions.

Notation as on the slides: $b_1, b_2$ are the OLS estimators of $\beta_1, \beta_2$ , and small letters are deviations from the mean: $x_i = X_i - \bar X$ , $y_i = Y_i - \bar Y$ .

The seven assumptions

#	Assumption	In symbols
1	Linear regression model: linear in the parameters	$Y_i = \beta_1 + \beta_2 X_i + u_i$
2	X values fixed in repeated samples (or independent of the error)	$cov(X_i, u_i) = 0$
3	Zero mean value of the disturbance	$E(u_i \mid X_i) = 0$
4	Homoscedasticity (equal variance of $u_i$ )	$Var(u_i \mid X_i) = E(u_i^2) = \sigma^2$
5	No autocorrelation between the disturbances	$cov(u_i, u_j) = 0$ for $i \neq j$
6	More observations than parameters	$n \gt 2$ here
7	Variability in the X values	$\sum x_i^2 \gt 0$

What each one means in words:

Linear in parameters. The $\beta$ ‘s enter to the first power. $X^2$ or $\ln X$ is fine; $\beta_2^2$ is not.
Fixed X. We choose the X values (income 80, 100, …, 260) and draw Y for them again and again. The analysis is conditional on these X values.
Zero mean. The factors left in $u$ do not push Y up or down on average at any X. Then $E(Y \mid X_i) = \beta_1 + \beta_2 X_i$ : the PRF is the line of conditional means.
Homoscedasticity. “Equal spread”: the variance of $u$ is the same number $\sigma^2$ at every X. The violation is heteroscedasticity: $Var(u_i \mid X_i) = \sigma_i^2$ , changing with $i$ .
No autocorrelation. Knowing one disturbance tells you nothing about another. Typical violation: in time series, a shock this quarter carries over to the next.
n greater than the number of parameters. Two parameters need at least three observations; with two points the line goes through both and $n - 2 = 0$ leaves nothing to estimate $\sigma^2$ .
Variability in X. If every $X_i$ were the same, $\sum x_i^2 = 0$ and $b_2 = \sum x_i y_i / \sum x_i^2$ cannot be computed. You cannot learn how Y responds to X if X never changes.

Precision: variances and standard errors

$b_1$ and $b_2$ change from sample to sample. Under the assumptions, their variances are:

Variance and standard error of the slopeKey formulaFormula sheet →

Var(b_2) = \frac{\sigma^2}{\sum x_i^2} \qquad SE(b_2) = \frac{\sigma}{\sqrt{\sum x_i^2}}

$\sigma^2$: variance of the disturbance u
$x_i$: X_i - \bar X
$\sum x_i^2$: total variation of X around its mean

Use it when you need the precision of the slope estimate, or the SE for a t test or a confidence interval.

Variance and standard error of the interceptKey formulaFormula sheet →

Var(b_1) = \frac{\sum X_i^2}{n\sum x_i^2}\,\sigma^2 \qquad SE(b_1) = \sqrt{\frac{\sum X_i^2}{n\sum x_i^2}}\,\sigma

$\sum X_i^2$: sum of the squared raw X values (capital X)
$n$: sample size

Use it when you need the precision of the intercept. Careful: the numerator has capital X, the denominator small x.

$\sigma^2$ itself is unknown, so we estimate it from the residuals:

Estimated error varianceKey formulaFormula sheet →

\hat\sigma^2 = \frac{\sum e_i^2}{n-2} = \frac{RSS}{n-2} \qquad \hat\sigma = \sqrt{\hat\sigma^2}

$RSS$: \sum e_i^2 = residual sum of squares
$n - 2$: degrees of freedom (two parameters estimated)
$\hat\sigma$: standard error of the regression

Use it when you need a number for sigma-squared before computing any variance or standard error. EViews prints sigma-hat as 'S.E. of regression'.

Covariance of the two estimatorsFormula sheet →

cov(b_1, b_2) = -\bar X\, Var(b_2)

$\bar X$: sample mean of X

Use it when you are asked how b1 and b2 move together: with a positive mean of X they are negatively related (overestimate the slope, underestimate the intercept).

The Gauss–Markov theorem: OLS is BLUE

For the slope $b_2$ :

L, linear: $b_2$ is a linear function of the $Y_i$ : $b_2 = \sum k_i Y_i$ with weights $k_i = x_i / \sum x_i^2$ .
U, unbiased: $E(b_2) = \beta_2$ . Over repeated samples $b_2$ is right on average.
B, best: among all linear unbiased estimators, $b_2$ has the smallest variance. An estimator with this property is called efficient.

r²: goodness of fit and its properties

From Lesson 3: $TSS = ESS + RSS$ , that is $\sum y_i^2 = \sum \hat y_i^2 + \sum e_i^2$ .

Coefficient of determinationKey formulaFormula sheet →

r^2 = \frac{ESS}{TSS} = 1 - \frac{RSS}{TSS} = b_2^2\,\frac{\sum x_i^2}{\sum y_i^2} = \frac{\left(\sum x_i y_i\right)^2}{\sum x_i^2 \sum y_i^2}

$r^2$: share of the total variation in Y explained by the regression
$y_i$: Y_i - \bar Y

Use it when you need the fit of a two-variable regression; the last two forms work straight from the deviation sums without residuals.

Correlation coefficientFormula sheet →

r = \pm\sqrt{r^2} = \frac{\sum x_i y_i}{\sqrt{\sum x_i^2 \sum y_i^2}}

$r$: sample correlation between X and Y, with the same sign as b_2

Use it when you are asked for the correlation between X and Y after a two-variable regression.

Exam practice

Summary and review

CLRM = PRF $Y_i = \beta_1 + \beta_2 X_i + u_i$ plus seven assumptions: linear in parameters, fixed X, $E(u_i \mid X_i) = 0$ , $Var(u_i \mid X_i) = \sigma^2$ , $cov(u_i, u_j) = 0$ , $n$ greater than the number of parameters, variability in X.
Graphs: bell curves on the PRF (centred = zero mean; equal width = homoscedastic; widening = heteroscedastic); runs in $u_t$ over time = autocorrelation.
$\hat\sigma^2 = RSS/(n-2)$ ; $Var(b_2) = \sigma^2/\sum x_i^2$ ; $Var(b_1) = \sigma^2 \sum X_i^2/(n\sum x_i^2)$ ; $cov(b_1, b_2) = -\bar X Var(b_2)$ .
More spread in X and a smaller $\sigma^2$ make the slope more precise; a larger $n$ also helps the intercept.
Gauss–Markov: OLS is BLUE. $b_2 = \sum k_i Y_i$ (linear), $E(b_2) = \beta_2$ (unbiased), minimum variance among linear unbiased estimators (best, efficient).
$r^2 = ESS/TSS = b_2^2 \sum x_i^2/\sum y_i^2$ , $0 \le r^2 \le 1$ ; $r = \pm\sqrt{r^2}$ with the sign of $b_2$ .
Consumption example: $\hat\sigma = 6.493$ , $SE(b_1) = 6.414$ , $SE(b_2) = 0.0357$ , $r^2 = 0.962$ , $r = 0.981$ .