Lecture 2 · Week 2 · Gujarati ch. 2 · about 45 min

Two-Variable Regression: Basic Ideas

What 'regressing Y on X' actually means: the population regression function as a locus of conditional means, why a disturbance term turns it into a stochastic model, the sample regression function that estimates it, and what 'linear' regression really requires.

By the end you can

Explain the population regression function as the conditional mean of Y given X
Write the stochastic and sample regression functions and explain the difference between a disturbance and a residual
Explain why 'linear' regression means linear in the parameters, not necessarily in the variables
Compute a conditional mean from grouped data, the empirical seed of the PRF

Regression as a conditional mean

“Regressing $Y$ on $X$ ” means studying how the average value of $Y$ changes as $X$ takes different fixed values — not predicting any one individual $Y$ perfectly, but describing the pattern in $Y$ ‘s behaviour across values of $X$ .

The population regression function (PRF)

Population regression functionKey definitionFormula sheet →

E(Y \mid X_i) = \beta_0 + \beta_1 X_i

$E(Y \mid X_i)$: the conditional mean of Y for a given value X_i
$\beta_0, \beta_1$: the true, unknown population parameters

Use it when you need to state the theoretical object regression is trying to uncover — the true relationship in the whole population, not just in one sample.

The PRF is the line traced out by plotting $E(Y|X)$ against $X$ for every possible value of $X$ — connecting the conditional means, like the 65 and 77.5 just computed, across the whole range of income. It is a property of the entire population, and in practice it is never directly observed: only a finite sample is available.

From PRF to a stochastic model

No single family’s actual consumption sits exactly on the conditional mean line — some spend more than their group’s average, some less. This individual deviation is captured by a disturbance term.

Stochastic PRFKey formulaFormula sheet →

Y_i = E(Y\mid X_i) + u_i = \beta_0 + \beta_1 X_i + u_i

$u_i$: the disturbance (error) term for observation i, capturing everything that pulls Y_i away from its conditional mean

Use it when you write down the full population model, ready to be estimated from a sample — this equation is the starting point of every two-variable regression in this course.

The sample regression function (SRF)

Since the PRF is unobservable, econometricians estimate it from a sample, producing the sample regression function, with residuals standing in for the unobservable disturbances.

Sample regression functionKey formulaFormula sheet →

\hat Y_i = \hat\beta_0 + \hat\beta_1 X_i \qquad Y_i = \hat Y_i + e_i

$\hat\beta_0, \hat\beta_1$: estimated parameters (also written b_0, b_1)
$\hat Y_i$: the fitted (predicted) value
$e_i$: the residual, the SAMPLE counterpart of u_i

Use it when you are working with actual estimated numbers from a specific dataset, as opposed to the unobservable, purely theoretical PRF.

What “linear” regression really means

Linearity in the parametersKey ideaFormula sheet →

\text{Linear regression} \iff \text{linear in the } \beta\text{'s, not necessarily in the } X\text{'s}

Use it when a model has a squared or otherwise nonlinear term in X, and you need to decide whether ordinary least squares can still be used.

$Y_i = \beta_0 + \beta_1 X_i^2 + u_i$ is a linear regression model, because it is linear in $\beta_0$ and $\beta_1$ — $X^2$ can simply be treated as a single, ordinary explanatory variable. But $Y_i = \beta_0 + \beta_1^2 X_i + u_i$ is not linear regression, because the parameter $\beta_1$ itself appears squared.

Simple versus multiple regression

A two-variable (simple) regression has one dependent variable and one explanatory variable, exactly the form studied so far. A multiple regression has one dependent variable and two or more explanatory variables — the subject of Lessons 7 to 9 later in this course.

Exam practice

Summary and review

Regression studies how the conditional mean of Y changes as X varies.
PRF: $E(Y|X_i) = \beta_0+\beta_1X_i$ , the true, unobservable population relationship.
Stochastic PRF: $Y_i = \beta_0+\beta_1X_i+u_i$ .
SRF: $\hat Y_i = \hat\beta_0+\hat\beta_1X_i$ , with residual $e_i = Y_i - \hat Y_i$ , estimated from an actual sample.
Disturbance $u_i$ (population, unobservable) versus residual $e_i$ (sample, observable) is a distinction that recurs throughout the course.
“Linear” regression means linear in the parameters, not necessarily in the variables.
Simple (two-variable) regression has one explanatory variable; multiple regression has two or more.