Two-Variable Regression: Basic Ideas

How the exam asks it: three core exam templates

Midterm exams on two-variable regression test three canonical problems: (1) calculating conditional means and true disturbances from the PRF, (2) computing fitted values and sample residuals from the SRF, and (3) testing linearity in parameters.

Exam template 1: Conditional expectation and the disturbance term

Question: A sample survey observes 5 families at an income level of $X = 120$ thousand TL with consumption expenditures $Y = 80, 84, 86, 90, 100$ (thousand TL). The true population regression function is $E(Y \mid X) = 28 + 0.5X$. One specific family in this group consumes $Y_i = 95$ thousand TL. a) Calculate the conditional expectation $E(Y \mid X = 120)$ directly from the data.
b) Confirm that the population regression equation gives the same conditional mean.
c) Compute this family’s disturbance $u_i$ and interpret its economic meaning.

Exam template 2: Fitted values and the sample residual

Question: The true PRF from Template 1 is estimated from a sample, yielding the sample regression function: $\hat{Y}_i = 26 + 0.52X_i$ For the family with income $X_i = 120$ thousand TL and actual consumption $Y_i = 95$ thousand TL: a) Calculate the fitted value $\hat{Y}_i$.
b) Calculate the sample residual $e_i$.
c) Explain why the residual $e_i$ differs from the disturbance $u_i = +7$ found in Template 1.

Exam template 3: Testing linearity in parameters

Question: State whether each regression model is linear under the definition required for classical linear regression: a) $Y_i = \beta_1 + \beta_2 X_i^2 + u_i$
b) $Y_i = \beta_1 + \beta_2^2 X_i + u_i$
c) $Y_i = \beta_1 + \beta_2 \ln(X_i) + u_i$
d) $\ln Y_i = \beta_1 + \beta_2 \ln X_i + u_i$

Cheat-sheet & the 4-step method

Regression as a conditional mean

“Regressing $Y$ on $X$” means studying how the average value of $Y$ changes as $X$ takes different fixed values — not predicting any one individual $Y$ perfectly, but describing the pattern in $Y$‘s behaviour across values of $X$.

The population regression function (PRF)

The PRF is the line traced out by plotting $E(Y|X)$ against $X$ for every possible value of $X$ — connecting the conditional means across the whole range of income. In the textbook’s family consumption example, weekly family income runs $X = 80, 100, \ldots, 260$ dollars and the conditional mean is $E(Y\mid X) = 17 + 0.6X$: at $X=80$ it is $17 + 0.6(80) = 65$, and at $X=260$ it is $17 + 0.6(260) = 173$. The PRF joins all such points. It is a property of the entire population, and in practice it is never directly observed: only a finite sample is available.

Try it with the real 60 families: pick an income column and draw samples.

From PRF to a stochastic model

No single family’s actual consumption sits exactly on the conditional mean line — some spend more than their group’s average, some less. This individual deviation is captured by a disturbance term.

Look at how one family’s expenditure decomposes. Given the family’s income $X_i$, its observed $Y_i$ is the sum of two parts: (1) $E(Y\mid X_i)$, the average spending of all families with that same income — the systematic (deterministic) component; and (2) $u_i$, the family’s own deviation from that average — the random (nonsystematic) component.

Why is there a disturbance term?

Seven standard reasons explain why a stochastic disturbance belongs in the model, all illustrated with the consumption example:

1. Vagueness of theory. The theory behind $Y$‘s behaviour is often incomplete, so $u_i$ stands in for every relevant variable the model leaves out — a family’s spending responds to tastes, habits and health, and no small theory names them all.
2. Unavailability of data. Even when theory names a variable, quantitative information about it may not exist — thriftiness clearly affects saving, but no survey records it as a number.
3. Core versus peripheral variables. Some influences are central and others minor; the weak peripheral ones are not worth modelling one by one and get lumped into $u_i$ — income dominates consumption, while the spending habits of distant relatives still nudge $Y_i$.
4. Intrinsic randomness in human behaviour. Human beings are not fully predictable, so behaviour contains a genuinely random element — two families with the same income may still spend differently on impulse.
5. Poor proxy variables. Measured variables are often imperfect stand-ins for the theoretical ones, or are measured with error — reported income may omit informal earnings, so recorded $X$ imperfectly proxies true income.
6. Principle of parsimony. A good model stays as simple as possible, and $u_i$ carries the countless small omitted influences — one keeps income alone rather than adding dozens of minor regressors.
7. Wrong functional form. The true relationship may not have the shape assumed, and the misfit of a misspecified form shows up in $u_i$ — if consumption bends nonlinearly with income, a straight-line model misses the bend.

The sample regression function (SRF)

Since the PRF is unobservable, econometricians estimate it from a sample, producing the sample regression function, with residuals standing in for the unobservable disturbances.

Estimators, estimates and SRF notation

The sample mean shows the difference. The formula $\bar X = \dfrac{\sum X_i}{n}$ is the estimator: it is the general rule. Feed it one particular sample — say the ten weekly incomes 80, 100, …, 260 — and it returns $\bar X = 1700/10 = 170$ dollars; that single number is the estimate. A different sample would generally produce a different estimate.

The slides write the SRF as $\hat Y_i = b_1 + b_2 X_i$ ($b_1 = \hat\beta_1$, $b_2 = \hat\beta_2$; “Y-hat” is read as the estimator of the conditional mean). Each sample quantity estimates one population quantity:

• $\hat Y_i$ estimates $E(Y\mid X_i)$;
• $b_1$ estimates the intercept $\beta_1$;
• $b_2$ estimates the slope $\beta_2$.

In stochastic form the SRF reads $Y_i = b_1 + b_2 X_i + e_i$. For $X = X_i$ we observe one value $Y = Y_i$; in terms of the SRF it splits as $Y_i = \hat Y_i + e_i$, while in terms of the PRF it splits as $Y_i = E(Y\mid X_i) + u_i$. The residual $e_i$ is thus the observable sample stand-in for the unobservable disturbance $u_i$. Because of sampling fluctuation, the SRF-based estimate of the PRF is at best approximate.

What “linear” regression really means

$Y_i = \beta_1 + \beta_2 X_i^2 + u_i$ is a linear regression model, because it is linear in $\beta_1$ and $\beta_2$ — $X^2$ can simply be treated as a single, ordinary explanatory variable. But $Y_i = \beta_1 + \beta_2^2 X_i + u_i$ is not linear regression, because the parameter $\beta_2$ itself appears squared.

Simple versus multiple regression

A two-variable (simple) regression has one dependent variable and one explanatory variable, exactly the form studied so far. A multiple regression has one dependent variable and two or more explanatory variables — the subject of Lessons 7 to 9 later in this course.

Exam practice

Summary and review

• Regression studies how the conditional mean of Y changes as X varies.
• PRF: $E(Y|X_i) = \beta_1+\beta_2X_i$, the true, unobservable population relationship.
• Stochastic PRF: $Y_i = \beta_1+\beta_2X_i+u_i$.
• Each $Y_i$ splits into a systematic part $E(Y\mid X_i)$ and a random part $u_i$.
• Seven reasons for the disturbance: vagueness of theory, unavailable data, core versus peripheral variables, intrinsic randomness, poor proxies, parsimony, wrong functional form.
• Estimator = the rule (e.g. $\bar X = \sum X_i/n$); estimate = the number it returns for one sample.
• Stochastic SRF: $Y_i = b_1 + b_2X_i + e_i$, where $b_1$ and $b_2$ estimate the intercept and slope.
• SRF: $\hat Y_i = \hat\beta_1+\hat\beta_2X_i$, with residual $e_i = Y_i - \hat Y_i$, estimated from an actual sample.
• Disturbance $u_i$ (population, unobservable) versus residual $e_i$ (sample, observable) is a distinction that recurs throughout the course.
• “Linear” regression means linear in the parameters, not necessarily in the variables.
• Simple (two-variable) regression has one explanatory variable; multiple regression has two or more.

Step-by-step examples

The ladder below climbs from easy to exam level. Try each step on paper before you reveal it — attempting an example yourself, even badly, teaches more than reading a finished solution.

Easy

Medium

Exam level