OLS (Robust SEs, Clustering)

Motivating Example

Imagine you are a labor economist in the 1970s, and you want to answer a deceptively simple question: how much more does an additional year of education earn you?

Jacob Mincer formalized this question in what became the most-estimated equation in all of economics (Mincer, 1974). The Mincer earnings equation says:

You run this regression on a sample of workers and find $\hat{\beta}_1 = 0.10$. Does that mean one more year of school causes a 10% increase in wages?

Maybe. But maybe not. People who get more education are also different in ways you cannot observe — they may be more motivated, have wealthier parents, or live in areas with better schools. If those unobserved factors also affect wages, your OLS estimate is picking up a mix of the true effect of education and the effect of everything correlated with education that you did not control for.

This tension — between what OLS estimates and what you want it to estimate — is the central challenge of applied empirical research, and it is the reason this entire website exists. OLS is the starting point. Every other method on this site exists because OLS alone is often not enough.

A. Overview

What OLS Does

OLS (Ordinary Least Squares) finds the linear function that best predicts your outcome variable, where "best" means the one that minimizes the sum of squared prediction errors. If you have an outcome $Y$ and regressors $X_1, X_2, \ldots, X_k$, OLS finds the coefficients $\hat{\beta}_0, \hat{\beta}_1, \ldots, \hat{\beta}_k$ that minimize:

In plain language: OLS draws the line (or hyperplane) through your data that makes the vertical distances from the points to the line as small as possible, on average.

What OLS Estimates

The OLS coefficient $\hat{\beta}_1$ estimates the conditional expectation function — specifically, the linear projection of $Y$ onto $X$. This coefficient tells you: "On average, how does $Y$ differ between observations that differ by one unit of $X_1$, holding the other regressors constant?"

This projection is a descriptive statement. It becomes causal only under additional assumptions (see Section B).

When to Use OLS

• You have a continuous outcome variable
• You want to characterize the conditional mean relationship between variables
• You want a starting point before applying more sophisticated methods
• Your research design has addressed endogeneity (e.g., through randomization, controls, or as a first stage for IV)

When NOT to Use OLS

• Your outcome is binary (use logit/probit), a count (use Poisson/negative binomial), or censored at zero or some threshold (consider a Tobit or other censored regression model, not covered in this catalog)
• You suspect important unobserved confounders and have no strategy to address them — see the discussion of selection bias for why this matters
• You want to claim causality but have no identification strategy

Common Confusions

B. Identification

For OLS to give you unbiased and consistent estimates, you need the following assumptions. We state them first in plain language, then formally.

Assumption 1: Linearity

Plain language: The true relationship between $Y$ and the $X$'s is (approximately) linear. If the relationship is actually curved, your straight line will miss it.

Formally: $Y_i = X_i'\beta + u_i$ for some true parameter vector $\beta$.

Assumption 2: Random Sampling

Plain language: Each observation in your dataset is drawn independently from the same population. This assumption fails, for example, if you oversample certain groups or if observations within clusters are correlated.

Assumption 3: No Perfect Multicollinearity

Plain language: No regressor is a perfect linear function of the others. If two variables are perfectly correlated (e.g., you include both "age" and "years since birth"), OLS cannot separate their effects and the estimation breaks entirely.

Assumption 4: Zero Conditional Mean (Exogeneity)

Plain language: The error term $u_i$ — which captures everything affecting $Y$ that is not in your regressors — is on average zero for every value of $X$. This condition is the critical assumption. If people who get more education also have higher unobserved ability, and ability affects wages, then the error term is correlated with education and this assumption fails.

Formally: $E[u_i | X_i] = 0$.

Gauss-Markov Assumptions (for efficiency)

If Assumptions 1-4 hold, OLS is unbiased. For OLS to be the Best Linear Unbiased Estimator (BLUE) — meaning no other linear estimator has smaller variance — you additionally need:

Assumption 5: Homoscedasticity

Plain language: The spread of the error terms is the same regardless of the value of $X$. If errors are larger for some observations than others (e.g., wage variance is higher for highly educated workers), OLS estimates remain unbiased but the conventional standard errors are wrong.

Formally: $\text{Var}(u_i | X_i) = \sigma^2$ for all $i$.

Assumption 6: No Serial Correlation

Plain language: The error for one observation is not correlated with the error for another. This assumption typically fails with time series or panel data.

C. Visual Intuition

Think of OLS geometrically. You have a cloud of data points in a scatter plot. OLS draws the line through the cloud that minimizes the sum of the squared vertical distances from each point to the line.

When confounding strength is zero, OLS recovers the true effect. As you increase confounding — meaning the omitted variable is more strongly correlated with both the regressor and the outcome — the OLS estimate drifts away from the truth. This drift is omitted variable bias in action.

The simulation above generates synthetic data where Y = β·X + 2·U + ε and X is correlated with the unobserved confounder U. The "biased" OLS estimate omits U; the "controlled" estimate includes U. Adjust the confounding strength slider to see how the bias magnitude changes. Try setting the confounding to zero — both estimators converge to the true β.

D. Mathematical Derivation

E. Implementation

Basic OLS with Robust Standard Errors

1library(estimatr)
2
3# OLS with HC1 robust standard errors (matches Stata's vce(robust))
4m1 <- lm_robust(lwage ~ educ + exper + I(exper^2),
5              data = df,
6              se_type = "HC1")
7summary(m1)
8
9# Clustered standard errors (CR2 is the estimatr default; bias-reduced)
10m2 <- lm_robust(lwage ~ educ + exper + I(exper^2),
11              data = df,
12              clusters = state,
13              se_type = "CR2")
14summary(m2)

F. Diagnostics

1# --- Fit the base model ---
2library(lmtest)
3library(car)
4
5m <- lm(lwage ~ educ + exper + I(exper^2), data = df)
6
7# F.1 Residual plot: residuals vs fitted values
8plot(fitted(m), resid(m),
9   xlab = "Fitted values", ylab = "Residuals",
10   main = "Residuals vs Fitted")
11abline(h = 0, lty = 2, col = "red")
12
13# F.2 Variance Inflation Factors
14vif(m)
15
16# F.3 Breusch-Pagan test for heteroscedasticity
17bptest(m)
18
19# F.4 Normality of residuals
20shapiro.test(resid(m)[1:5000])  # Shapiro-Wilk (max 5000 obs)
21qqnorm(resid(m)); qqline(resid(m), col = "red")
22
23# F.5 Ramsey RESET test for functional form
24resettest(m, power = 2:3, type = "fitted")

F.1 Residual Plots

Plot residuals ($\hat{u}_i$) against fitted values ($\hat{Y}_i$) and against each regressor. You should see a random scatter with no pattern. A funnel shape indicates heteroscedasticity; a curve indicates misspecification.

F.2 Variance Inflation Factors (VIF)

VIF measures how much multicollinearity inflates the variance of each coefficient. A common rule of thumb: VIF > 10 signals a problem. In Stata: estat vif. In R: car::vif(model). In Python: statsmodels variance_inflation_factor.

F.3 Heteroscedasticity Tests

The Breusch-Pagan test and White test formally test whether the error variance is constant. In practice, robust standard errors are a common default — they are valid whether or not heteroscedasticity is present, though they can be anti-conservative in small samples (see Section E above).

F.4 Normality of Residuals

OLS does not require normally distributed errors for consistency or unbiasedness. Normality is only needed for exact finite-sample inference (the t and F distributions). With large samples, the Central Limit Theorem ensures approximate normality of $\hat{\beta}$ regardless. Do not reject an otherwise good model because of non-normal residuals.

F.5 Ramsey RESET Test

Tests for omitted nonlinear terms. If the test rejects, consider adding quadratic or interaction terms.

Interpreting Your Results

How to Interpret Coefficients

Linear-linear: $\hat{\beta}_1 = 0.10$ means a one-unit increase in $X$ is associated with a 0.10-unit increase in $Y$, holding other variables constant.

Log-linear (log outcome): $\hat{\beta}_1 = 0.10$ means a one-unit increase in $X$ is associated with approximately a 10% increase in $Y$.

Log-log (both logged): $\hat{\beta}_1 = 0.10$ means a 1% increase in $X$ is associated with a 0.10% increase in $Y$ (an elasticity).

Confidence Intervals

A 95% confidence interval of $[0.05, 0.15]$ means: if you repeated this study many times and computed a confidence interval each time, 95% of those intervals would contain the true $\beta$. It does not mean there is a 95% probability that $\beta$ lies in this particular interval.

Statistical vs. Economic Significance

A coefficient can be statistically significant (small p-value) but economically trivial (tiny effect size), or statistically insignificant (large p-value) but economically meaningful (large point estimate with wide confidence interval due to limited data). It is important to discuss both.

Common Misstatements to Avoid

• Do not say "X causes Y" unless your design supports it
• Do not say "X has no effect" when you mean "the coefficient is not statistically significant" — a null result could mean no effect or insufficient power
• Do not interpret R-squared as a measure of model quality or causal validity
• Do not compare R-squared across models with different dependent variables

G. What Can Go Wrong

H. Practice

H.1 Concept Checks

H.2 Guided Exercise

H.3 Error Detective

H.5 You Are the Referee

I. Swap-In: When to Use Something Else

• Fixed effects: When unobserved time-invariant confounders are the primary concern and panel data are available.
• IV / 2SLS: When endogeneity from simultaneity, measurement error, or omitted variables cannot be addressed by controls alone, and a valid instrument is available.
• Logit / Probit: When the outcome is binary (0/1). OLS (the linear probability model) can still be useful for average marginal effects, but logit/probit avoids predictions outside [0, 1].
• Poisson / Negative Binomial: When the outcome is a non-negative count or inherently multiplicative (e.g., trade flows, patent counts).
• Difference-in-differences: When a policy change creates a natural experiment with a comparison group and temporal variation.

J. Reviewer Checklist