OLS (Robust SEs, Clustering)

Motivating Example: The Mincer Earnings Equation#

Imagine you are a labor economist in the 1970s, and you want to answer a deceptively simple question: how much more does an additional year of education earn you?

Jacob Mincer formalized this question in what became the most-estimated equation in all of economics (Mincer, 1974). The Mincer earnings equation says:

You run this regression on a sample of workers and find $\hat{\beta}_1 = 0.10$. Does that mean one more year of school causes a 10% increase in wages?

Maybe. But maybe not. People who get more education are also different in ways you cannot observe — they may be more motivated, have wealthier parents, or live in areas with better schools. If those unobserved factors also affect wages, your OLS estimate is picking up a mix of the true effect of education and the effect of everything correlated with education that you did not control for.

This tension — between what OLS estimates and what you want it to estimate — is the central challenge of applied empirical research, and it is the reason this entire website exists. OLS is the starting point. Every other method on this site exists because OLS alone is often not enough.

What OLS Does#

OLS (Ordinary Least Squares) finds the linear function that best predicts your outcome variable, where "best" means the one that minimizes the sum of squared prediction errors. If you have an outcome $Y$ and regressors $X_1, X_2, \ldots, X_k$, OLS finds the coefficients $\hat{\beta}_0, \hat{\beta}_1, \ldots, \hat{\beta}_k$ that minimize:

In plain language: OLS draws the line (or hyperplane) through your data that makes the vertical distances from the points to the line as small as possible, on average.

What OLS Estimates#

The OLS coefficient $\hat{\beta}_1$ estimates the best linear predictor (linear projection) of $Y$ onto $X$, which coincides with the conditional expectation function (CEF) when the CEF is linear. This coefficient tells you: "On average, how does $Y$ differ between observations that differ by one unit of $X_1$, holding the other regressors constant?"

Even when the CEF is nonlinear, OLS provides the minimum mean squared error linear approximation to it (Angrist & Pischke, 2009). This projection is a descriptive statement. It becomes causal only under additional assumptions (see Section B).

When to Use OLS#

• You have a continuous outcome variable
• You want to characterize the conditional mean relationship between variables
• You want a starting point before applying more sophisticated methods
• Your research design has addressed endogeneity (e.g., through randomization, controls, or as a first stage for instrumental variables (IV))

When NOT to Use OLS#

• Your outcome is binary (use logit/probit), a count (use Poisson/negative binomial), or censored at zero or some threshold (consider a Tobit or other censored regression model, not covered in this catalog)
• You suspect important unobserved confounders and have no strategy to address them — see the discussion of selection bias for why this matters
• You want to claim causality but have no identification strategy

Common Confusions#

For OLS to give you unbiased estimates (and consistent estimates under weaker conditions), you need the following assumptions. We state them first in plain language, then formally.

Assumption 1: Linearity#

Plain language: The true relationship between $Y$ and the $X$'s is (approximately) linear. If the relationship is actually curved, your straight line will miss it.

Formally: $Y_i = X_i'\beta + \varepsilon_i$ for some true parameter vector $\beta$.

Assumption 2: Random Sampling#

Plain language: Each observation in your dataset is drawn independently from the same population. This assumption fails, for example, if you oversample certain groups or if observations within clusters are correlated.

Assumption 3: No Perfect Multicollinearity#

Plain language: No regressor is a perfect linear function of the others. If two variables are perfectly correlated (e.g., you include both "age" and "years since birth"), OLS cannot separate their effects and the estimation breaks entirely.

Assumption 4: Zero Conditional Mean (Exogeneity)#

Plain language: The error term $\varepsilon_i$ — which captures everything affecting $Y$ that is not in your regressors — is on average zero for every value of $X$. This condition is the critical assumption. If people who get more education also have higher unobserved ability, and ability affects wages, then the error term is correlated with education and the exogeneity assumption fails.

Formally: $E[\varepsilon_i | X_i] = 0$.

Gauss-Markov Assumptions (for efficiency)#

If Assumptions 1-4 hold, OLS is unbiased. For OLS to be the Best Linear Unbiased Estimator (BLUE) — meaning no other linear estimator has smaller variance — you additionally need:

Assumption 5: Homoscedasticity#

Plain language: The spread of the error terms is the same regardless of the value of $X$. If errors are larger for some observations than others (e.g., wage variance is higher for highly educated workers), OLS estimates remain unbiased but the conventional standard errors are wrong.

Formally: $\text{Var}(\varepsilon_i | X_i) = \sigma^2$ for all $i$.

Assumption 6: No Serial Correlation (time series/panel only)#

Plain language: The error for one observation is not correlated with the error for another. This assumption is automatically satisfied with cross-sectional data under random sampling (Assumption 2) and is only a separate concern with time series or panel data.

Think of OLS geometrically. You have a cloud of data points in a scatter plot. OLS draws the line through the cloud that minimizes the sum of the squared vertical distances from each point to the line.

Each coefficient in a multivariate regression equals the bivariate slope coefficient from regressing $Y$ on the residual from regressing that $X$ on all other covariates. This result is the Frisch-Waugh-Lovell theorem, also called the regression anatomy formula (Angrist & Pischke, 2009). The practical implication is that a multivariate OLS coefficient isolates only the variation in $X$ that is not explained by the other regressors.

Basic OLS with Robust Standard Errors#

1# Requires: estimatr
2library(estimatr)
3
4# --- Step 1: OLS with Robust Standard Errors ---
5# lm_robust() fits OLS and computes heteroskedasticity-robust SEs.
6# HC1 matches Stata's vce(robust) with degrees-of-freedom correction.
7# I(exper^2) includes experience-squared to capture diminishing returns.
8m1 <- lm_robust(lwage ~ educ + exper + I(exper^2),
9              data = df,
10              se_type = "HC1")
11# summary() reports coefficients, robust SEs, t-stats, and R-squared.
12# The educ coefficient = approximate % wage increase per year of education.
13summary(m1)
14
15# --- Step 2: Clustered Standard Errors ---
16# When observations within a group (e.g., state) are correlated,
17# cluster at that level to avoid understating uncertainty.
18# CR2 (bias-reduced) is the estimatr default; better for few clusters.
19m2 <- lm_robust(lwage ~ educ + exper + I(exper^2),
20              data = df,
21              clusters = state,
22              se_type = "CR2")
23# Compare SEs to Step 1: clustered SEs are typically larger,
24# reflecting within-cluster correlation.
25summary(m2)

1# --- Fit the base model ---
2library(lmtest)
3library(car)
4
5m <- lm(lwage ~ educ + exper + I(exper^2), data = df)
6
7# F.1 Residual plot: residuals vs fitted values
8plot(fitted(m), resid(m),
9   xlab = "Fitted values", ylab = "Residuals",
10   main = "Residuals vs Fitted")
11abline(h = 0, lty = 2, col = "red")
12
13# F.2 Variance Inflation Factors
14vif(m)
15
16# F.3 Breusch-Pagan test for heteroscedasticity
17bptest(m)
18
19# F.4 Normality of residuals
20shapiro.test(resid(m)[1:5000])  # Shapiro-Wilk (max 5000 obs)
21qqnorm(resid(m)); qqline(resid(m), col = "red")
22
23# F.5 Ramsey RESET test for functional form
24resettest(m, power = 2:3, type = "fitted")

F.1 Residual Plots#

Plot residuals ($\hat{\varepsilon}_i$) against fitted values ($\hat{Y}_i$) and against each regressor. You should see a random scatter with no pattern. A funnel shape indicates heteroscedasticity; a curve indicates misspecification.

F.2 Variance Inflation Factors (VIF)#

VIF measures how much multicollinearity inflates the variance of each coefficient. A common rule of thumb: VIF > 10 signals a problem. In Stata: estat vif. In R: car::vif(model). In Python: statsmodels variance_inflation_factor.

F.3 Heteroscedasticity Tests#

The Breusch-Pagan test and White test formally test whether the error variance is constant. In practice, robust standard errors are a common default — they are valid whether or not heteroscedasticity is present, though they can be anti-conservative in small samples (see Section E above).

F.4 Normality of Residuals#

OLS does not require normally distributed errors for consistency or unbiasedness. Normality is only needed for exact finite-sample inference (the t and F distributions). With large samples, the Central Limit Theorem ensures approximate normality of $\hat{\beta}$ regardless. Do not reject an otherwise good model because of non-normal residuals.

F.5 Ramsey RESET Test#

Tests for omitted nonlinear terms. If the test rejects, consider adding quadratic or interaction terms.

Interpreting Your Results#

Correct interpretation depends on the functional form of the regression and the units of measurement.

How to Interpret Coefficients#

Linear-linear: $\hat{\beta}_1 = 0.10$ means a one-unit increase in $X$ is associated with a 0.10-unit increase in $Y$, holding other variables constant.

Log-linear (log outcome): $\hat{\beta}_1 = 0.10$ means a one-unit increase in $X$ is associated with approximately a 10% increase in $Y$.

Log-log (both logged): $\hat{\beta}_1 = 0.10$ means a 1% increase in $X$ is associated with a 0.10% increase in $Y$ (an elasticity).

Confidence Intervals#

A 95% confidence interval of $[0.05, 0.15]$ means: if you repeated this study many times and computed a confidence interval each time, 95% of those intervals would contain the true $\beta$. It does not mean there is a 95% probability that $\beta$ lies in this particular interval.

Statistical vs. Economic Significance#

A coefficient can be statistically significant (small p-value) but economically trivial (tiny effect size), or statistically insignificant (large p-value) but economically meaningful (large point estimate with wide confidence interval due to limited data). It is important to discuss both.

Common Misstatements to Avoid#

• Do not say "X causes Y" unless your design supports it
• Do not say "X has no effect" when you mean "the coefficient is not statistically significant" — a null result could mean no effect or insufficient power
• Do not interpret R-squared as a measure of model quality or causal validity
• Do not compare R-squared across models with different dependent variables

The short regression coefficient equals the long regression coefficient plus the effect of the omitted variable times the regression of the omitted on the included: $\text{plim}(\hat{\beta}_{\text{short}}) = \beta_{\text{long}} + \beta_{\text{omitted}} \times \delta$, where $\delta$ is the coefficient from regressing the omitted variable on the included variable (Wooldridge, 2010). This formula shows that OVB is zero only when the omitted variable is uncorrelated with the included regressor ($\delta = 0$) or has no effect on the outcome ($\beta_{\text{omitted}} = 0$).

Classical measurement error in an explanatory variable produces attenuation bias — the coefficient is biased toward zero by the factor $\sigma_x^2 / (\sigma_x^2 + \sigma_e^2)$, called the reliability ratio. Measurement error in the dependent variable does not bias coefficients but inflates standard errors (Wooldridge, 2010).

H.1 Concept Checks#

H.2 Guided Exercise#

H.3 Error Detective#

H.4 You Are the Referee#

• Fixed effects: When unobserved time-invariant confounders are the primary concern and panel data are available.
• IV / 2SLS: When endogeneity from simultaneity, measurement error, or omitted variables cannot be addressed by controls alone, and a valid instrument is available.
• Logit / Probit: When the outcome is binary (0/1). OLS (the linear probability model) can still be useful for average marginal effects, but logit/probit avoids predictions outside [0, 1].
• Poisson / Negative Binomial: When the outcome is a non-negative count or inherently multiplicative (e.g., trade flows, patent counts).
• Difference-in-differences: When a policy change creates a natural experiment with a comparison group and temporal variation.