Doubly Robust / AIPW Estimation

Motivating Example: Evaluating a Job Training Program#

A government wants to evaluate its job training program. Randomization was not possible — the program was offered to anyone who signed up, and the people who signed up are different from those who did not. You observe a rich set of covariates (education, age, prior earnings, employment history, neighborhood characteristics) and are willing to assume that, conditional on these covariates, participation is as good as random.

You have two obvious strategies:

Strategy 1: Outcome regression. Model the outcome (post-program earnings) as a function of treatment and covariates using, say, OLS. If the model is correctly specified, this regression gives you the treatment effect. But if you get the functional form wrong — maybe the true relationship is nonlinear, or there are interactions you missed — your estimate is biased.

Strategy 2: Propensity score weighting. Model the treatment assignment (who signs up) as a function of covariates, as in matching methods. Re-weight observations so that the treated and control groups are balanced on observables. If the propensity score model is correctly specified, this reweighting gives you the treatment effect. But if you misspecify the selection model, the weights are wrong and your estimate is biased.

Each strategy relies on getting one model right. What if you are not sure which one you got right?

Doubly robust estimation combines both strategies into a single estimator that is consistent if either the outcome model or the propensity score model is correctly specified. You only need one of the two to work. This property — known as double robustness — is a key theoretical advantage of the estimator.

The doubly robust estimator, also called the augmented inverse probability weighted (AIPW) estimator, was developed by Robins and colleagues in the biostatistics literature (Robins et al., 1994) (Bang & Robins, 2005).

The intuition is instructive. The AIPW estimator starts with the outcome regression and then corrects it using the propensity score weights. If the outcome model is right, the correction term has expected value zero and does not hurt. If the outcome model is wrong but the propensity score model is right, the correction term exactly removes the bias. You get two chances to be right.

How It Works (Intuitively)#

Think of it in two steps:

1. Predict outcomes for everyone under both treatment and control using your outcome model. Compute the predicted treatment effect.

2. Correct the prediction errors using propensity score weights. For each observation, look at the residual (actual outcome minus predicted outcome). Weight these residuals by the inverse of the propensity score to correct for any remaining imbalance.

If your outcome model is perfect, the residuals are pure noise, and the correction adds nothing. If your outcome model is imperfect but your propensity scores are right, the weighted residuals exactly correct the bias.

Common Confusions#

"Does doubly robust mean I do not need the selection-on-observables assumption?" Absolutely not. Doubly robust estimation still requires conditional independence (also called unconfoundedness or ignorability). It protects against model misspecification, not against omitted variables. If there are unobserved confounders, no amount of modeling sophistication will save you.

"What if both models are wrong?" Then the doubly robust estimator is also wrong. "Doubly robust" does not mean "right in all cases." It means "right if either component is right." If both are wrong, the bias is generally of order equal to the product of the two individual estimation errors — typically smaller than either individual bias, but not zero.

"Is doubly robust the same as double machine learning (DML)?" They are related but distinct. DML uses ML methods for both nuisance functions (outcome model and propensity score) and adds cross-fitting to prevent overfitting bias. The doubly robust property is a building block of DML, but DML adds important refinements. See the DML page for details.

"Should I use the same covariates in both models?" You can, but you do not have to. The outcome model and the propensity score model can include different covariates. In practice, including the same core covariates in both is a reasonable default, but you might want additional variables in one model based on domain knowledge.

The Target Estimand#

We want the Average Treatment Effect on the Treated (ATT) or the Average Treatment Effect (ATE):

Under the conditional independence assumption (CIA):

where $D$ is the treatment indicator and $X$ are observed covariates.

The AIPW Estimator#

The AIPW estimator for the ATE is:

where:

• $\hat{\mu}_1(X_i) = \hat{E}[Y | X = X_i, D = 1]$ is the estimated outcome under treatment
• $\hat{\mu}_0(X_i) = \hat{E}[Y | X = X_i, D = 0]$ is the estimated outcome under control
• $\hat{e}(X_i) = \hat{P}(D = 1 | X = X_i)$ is the estimated propensity score

The Doubly Robust Property#

This estimator is consistent if:

1. The outcome models $\hat{\mu}_1, \hat{\mu}_0$ are correctly specified, OR
2. The propensity score model $\hat{e}$ is correctly specified.

You need at least one to be right, but you do not need both.

Positivity Assumption#

In addition to CIA, you need the positivity (or overlap) assumption:

In words: for every combination of covariate values, there must be a positive probability of being either treated or untreated. Without this condition, the inverse probability weights become unbounded.

Imagine a scatterplot of propensity scores for treated and control units. Good overlap means the two distributions overlap substantially. Poor overlap means there are regions where only treated (or only control) units exist. In those regions, the propensity score is near 0 or 1, and the inverse probability weighting (IPW) weights become extreme.

The doubly robust estimator is most valuable when the propensity score distributions overlap reasonably well, but you are uncertain about whether your outcome model captures the true functional form.

1# Requires: AIPW
2# AIPW: R package for augmented inverse probability weighting
3library(AIPW)
4
5# --- Step 1: Fit AIPW with flexible ML for both nuisance models ---
6# AIPW is consistent if EITHER the outcome or propensity model is correct
7# Q.SL.library: ensemble ML for outcome model E[Y|X,A]
8# g.SL.library: ensemble ML for propensity score P(A=1|X)
9aipw_obj <- AIPW$new(
10Y = df$outcome,
11A = df$treatment,
12W = df[, c("x1", "x2", "x3")],
13Q.SL.library = c("SL.glm", "SL.ranger"),   # outcome: GLM + random forest
14g.SL.library = c("SL.glm", "SL.ranger")    # propensity: GLM + random forest
15)
16aipw_obj$fit()
17aipw_obj$summary()
18# Reports ATE and ATT with confidence intervals
19
20# --- Step 2: Alternative approach using WeightIt + cobalt ---
21library(WeightIt)   # WeightIt: propensity score weighting
22library(cobalt)     # cobalt: covariate balance assessment
23
24# Estimate propensity scores and compute IPW weights for ATE
25w <- weightit(treatment ~ x1 + x2 + x3, data = df, method = "ps",
26            estimand = "ATE")
27# Check covariate balance after weighting (target: SMD < 0.1)
28bal.tab(w)
29
30# --- Step 3: Doubly robust estimation via weighted regression ---
31# Combining IPW weights with outcome regression gives double robustness
32library(survey)
33d <- svydesign(ids = ~1, weights = ~w$weights, data = df)
34fit <- svyglm(outcome ~ treatment + x1 + x2 + x3, design = d)
35summary(fit)
36# Coefficient on treatment: DR estimate of the ATE

1. Propensity score overlap. Plot the propensity score distributions for treated and control groups. If there are regions with no overlap, consider trimming or truncating extreme scores.

2. Covariate balance after weighting. Use standardized mean differences to check whether the covariates are balanced after applying the IPW weights. Absolute SMDs below 0.1 are generally good.

3. Sensitivity analysis. Use the Oster (2019) coefficient stability approach or the Cinelli and Hazlett (2020) sensitivity analysis to assess how robust your results are to omitted variable bias.

4. Extreme weights. Check for observations with very large weights (propensity scores near 0 or 1). These extreme weights can dominate the estimator and inflate variance. Report the distribution of weights and consider trimming at the 1st and 99th percentiles.

5. Model specification tests. Run the outcome model and propensity score model separately and check their fit. For the propensity score, check the c-statistic (AUC). For the outcome model, check residual plots.

Interpreting Your Results#

AIPW, IPW, and regression adjustment agree: All three approaches give similar estimates. This agreement is reassuring and suggests your results are not sensitive to the specific modeling choices.

AIPW and regression disagree, but AIPW and IPW agree: The outcome model may be misspecified. The propensity score model is doing the heavy lifting. Report AIPW as your main result but discuss the sensitivity.

AIPW and IPW disagree, but AIPW and regression agree: The propensity score model may be misspecified. Regression is doing the heavy lifting.

All three disagree: Something fundamental is wrong. Check for positivity violations, influential observations, or misspecification of both models.

Propensity score range: [0.01, 0.99]
Max IPW weight: 142
Covariate balance (max SMD after weighting): 0.18

• Matching: When a transparent matched-pair design is preferred and the propensity score model is well-specified — matching discards unmatched units rather than reweighting them.
• DML (Double/Debiased Machine Learning): When nuisance functions are high-dimensional and cross-fitting is needed for valid inference — DML extends the doubly robust logic with machine-learning estimators.
• Inverse probability weighting (IPW): When the propensity score model is well-specified and a simpler reweighting estimator without an outcome model suffices.
• OLS with controls: When the outcome model is correctly specified and selection bias is modest — regression adjustment alone may be sufficient.