Doubly Robust / AIPW Estimation

Motivating Example

A government wants to evaluate its job training program. Randomization was not possible — the program was offered to anyone who signed up, and the people who signed up are different from those who did not. You observe a rich set of covariates (education, age, prior earnings, employment history, neighborhood characteristics) and are willing to assume that, conditional on these covariates, participation is as good as random.

You have two obvious strategies:

Strategy 1: Outcome regression. Model the outcome (post-program earnings) as a function of treatment and covariates using, say, OLS. If the model is correctly specified, this regression gives you the treatment effect. But if you get the functional form wrong — maybe the true relationship is nonlinear, or there are interactions you missed — your estimate is biased.

Strategy 2: Propensity score weighting. Model the treatment assignment (who signs up) as a function of covariates, as in matching methods. Re-weight observations so that the treated and control groups are balanced on observables. If the propensity score model is correctly specified, this reweighting gives you the treatment effect. But if you misspecify the selection model, the weights are wrong and your estimate is biased.

Each strategy relies on getting one model right. What if you are not sure which one you got right?

Doubly robust estimation combines both strategies into a single estimator that is consistent if either the outcome model or the propensity score model is correctly specified. You only need one of the two to work. This property — known as double robustness — is a key theoretical advantage of the estimator.

A. Overview

The doubly robust estimator, also called the augmented inverse probability weighted (AIPW) estimator, was developed by Robins and colleagues in the biostatistics literature.

The intuition is instructive. The AIPW estimator starts with the outcome regression and then corrects it using the propensity score weights. If the outcome model is right, the correction term has expected value zero and does not hurt. If the outcome model is wrong but the propensity score model is right, the correction term exactly removes the bias. You get two chances to be right.

How It Works (Intuitively)

Think of it in two steps:

1. Predict outcomes for everyone under both treatment and control using your outcome model. Compute the predicted treatment effect.

2. Correct the prediction errors using propensity score weights. For each observation, look at the residual (actual outcome minus predicted outcome). Weight these residuals by the inverse of the propensity score to correct for any remaining imbalance.

If your outcome model is perfect, the residuals are pure noise, and the correction adds nothing. If your outcome model is imperfect but your propensity scores are right, the weighted residuals exactly correct the bias.

Common Confusions

"Does doubly robust mean I do not need the selection-on-observables assumption?" Absolutely not. Doubly robust estimation still requires conditional independence (also called unconfoundedness or ignorability). It protects against model misspecification, not against omitted variables. If there are unobserved confounders, no amount of modeling sophistication will save you.

"What if both models are wrong?" Then the doubly robust estimator is also wrong. "Doubly robust" does not mean "right in all cases." It means "right if either component is right." If both are wrong, the bias could actually be worse than either individual estimator.

"Is doubly robust the same as double machine learning (DML)?" They are related but distinct. DML uses ML methods for both nuisance functions (outcome model and propensity score) and adds cross-fitting to prevent overfitting bias. The doubly robust property is a building block of DML, but DML adds important refinements. See the DML page for details.

"Should I use the same covariates in both models?" You can, but you do not have to. The outcome model and the propensity score model can include different covariates. In practice, including the same core covariates in both is a reasonable default, but you might want additional variables in one model based on domain knowledge.

B. Identification

The Target Estimand

We want the Average Treatment Effect on the Treated (ATT) or the Average Treatment Effect (ATE):

Under the conditional independence assumption (CIA):

where $D$ is the treatment indicator and $X$ are observed covariates.

The AIPW Estimator

The AIPW estimator for the ATE is:

where:

• $\hat{\mu}_1(X_i) = \hat{E}[Y | X = X_i, D = 1]$ is the estimated outcome under treatment
• $\hat{\mu}_0(X_i) = \hat{E}[Y | X = X_i, D = 0]$ is the estimated outcome under control
• $\hat{e}(X_i) = \hat{P}(D = 1 | X = X_i)$ is the estimated propensity score

The Doubly Robust Property

This estimator is consistent if:

1. The outcome models $\hat{\mu}_1, \hat{\mu}_0$ are correctly specified, OR
2. The propensity score model $\hat{e}$ is correctly specified.

You need at least one to be right, but you do not need both.

Positivity Assumption

In addition to CIA, you need the positivity (or overlap) assumption:

In words: for every combination of covariate values, there must be a positive probability of being either treated or untreated. Without this condition, the inverse probability weights become unbounded.

C. Visual Intuition

Imagine a scatterplot of propensity scores for treated and control units. Good overlap means the two distributions overlap substantially. Poor overlap means there are regions where only treated (or only control) units exist. In those regions, the propensity score is near 0 or 1, and the IPW weights become extreme.

The doubly robust estimator is most valuable when the propensity score distributions overlap reasonably well, but you are uncertain about whether your outcome model captures the true functional form.

D. Mathematical Derivation

E. Implementation

1# Using the AIPW package
2library(AIPW)
3
4# Fit AIPW with SuperLearner for both models
5aipw_obj <- AIPW$new(
6Y = df$outcome,
7A = df$treatment,
8W = df[, c("x1", "x2", "x3")],
9Q.SL.library = c("SL.glm", "SL.ranger"),   # outcome model
10g.SL.library = c("SL.glm", "SL.ranger")    # propensity score
11)
12aipw_obj$fit()
13aipw_obj$summary()
14
15# Simpler: using the WeightIt + cobalt packages
16library(WeightIt)
17library(cobalt)
18
19# Estimate propensity scores and weights
20w <- weightit(treatment ~ x1 + x2 + x3, data = df, method = "ps",
21            estimand = "ATE")
22bal.tab(w)  # Check balance
23
24# Outcome regression with weights
25library(survey)
26d <- svydesign(ids = ~1, weights = ~w$weights, data = df)
27fit <- svyglm(outcome ~ treatment + x1 + x2 + x3, design = d)
28summary(fit)

F. Diagnostics

1. Propensity score overlap. Plot the propensity score distributions for treated and control groups. If there are regions with no overlap, consider trimming or truncating extreme scores.

2. Covariate balance after weighting. Use standardized mean differences to check whether the covariates are balanced after applying the IPW weights. Absolute SMDs below 0.1 are generally good.

3. Sensitivity analysis. Use the Oster (2019) coefficient stability approach or the Cinelli and Hazlett (2020) sensitivity analysis to assess how robust your results are to omitted variable bias.

4. Extreme weights. Check for observations with very large weights (propensity scores near 0 or 1). These extreme weights can dominate the estimator and inflate variance. Report the distribution of weights and consider trimming at the 1st and 99th percentiles.

5. Model specification tests. Run the outcome model and propensity score model separately and check their fit. For the propensity score, check the c-statistic (AUC). For the outcome model, check residual plots.

Interpreting Your Results

AIPW, IPW, and regression adjustment agree: All three approaches give similar estimates. This agreement is reassuring and suggests your results are not sensitive to the specific modeling choices.

AIPW and regression disagree, but AIPW and IPW agree: The outcome model may be misspecified. The propensity score model is doing the heavy lifting. Report AIPW as your main result but discuss the sensitivity.

AIPW and IPW disagree, but AIPW and regression agree: The propensity score model may be misspecified. Regression is doing the heavy lifting.

All three disagree: Something fundamental is wrong. Check for positivity violations, influential observations, or misspecification of both models.

G. What Can Go Wrong

H. Practice

Propensity score range: [0.01, 0.99]
Max IPW weight: 142
Covariate balance (max SMD after weighting): 0.18

I. Swap-In: When to Use Something Else

• Matching: When a transparent matched-pair design is preferred and the propensity score model is well-specified — matching discards unmatched units rather than reweighting them.
• DML (Double/Debiased Machine Learning): When nuisance functions are high-dimensional and cross-fitting is needed for valid inference — DML extends the doubly robust logic with machine-learning estimators.
• Inverse probability weighting (IPW): When the propensity score model is well-specified and a simpler reweighting estimator without an outcome model suffices.
• OLS with controls: When the outcome model is correctly specified and selection bias is modest — regression adjustment alone may be sufficient.

J. Reviewer Checklist