Staggered Adoption DiD (Modern Estimators)

Motivating Example

Imagine you are studying the effect of state-level anti-discrimination laws on hiring outcomes for minority workers. These laws were not all passed at once — different states adopted them at different times between 1970 and 2000. This pattern is staggered adoption, and it is incredibly common in policy evaluation. The canonical 2x2 DiD framework handles a single treatment date cleanly, but staggered rollout introduces new complications.

You might naturally reach for a two-way fixed effects regression:

where $D_{it} = 1$ once state $i$ has adopted the law. This specification seems perfectly reasonable. You have unit fixed effects to absorb time-invariant state differences, year fixed effects to absorb common shocks, and $\delta$ captures the treatment effect.

But here is the surprising result that has reshaped applied econometrics since roughly 2019: when treatment effects are heterogeneous across cohorts or evolve dynamically over time, this regression can give you a negative estimate even when the true treatment effect is positive for every single unit (Goodman-Bacon, 2021; de Chaisemartin & D'Haultfoeuille, 2020).

This result is not a typo. It is a consequence of how TWFE constructs comparisons under staggered timing — specifically, the use of already-treated units as controls, whose changing outcomes contaminate the estimate. Understanding this problem is now important for applied researchers working with staggered adoption designs.

A. Overview: The TWFE Problem

The core issue is the following: when treatment rolls out at different times, TWFE does not just compare treated units to never-treated units. It also makes "forbidden comparisons" — comparing later-treated units to already-treated units. When already-treated units serve as controls, their treatment effects contaminate the estimate.

If treatment effects are homogeneous (the same for every unit at every point in time), this weighting does not matter. But if effects differ across cohorts or evolve over time — which is common in practice — TWFE can assign negative weights to some treatment effects, producing an estimate that is a distorted average of the true effects. Visualizing these dynamics through an event study is essential for diagnosing the problem.

Goodman-Bacon (2021) showed that the TWFE estimator is a weighted average of all possible 2x2 DiD comparisons, including the problematic ones where early-treated units serve as controls for later-treated units.

Common Confusions

"Does TWFE always give wrong answers?" No. If treatment effects are truly homogeneous across cohorts and over time, TWFE works fine. Running a sensitivity analysis on the degree of heterogeneity can help you assess how much the bias matters in your specific application. The problem arises specifically when effects are heterogeneous. Unfortunately, heterogeneous effects are common in many applied settings — treatment effects often vary across cohorts, grow or fade over time, or depend on implementation context. And the difficulty is that you usually cannot tell in advance whether the heterogeneity is severe enough to matter.

"Can I just add cohort-by-time interactions?" Adding interactions can help, but it changes what you are estimating and can create its own problems. The modern estimators are specifically designed to handle this complication correctly.

"Do I need to re-do all my old papers?" Not necessarily. It is advisable to run the Goodman-Bacon decomposition on your old estimates to check whether the problematic comparisons drive your results. If the weights are mostly positive and the problematic comparisons are a small share, your original results may be fine.

"Which modern estimator should I use?" There is no single best answer. Callaway and Sant'Anna (2021) is among the most flexible and widely adopted options as of 2024. de Chaisemartin and D'Haultfoeuille (2020) is well-suited when you want a simple overall average effect. Borusyak et al. (2024) offer an imputation-based approach that is efficient and intuitive. We discuss each below.

B. Identification

The Goodman-Bacon Decomposition

Goodman-Bacon (2021) proved that the TWFE estimator $\hat{\delta}^{TWFE}$ can be written as:

where $\hat{\delta}_{kl}$ is the simple 2x2 DiD estimate comparing cohort $k$ (treated) to cohort $l$ (control), and $w_{kl}$ are non-negative weights (proportional to group sizes and treatment timing variance) that sum to one. The Goodman-Bacon weights themselves are never negative; the problem is that some $\hat{\delta}_{kl}$ comparisons use already-treated units as controls, which biases those individual estimates. (Separately, de Chaisemartin and D'Haultfoeuille (2020) show a different decomposition where TWFE weights on individual treatment effects can be negative.)

The weights depend on:

• Group sizes (larger groups get more weight)
• Variance in treatment timing (more variation = more weight)
• Whether the comparison is "clean" (treated vs. never-treated) or "contaminated" (treated vs. already-treated)

The contaminated comparisons are the problem. When an already-treated unit's outcomes have been affected by treatment, using it as a control biases the comparison.

Modern Solutions

All modern estimators share a common strategy: avoid the forbidden comparisons. They differ in how they achieve this goal.

Callaway and Sant'Anna (2021) estimate group-time average treatment effects $ATT(g,t)$ — the effect for cohort $g$ at time $t$ — using only never-treated or not-yet-treated units as controls. These building blocks are then aggregated into summary measures.

de Chaisemartin and D'Haultfoeuille (2020) estimate a different parameter: the average effect of switching treatment on among the switchers. They show that this parameter is robust to heterogeneous effects.

Key Assumptions

All approaches require:

1. Parallel trends for each cohort: In the absence of treatment, each cohort would have followed the same trend as its control group.
2. No anticipation: Treatment has no effect before it is implemented.
3. Irreversibility (often): Units do not switch treatment off once adopted. (Some estimators, such as de Chaisemartin and D'Haultfoeuille, can accommodate treatment reversals.)

C. Visual Intuition

The Goodman-Bacon decomposition provides an illuminating diagnostic plot. It shows each 2x2 comparison (treated vs. control cohort pair) as a dot, with the x-axis showing the weight and the y-axis showing the 2x2 estimate. The TWFE estimate is the weighted average.

If the dots are clustered together, TWFE is fine. If the "clean" comparisons (treated vs. never-treated) give very different estimates from the "contaminated" comparisons (treated vs. already-treated), and the contaminated comparisons have large weights, you have a problem.

D. Mathematical Derivation

E. Implementation

1library(did)          # Callaway & Sant'Anna
2library(fixest)       # Sun & Abraham via sunab()
3library(bacondecomp)  # Goodman-Bacon decomposition
4library(ggplot2)      # Plotting
5
6# Step 1: Goodman-Bacon decomposition
7# Decomposes the TWFE estimate into all 2x2 DiD comparisons
8bacon_out <- bacon(y ~ D, data = df, id_var = "unit_id", time_var = "year")
9
10# Plot each 2x2 comparison: weight on x-axis, estimate on y-axis
11# Points colored by type (treated vs never-treated, already-treated, etc.)
12ggplot(bacon_out) + aes(x = weight, y = estimate, color = type) +
13geom_point() + geom_hline(yintercept = 0)
14
15# Step 2: Callaway & Sant'Anna estimator
16# Estimates group-time ATT(g,t) using only clean comparisons
17# control_group = "notyettreated" uses not-yet-treated units as controls
18cs_out <- att_gt(
19yname = "y", tname = "year", idname = "unit_id", gname = "first_treat",
20data = df, control_group = "notyettreated"
21)
22summary(cs_out)
23ggdid(cs_out)  # Event-study plot of group-time ATTs
24
25# Step 3: Aggregate to overall ATT
26# type = "simple" gives the weighted average across all group-time cells
27agg_cs <- aggte(cs_out, type = "simple")
28summary(agg_cs)
29
30# Step 4: Sun & Abraham via fixest
31# sunab(cohort_var, time_var) implements interaction-weighted estimation
32est_sa <- feols(y ~ sunab(first_treat, year) | unit_id + year,
33              data = df, vcov = ~state)
34iplot(est_sa)

F. Diagnostics

1. Run the Goodman-Bacon decomposition first. Before reaching for a modern estimator, decompose your TWFE estimate. If the problematic (already-treated as control) comparisons have small weights, TWFE may be fine.

2. Compare TWFE to modern estimators. If they give similar results, the heterogeneity bias is small. If they diverge, report the modern estimator.

3. Check group-time effects. Plot all the $ATT(g,t)$ estimates from Callaway and Sant'Anna. This disaggregation reveals heterogeneity across cohorts and over time.

4. Test for pre-trends within each cohort. The overall event study might look clean, but individual cohorts might have pre-trends that cancel each other out.

5. Sensitivity to control group choice. Compare results using never-treated vs. not-yet-treated as the control group. If they diverge substantially, investigate why.

Interpreting Your Results

TWFE and modern estimator agree: Encouraging, but not conclusive on its own. Agreement suggests the heterogeneity bias may be small in your setting, especially if the Goodman-Bacon decomposition confirms that contaminated comparisons receive little weight. Report both estimators and note the agreement, but do not treat agreement alone as proof that TWFE is unbiased — agreement can also occur when both estimators are affected by a common violation (e.g., a parallel trends failure that affects all cohorts similarly).

TWFE and modern estimator diverge: Report the modern estimator as your main result. Show the Goodman-Bacon decomposition to explain why TWFE differs. This divergence is actually a compelling narrative for your paper — it shows you understand the methodology.

Group-time effects vary substantially: This heterogeneity is often substantively interesting. Why do early adopters have different effects than late adopters? Is it because the policy is different, the context is different, or the treated populations are different?

G. What Can Go Wrong

H. Practice

Goodman-Bacon Decomposition:
  Treated vs Never-treated:    3.1 (weight: 0.35)
  Treated vs Not-yet-treated:  2.4 (weight: 0.25)
  Already-treated vs Later:    -0.6 (weight: 0.40)

I. Swap-In: When to Use Something Else

• Canonical 2×2 DiD: When treatment is adopted simultaneously by all treated units — the classic two-group, two-period design avoids the negative-weighting issues of staggered settings.
• Event studies: When the full time profile of dynamic treatment effects is of primary interest, rather than a single summary parameter.
• Synthetic DiD: When the parallel trends assumption is suspect and reweighting control units to match the pre-treatment trajectory of treated units improves credibility.
• Synthetic control: When the number of treated units is very small (one to five) and constructing a data-driven counterfactual is more transparent than assuming parallel trends.

J. Reviewer Checklist