Staggered DiD

Motivating Example: Staggered Anti-Discrimination Laws#

Imagine you are studying the effect of state-level anti-discrimination laws on hiring outcomes for minority workers. These laws were not all passed at once — different states adopted them at different times between 1970 and 2000. This pattern is staggered adoption, and it is incredibly common in policy evaluation. The canonical 2x2 DiD framework handles a single treatment date cleanly, but staggered rollout introduces new complications.

You might naturally reach for a two-way fixed effects regression:

where $D_{it} = 1$ once state $i$ has adopted the law. This specification seems perfectly reasonable. You have unit fixed effects to absorb time-invariant state differences, year fixed effects to absorb common shocks, and $\delta$ captures the treatment effect.

But here is the surprising result that has reshaped applied econometrics since roughly 2019: when treatment effects are heterogeneous across cohorts or evolve dynamically over time, the TWFE regression can give you a negative estimate even when the true treatment effect is positive for every single unit (Goodman-Bacon, 2021); (de Chaisemartin & D'Haultfoeuille, 2020).

This result is not a typo. It is a consequence of how TWFE constructs comparisons under staggered timing — specifically, the use of already-treated units as controls, whose changing outcomes contaminate the estimate. Understanding this problem is now important for applied researchers working with staggered adoption designs.

The core issue is the following: when treatment rolls out at different times, TWFE does not just compare treated units to never-treated units. It also makes "forbidden comparisons" — comparing later-treated units to already-treated units. When already-treated units serve as controls, their treatment effects contaminate the estimate.

If treatment effects are homogeneous (the same for every unit at every point in time), the weighting does not matter. But if effects differ across cohorts or evolve over time — which is common in practice — TWFE can assign negative weights to some treatment effects, producing an estimate that is a distorted average of the true effects. Visualizing these dynamics through an event study is essential for diagnosing the problem.

Goodman-Bacon (2021) showed that the TWFE estimator is a weighted average of all possible 2x2 Difference-in-Differences (DiD) comparisons, including the problematic ones where early-treated units serve as controls for later-treated units.

Common Confusions#

"Does TWFE always give wrong answers?" No. If treatment effects are truly homogeneous across cohorts and over time, TWFE works fine. Running a sensitivity analysis on the degree of heterogeneity can help you assess how much the bias matters in your specific application. The problem arises specifically when effects are heterogeneous. Unfortunately, heterogeneous effects are common in many applied settings — treatment effects often vary across cohorts, grow or fade over time, or depend on implementation context. And the difficulty is that you usually cannot tell in advance whether the heterogeneity is severe enough to matter.

"Can I just add cohort-by-time interactions?" Adding interactions can help, but it changes what you are estimating and can create its own problems. The modern estimators are specifically designed to handle this complication correctly.

"Do I need to re-do all my old papers?" Not necessarily. It is advisable to run the Goodman-Bacon decomposition on your old estimates to check whether the problematic comparisons drive your results. If the weights are mostly positive and the problematic comparisons are a small share, your original results may be fine.

"Which modern estimator should I use?" There is no single best answer. Callaway and Sant'Anna (2021) is among the most flexible and widely adopted options as of 2024. de Chaisemartin and D'Haultfoeuille (2020) is well-suited when you want a simple overall average effect. Borusyak et al. (2024) offer an imputation-based approach that is efficient and intuitive. We discuss each below.

The Goodman-Bacon Decomposition#

Goodman-Bacon (2021) proved that the TWFE estimator $\hat{\delta}^{TWFE}$ can be written as:

where $\hat{\delta}_{kl}$ is the simple 2x2 DiD estimate comparing cohort $k$ (treated) to cohort $l$ (control), and $w_{kl}$ are non-negative weights (proportional to group sizes and treatment timing variance) that sum to one. The Goodman-Bacon weights themselves are never negative; the problem is that some $\hat{\delta}_{kl}$ comparisons use already-treated units as controls, which biases those individual estimates. (Separately, de Chaisemartin and D'Haultfoeuille (2020) show a different decomposition where TWFE weights on individual treatment effects can be negative.)

The weights depend on:

• Group sizes (larger groups get more weight)
• Variance in treatment timing (more variation = more weight)
• Whether the comparison is "clean" (treated vs. never-treated) or "contaminated" (treated vs. already-treated)

The contaminated comparisons are the problem. When an already-treated unit's outcomes have been affected by treatment, using it as a control biases the comparison.

Modern Solutions#

All modern estimators share a common strategy: avoid the forbidden comparisons. They differ in how they achieve this goal.

Callaway and Sant'Anna (2021) estimate group-time Average Treatment Effect on the Treated (ATT) parameters $ATT(g,t)$ — the effect for cohort $g$ at time $t$ — using only never-treated or not-yet-treated units as controls. These building blocks are then aggregated into summary measures.

de Chaisemartin and D'Haultfoeuille (2020) estimate a different parameter: the average effect of switching treatment on among the switchers. They show that this parameter is robust to heterogeneous effects.

Key Assumptions#

All approaches require:

1. Parallel trends for each cohort: In the absence of treatment, each cohort would have followed the same trend as its control group.
2. No anticipation: Treatment has no effect before it is implemented.
3. Irreversibility (often): Units do not switch treatment off once adopted. (Some estimators, such as de Chaisemartin and D'Haultfoeuille, can accommodate treatment reversals.)
4. SUTVA (Stable Unit Treatment Value Assumption): No interference between units — one unit's treatment assignment does not affect another unit's outcomes — and no hidden variations of treatment. If an anti-discrimination law in one state causes firms or workers to relocate to other states, those spillovers contaminate the control group and bias the estimated treatment effect.

The Goodman-Bacon decomposition provides an illuminating diagnostic plot. It shows each 2x2 comparison (treated vs. control cohort pair) as a dot, with the x-axis showing the weight and the y-axis showing the 2x2 estimate. The TWFE estimate is the weighted average.

If the dots are clustered together, TWFE is fine. If the "clean" comparisons (treated vs. never-treated) give very different estimates from the "contaminated" comparisons (treated vs. already-treated), and the contaminated comparisons have large weights, you have a problem.

1# Requires: did, fixest, bacondecomp, ggplot2
2library(did)          # did: Callaway & Sant'Anna (2021) estimator
3library(fixest)       # fixest: Sun & Abraham (2021) via sunab()
4library(bacondecomp)  # bacondecomp: Goodman-Bacon (2021) decomposition
5library(ggplot2)
6
7# --- Step 1: Goodman-Bacon decomposition (diagnostic) ---
8# Decomposes the TWFE estimate into all 2x2 DiD comparisons
9# Reveals whether TWFE uses problematic already-treated units as controls
10bacon_out <- bacon(y ~ D, data = df, id_var = "unit_id", time_var = "year")
11
12# Plot: weight on x-axis, estimate on y-axis, colored by comparison type
13# If "Already treated vs Later treated" comparisons have large weight and yield negative estimates,
14# TWFE may produce biased or even sign-reversed aggregate estimates
15ggplot(bacon_out) + aes(x = weight, y = estimate, color = type) +
16geom_point() + geom_hline(yintercept = 0)
17
18# --- Step 2: Callaway & Sant'Anna estimator ---
19# att_gt() estimates group-time ATT(g,t) using only clean comparisons
20# gname = column with first treatment period (0 or Inf for never-treated)
21# control_group = "notyettreated": uses not-yet-treated units as controls
22# (gives more power but requires treatment timing is exogenous)
23cs_out <- att_gt(
24yname = "y", tname = "year", idname = "unit_id", gname = "first_treat",
25data = df, control_group = "notyettreated"
26)
27summary(cs_out)
28ggdid(cs_out)  # Event-study plot of all group-time ATTs
29
30# --- Step 3: Aggregate to overall ATT ---
31# aggte() aggregates the group-time estimates into a single summary
32# type = "simple": equal-weighted average across all (g,t) cells
33agg_cs <- aggte(cs_out, type = "simple")
34summary(agg_cs)
35# The aggregated ATT is robust to heterogeneous treatment effects
36
37# --- Step 4: Sun & Abraham via fixest ---
38# sunab() implements interaction-weighted estimation within feols()
39# Avoids the "forbidden comparisons" that contaminate standard TWFE
40est_sa <- feols(y ~ sunab(first_treat, year) | unit_id + year,
41              data = df, vcov = ~state)
42iplot(est_sa)
43# Compare with TWFE: if estimates differ, treatment effect heterogeneity is present

1. Run the Goodman-Bacon decomposition first. Before reaching for a modern estimator, decompose your TWFE estimate. If the problematic (already-treated as control) comparisons have small weights, TWFE may be fine.

2. Compare TWFE to modern estimators. If they give similar results, the heterogeneity bias is small. If they diverge, report the modern estimator.

3. Check group-time effects. Plot all the $ATT(g,t)$ estimates from Callaway and Sant'Anna (2021). This disaggregation reveals heterogeneity across cohorts and over time.

4. Test for pre-trends within each cohort. The overall event study might look clean, but individual cohorts might have pre-trends that cancel each other out.

5. Sensitivity to control group choice. Compare results using never-treated vs. not-yet-treated as the control group. If they diverge substantially, investigate why.

Interpreting Your Results#

TWFE and modern estimator agree: Encouraging, but not conclusive on its own. Agreement suggests the heterogeneity bias may be small in your setting, especially if the Goodman-Bacon decomposition confirms that contaminated comparisons receive little weight. Report both estimators and note the agreement, but do not treat agreement alone as proof that TWFE is unbiased — agreement can also occur when both estimators are affected by a common violation (e.g., a parallel trends failure that affects all cohorts similarly).

TWFE and modern estimator diverge: Report the modern estimator as your main result. Show the Goodman-Bacon decomposition to explain why TWFE differs. This divergence is actually a compelling narrative for your paper — it shows you understand the methodology.

Group-time effects vary substantially: This heterogeneity is often substantively interesting. Why do early adopters have different effects than late adopters? Is it because the policy is different, the context is different, or the treated populations are different?

Goodman-Bacon Decomposition:
  Treated vs Never-treated:    3.1 (weight: 0.35)
  Treated vs Not-yet-treated:  2.4 (weight: 0.25)
  Already-treated vs Later:    -0.6 (weight: 0.40)

• Canonical 2×2 DiD: When treatment is adopted simultaneously by all treated units — the classic two-group, two-period design avoids the negative-weighting issues of staggered settings.
• Event studies: When the full time profile of dynamic treatment effects is of primary interest, rather than a single summary parameter.
• Synthetic DiD: When the parallel trends assumption is suspect and reweighting control units to match the pre-treatment trajectory of treated units improves credibility.
• Synthetic control: When the number of treated units is very small (one to five) and constructing a data-driven counterfactual is more transparent than assuming parallel trends.