Handling Correlation in Stacked Difference-in-Differences Estimates with Application to Medical Cannabis Policy

Abstract

Difference-in-differences (DiD) is a general framework for assessing causal effects of health policies. One issue with standard DiD methods is staggered adoption: different units (e.g., states) enact policies at different times. Stacking is one approach to DiD in this setting: for each unit enacting a policy, construct a comparison group of units that never enact (or had not yet enacted) the policy, then construct a large dataset with all such cohorts. Recent work, e.g., Callaway & Sant’Anna (2021), developed methods for aggregate data that yield unbiased estimates of average treatment effects with fewer restrictive assumptions than traditional two-way fixed effects models. However, in some cases data is aggregated from the individual level (e.g., from an insurance claims database), and some individuals in comparison units can contribute to comparison groups for multiple treated states, producing correlation between stacked cohorts. Existing methods do not quantify or account for this overlap in the estimation. We demonstrate a bootstrap approach to dealing with the overlap, applied to estimating the effects of state medical cannabis laws on opioid prescribing in the United States.