Policy evaluation in the real world: Experiences with translating cutting-edge methods for an applied audience

Abstract

The policy evaluation methods literature has been growing rapidly in recent years, with several new developments that enable stronger, more robust causal inference in this setting. However, translating these ideas into meaningful applied science can be challenging. In this talk, we present several strategies for successfully communicating the need for and the use of advanced causal inference methods in health policy evaluation. Throughout, we use experiences gleaned from recent work using augmented synthetic controls to investigate the effects of state opioid prescribing laws on use of opioids and other pain treatments among commercially insured adults in the U.S. In particular, we focus on the importance of study design in isolating the effect of a law in a complex policy environment: given the rapidly growing set of methods available, it is increasingly critical to select an approach appropriate for the research question. We also discuss strategies for clearly justifying the use of the chosen method as well as considerations for explaining effects and inference from cutting-edge methods to address a scientific question. With this, we hope to narrow the gap between theory and practice in the policy evaluation space.