A Statistical Framework for Understanding Causal Effects that Vary by Treatment Initiation Time in EHR-based Studies

Comparative effectiveness studies using electronic health records (EHR) consider data from patients who could ``enter'' the study cohort at any point during an interval that spans many years in calendar time. Unlike treatments in tightly controlled trials, real-world treatments can evolve over calendar time, especially if comparators include standard of care, or procedures where techniques may improve. Efforts to assess whether treatment efficacy itself is changing are complicated by changing patient populations, with potential covariate shift in key effect modifiers. In this work, we propose a statistical framework to estimate calendar-time specific average treatment effects and describe both how and why effects vary across treatment initiation time in EHR-based studies. Our approach projects doubly robust, time-specific treatment effect estimates onto candidate marginal structural models and uses a model selection procedure to best describe how effects vary by treatment initiation time. We further introduce a novel summary metric, based on standardization analysis, to quantify the role of covariate shift in explaining observed effect changes and disentangle changes in treatment effects from changes in the patient population receiving treatment. Extensive simulations using EHR data from Kaiser Permanente are used to validate the utility of the framework, which we apply to study changes in relative weight loss following two bariatric surgical interventions versus no surgery among patients with severe obesity between 2005-2011.