Abstract

Background: In real-world studies of geriatric and palliative care programs, policies, or treatments, treatment initiation may need to be staggered across units in ways that are outside of investigators’ control. If differences across cohorts or in organizational characteristics associated with both treatment timing and outcome are not controlled for in analyses, they may obscure estimates of true treatment effects. Heterogeneous treatment timing is inadequately addressed in most existing methods. Current methods to account for treatment effect timing heterogeneity do not allow a treatment’s effect to be isolated from effects of confounders associated with both timing and outcomes. Significance: Staggered rollouts of policies and practice changes are common within VA — the timing at which a new intervention (e.g., Veteran Directed Care) is rolled out cannot always be controlled. Innovation and Impact: A potential solution involves inverse probability of treatment weights (IPTW) to adjust for confounding across treatment groups defined by receipt timing, but IPTWs lead to biased estimates in cross-sectional evaluations comparing multiple treatments. We have developed an alternative method, vector- based kernel weighting (VBKW), that outperforms IPTW in cross-sectional evaluations. The degree to which VBKW reduces bias and improves efficiency over IPTW in longitudinal applications has not yet been explored. Entropy balancing (EB) weights also may produce unbiased and efficient treatment effect estimates when combined with DD. Researchers need practical guidance for when weighting adjustments with DD may be a superior analytic method to account for treatment timing heterogeneity in a difference-in-differences study, or for when traditional two-way fixed effects (TWFE) approaches may be sufficient. Specific Aims: We aim to compare bias and efficiency of estimates using VBKW, IPTW, EB, and TWFE approaches in analyses of retrospective cohort studies with staggered treatment timing within a cohort (Aim 1) and on data from longitudinal panel studies where treatment effects may vary across and within cohorts of data collection (Aim 2). We will identify the degree of heterogeneity required for VBKW, IPTW, EB, and TWFE to lead to different inferences (Aim 3). Methodology: We will use Monte Carlo (MC) and plasmode simulations to evaluate bias, efficiency, covariate balance, and processing time for each strategy in data obtained from observational studies with staggered treatment timing. MC simulations on investigator-generated data (n=600, 900, 9600) will allow us to examine the impact of different analytic scenarios (e.g., sample distribution across treatment timing groups, dynamic effects) on the relative performance of estimators. Plasmode simulations will allow us to verify that our results are robust to data generating process and will be derived from an observational analysis of Veterans’ self- directed care services (using Corporate Data Warehouse data) and from the Health and Retirement Study (HRS). Next Steps/Implementation: We will identify when VBKW, EB, IPTW, or TWFE is superior for estimating the effect of a treatment provided at different times and summarize our results in practical guidance that we will share with the HSRD community. Our results will improve investigators’ ability to generate rigorous evidence from studies of Veterans’ geriatric and palliative care in which treatment or event timing cannot be controlled.

Dimensions for VA

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

Learn more about Dimensions for VA.

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.     Search Dimensions for this project

DRA: Aging, Older Veterans' Health and Care, Other Conditions

DRE: TRL - Applied/Translational, Data Science

Keywords: Computational Modeling

MeSH Terms: None at this time.