Abstract — HSRD 2019
Back1002 — Comparative Effectiveness of Risk-Adjusted CUSUM and Episodic Surgical Performance Evaluation
Lead/Presenter: Nader Massarweh, COIN - Houston
All Authors: Massarweh NN (Center for Innovations in Quality, Effectiveness, and Safety (IQuESt), Michael E DeBakey VA Medical Center & Baylor College of Medicine), ; Rosen T (Center for Innovations in Quality, Effectiveness, and Safety (IQuESt), , Michael E DeBakey VA Medical Center); Dong Y (Center for Innovations in Quality, Effectiveness, and Safety (IQuESt), Michael E DeBakey VA Medical Center); Richardson PA (Center for Innovations in Quality, Effectiveness, and Safety (IQuESt), Michael E DeBakey VA Medical Center); Axelrod DA (Iowa City VA Medical Center); Wilson MA (VA Pittsburgh Healthcare System); Petersen LA (Center for Innovations in Quality, Effectiveness, and Safety (IQuESt), Michael E DeBakey VA Medical Center & Baylor College of Medicine)
Objectives:
The VA Surgical Quality Improvement Program (VASQIP) uses episodic, quarterly evaluation to identify hospital outliers in terms of perioperative morbidity and mortality. Decreasing the time between performance decline and hospital notification would represent an opportunity to improve current quality improvement (QI) efforts. The objective of this study is to compare the identification of low (i.e.: poor) performing VA hospitals using a more real-time method of performance evaluation (time-to-event cumulative sum [CUSUM]) to the current episodic approach, and national standard in surgical QI, used by VASQIP.
Methods:
Hospital-level analysis using VASQIP data (2011-2016). Quarterly identification of low outlier hospitals was performed using two approaches: 1.) risk-adjusted observed-to-expected ratios (O-E); 2.) time-to-event, risk-adjusted CUSUM with V-mask (defined by a radius and a slope to ascertain the rate at which performance is approaching a concerning threshold). V-mask slopes of 0.6-3.0 were evaluated in combination with a predefined radius of 1.0. Correlation between methods for identifying outliers was evaluated using tetrachoric correlation. Lag time for outlier identification using CUSUM relative to quarterly O-E was calculated.
Results:
In 139 VA hospitals, there was good correlation between episodic O-E and CUSUM (range: slope 0.6—rho = 0.52, p < 0.001; slope 3.0—rho = 0.54, p < 0.001). Negative predictive value for CUSUM relative to episodic O-E was > 95%. The V-mask slope with the best performance was 2.5 (area under the curve 0.71). Compared to episodic O-E, the CUSUM was able to identify outlier hospitals a median of 47-49 days (depending on the V-mask slope) from the end of the quarter.
Implications:
CUSUM identifies VA hospitals at risk for outlier performance at an earlier time point compared to episodic analysis and could be used as an early warning signal within VASQIP to prompt VA hospitals to critically evaluate their performance at a time when it is declining instead of when it has already reached an unacceptable level.
Impacts:
More real-time analytic frameworks could provide important information and opportunities for VA hospitals to more proactively (as opposed to reactively) engage in local QI. The CUSUM and other such approaches could help to make VA QI efforts in surgery and other specialties more robust.
