1152 — Risk-Adjustment Approach Impacts Identification of High and Low Performance Hospitals

Cook MA, Health Services Research and Development, VA Puget Sound Healthcare System, Department of Veterans Affairs; Hebert PL, Health Services Research and Development, VA Puget Sound Healthcare System, Department of Veterans Affairs; Wong ES, Health Services Research and Development, VA Puget Sound Healthcare System, Department of Veterans Affairs; Rinne ST, VA Connecticut Healthcare System, Department of Veterans Affairs, West Haven, CT; Sun H, Health Services Research and Development, VA Puget Sound Healthcare System, Department of Veterans Affairs; Au DH, Health Services Research and Development, VA Puget Sound Healthcare System, Department of Veterans Affairs; Liu CF, Health Services Research and Development, VA Puget Sound Healthcare System, Department of Veterans Affairs;

Objectives:
Risk-adjustment approach may influence detection of high and low performing hospitals, which impacts identification of organizational factors associated with performance. We compared the ability of two risk-adjustment approaches to reliably detect VA hospitals with relatively high and low 30-day all-cause readmission rates for COPD.

Methods:
The study sample included 129 VA hospitals with 35,307 COPD inpatients hospitalized between fiscal years 2009 and 2011. We modeled readmission rates using two risk-adjustment approaches. The first approach, used by Center for Medicare and Medicare Services (CMS), was a random effects model comparing predicted readmission rates for each hospital (shrinkage estimates) to rates expected if the same patients were treated at an average hospital. Adjusted rates were obtained by multiplying this ratio by the national average rate. The second approach used hospital-level fixed effects to predict readmission rates at each hospital while holding patient characteristics constant at the national average. K-fold validation was used to assess reliability, with out of sample predictions from each model compared to full sample predictions across ten runs.

Results:
The mean absolute difference in adjusted readmission rates between hospitals ranking in the bottom (poorest performing) and top ten was 2 percentage points (s.d 0.3%) for the CMS method and 10 percentage points (s.d. 0.4%) for the fixed effect method. The bottom 10 hospitals were correctly predicted 89% (s.d 7.3%) of the time by the CMS model and 82% (s.d. 11.3%) of the time by the fixed effects model. Both the CMS model and the fixed effects model correctly predicted the top 10 hospitals 78% of the time (s.d. 9.2% and 10.3%, respectively).

Implications:
Our measure of reliability did not favor either approach, but the approaches differed in their identification of extremely high or low performing hospitals. The CMS approach resulted in a narrow distribution of readmission rates across hospitals, while the fixed effect approach, holding patient characteristics constant, resulted in a wider distribution and facilitated identification of performance outliers.

Impacts:
Risk-adjustment method may influence the appearance of outliers in performance. Researchers should choose a risk-adjustment approach that is aligned with the aim of their analysis.