Risk adjustment is recognized as critical for accurate assessment of quality, fair comparison of providers, and benchmarking across healthcare systems. As chart-based data -- the "gold standard" for risk adjustment -- are costly and time-consuming to collect, administrative data have remained the basis of risk adjustment, despite inadequacies in capturing patient severity. Recent initiatives by the Agency of Healthcare Quality and Research (AHRQ) to enhance administrative data with automated data on laboratory tests and vital signs are an important step towards improving the accuracy of risk-adjustment models.
Taking advantage of readily available VA automated data, the primary goal of this project was to develop cost-effective and clinically sound enhanced risk-adjustment models to profile VA facilities on three discharge outcomes -- 30-day mortality, 30-day readmission and length of stay. For selected admission cohorts, we developed enhanced risk adjustment models by combining VA administrative data with Medicare data, and VA automated data on laboratory tests, vital signs and pharmacy claims. Our objective was to quantify the impact of enhanced risk adjustment on model performance and hospital risk profiles.
Separate enhanced risk-adjustment models were estimated for eight medical conditions that account for a sizable proportion of all VA admissions (16 percent) and inpatient deaths (30 percent) - acute myocardial infarction (AMI), heart failure (HF), pneumonia (PN), chronic obstructive pulmonary disease, gastrointestinal hemorrhage, hip fracture, acute renal failure and acute stroke. One cohort (cirrhosis and alcoholic hepatitis) was excluded due to small sample size. Based on all admissions to VA facilities during FY 2001-10, disease-specific cohorts were identified. We merged VA inpatient and outpatient administrative data with Medicare inpatient and outpatient data, VA laboratory test and pharmacy claims data from Decision Support System (DSS) files and VA vital signs data from Corporate Data Warehouse (CDW). Starting with a standard risk-adjustment (base) model based on administrative data, we evaluated the improvement in predicting discharge outcomes with each data enhancement. We examined the impact of enhancements for the two common profiling metrics of risk-adjusted outcomes: a) observed outcome rate (O/E) based on logistic regression, and b) predicted outcome rate (P/E) based on hierarchical logistic regression. Applying VA Hospital Compare methodology, we also examined the impact on classification of hospitals into above average, average and below average.
For clarity of inference, we examined the impact of the three enhancements - a) Medicare data, b) laboratory test and vital signs data, and c) pharmacy data - separately. Impact of each data enhancement was studied separately for three "high interest" cohorts (AMI, HF and PN) and for the combined cohort of all eight admissions.
Enhancement with Medicare data: Within and outside VA, hospital profiling is predominantly based on administrative data. As many Veterans receive healthcare outside the VA - for example, care covered by Medicare, Medicaid and commercial insurance - their health records are fragmented. In the first study we examined the difference in VA hospital profiles based on VA-only vs. combined VA/Medicare data, for Veterans aged 66 and older. Impact of adding Medicare data on hospital profiling was modest but significant across all four aforementioned admission cohorts (AMI, HF, PN and combined). Both risk-adjusted metrics increased in about half the hospitals and decreased in the other half. For the AMI cohort, risk-adjusted 30-day mortality increased 2.7% (O/E) and 1.4% (P/E) in half the hospitals and the decreased -2.6% (O/E) and -1.2% (P/E) in the other half. This enhancement had only modest impact on the classification of hospitals into above average, average or below average. Out of 130 hospitals, the number re-categorized was 4 (O/E) and 2 (P/E), and each case the magnitude of risk-adjusted mortality change was small.
Enhancement with laboratory test and vital signs data: Measurement of patient acuity at admission can be significantly improved with information on results of laboratory tests and vital signs recorded immediately preceding or following admission. For each of the four admission cohorts examined to date - AMI, HF, PN and acute ischemic stroke - this enhancement produced sizable impact on model performance, hospital risk-adjusted outcomes and hospital classification status. For the stroke cohort, 30-day mortality model performance (c-statistic) improved from 0.785 (base model) to 0.870 (enhanced model). Risk-adjusted mortality increased 19% (O/E) and 10% (P/E) in half the hospitals and the decreased -11% (O/E) and -9% (P/E) in the other half. Out of 103 hospitals, the number reclassified was 16 (O/E) and 5 (P/E), indicating a substantial impact on overall hospital profiles. Similar findings were observed for other admission cohorts.
Enhancement with pharmacy data: Since comorbid diagnostic indicators do not discriminate severity, adding data on use of (outpatient) pharmacy prior to the index hospitalization can potentially improve model performance. For three admission cohorts (AMI, HF and PN), we developed enhanced models by adding VA outpatient pharmacy records for one year preceding index hospitalizations, and grouping all drugs into 251 clinically relevant RxGroups. This enhancement did not have any sizable impact on model discrimination or hospital risk-adjusted outcomes (30-day mortality and 30-day readmission). For the AMI cohort, model performance (c-statistic) improved from 0.779 (base model) to 0.785 (enhanced model). Therefore, enhancement using pharmacy data had little impact on hospital profiles.
Evaluating hospital performance based on patient outcomes, has become a mainstay of ongoing health care quality improvement initiatives. Findings from this study will complement ongoing VA inpatient outcome monitoring programs, including VA HospitalCompare, by providing (a) improved methods for quantifying change in hospital profiles, and (b) hospital profiles for admission cohorts not previously examined.
- Carey K, Stefos T, Shibei Zhao, Borzecki AM, Rosen AK. Excess costs attributable to postoperative complications. Medical care research and review : MCRR. 2011 Aug 1; 68(4):490-503.
- Rosen AK. Quality of Care for Population-Patients from PHN Interventions: State of the Art. Paper presented at: University of Illinois at Chicago Quality of Public Health Nursing Care: Prioritizing the Research Agenda Conference; 2010 Oct 1; Oak Brook, IL.