Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

Health Services Research & Development

Veterans Crisis Line Badge
Go to the ORD website
Go to the QUERI website

IIR 09-369 – HSR&D Study

New | Current | Completed | DRA | DRE | Portfolios/Projects | Centers | Career Development Projects

IIR 09-369
Validating and Classifying VA Readmissions For Quality Assessment and Improvement
Amy K. Rosen PhD
VA Boston Healthcare System Jamaica Plain Campus, Jamaica Plain, MA
Boston, MA
Funding Period: March 2011 - May 2014

BACKGROUND/RATIONALE:
Readmissions are common and costly, and can lead to further clinical decline of the patient. Many readmissions may be potentially preventable, the risk being modifiable by the quality and type of care provided. Identifying this subset of patients would help direct quality improvement (QI) efforts more effectively in reducing readmission rates. Focusing on specific subgroups in the VA who are at relatively high risk of readmission [patients with heart failure (HF), acute myocardial infarction (AMI), and pneumonia (PNA)] may yield a higher reward in terms of improved patient care and successful QI initiatives. These conditions are also publicly reported on the VA and CMS Hospital Compare websites, and are being used to assess the performance of VA hospitals. By furthering our understanding of potentially preventable readmissions, we will be better positioned to target those areas that need QI as well as understand those processes of care that indicate high quality of care and that may be less likely to lead to readmission.

OBJECTIVE(S):
1) Estimate risk-adjusted models to predict 30-day readmissions for patients discharged with HF, AMI, or PNA from an acute-care VA hospital; 2) Investigate rates of potentially preventable readmissions for HF, AMI, and PNA using readmission classification software [3M Potentially Preventable Readmissions (PPRs)] designed to identify potentially preventable readmissions based on administrative data; 3) Develop chart abstraction tools to identify potentially preventable readmissions for patients discharged with HF, AMI, or PNA; 4) Apply chart abstraction tools to VA electronic medical records (EMR) to classify HF, AMI, and PNA all-cause readmissions; and 5) Re-estimate hospital-specific risk-adjusted rates of potentially preventable readmissions in VA using supplemental automated data.

METHODS:
We conducted a 3-year retrospective observational study using FY2006-2010 VA inpatient and outpatient administrative and EMR data, supplemented by CMS Medicare files. Assessment of potential preventability of 30-day readmissions among 100 cases per condition-cohort was compared between EMR-abstracted data and the 3M PPR software. We developed and pilot-tested EMR abstraction tools to assess processes of care (i.e., quality) during the index and post-discharge periods. Two trained nurse-abstractors reviewed EMRs, with inter-rater agreement >90 in all conditions. We compared PPR-flagged and non-flagged cases on total and section-specific mean quality scores using t-tests. We also reclassified the PPR algorithm by incorporating lab values, vital signs, prior utilization, and medication data into the risk adjustment; we then estimated regression models to predict PPR readmissions (yes/no) and examined whether the re-classified algorithms improved model performance or led to changes in hospital ranks and performance using hospitals' original PPR observed-to-expected ratios versus enhanced PPR observed-to-expected ratios (with supplemental data).

FINDINGS/RESULTS:
1) Hospital risk-adjusted 30-day readmission rates ranged from 14.9% to 23.6% for AMI; 16.5% to 31.0% for HF, and 12.8% to 20.7% for PNA.
2) The overall all-condition PPR rate was 10.8%; hospital risk-adjusted rates ranged from 5.9%-18.6%. The range in hospital risk-adjusted PPR rates was 8.0%-60.0% for AMI, 9.0%-31.3% for HF, and 3.9%-29.2% for PNA.
3) Explicit criteria to assess processes of care for each condition were selected and refined by expert clinical panels following the RAND/UCLA Appropriateness Method. These were incorporated into 4 sections of chart abstraction tools: admission work-up, evaluation and treatment during stay, discharge readiness, and post-discharge period (a maximum obtainable quality score was 100 overall and 25 for each section).
4) For all three cohorts, we did not find any significant associations between PPR outcomes and the overall or section quality scores, whether we used equal, section, or Delphi-based weights for each chart-based data element. For example, among 100 HF cases, the overall mean quality score was 61.5+10.3. Section scores were highest for discharge readiness (18.8+2.4) and lowest for post-discharge care (7.3+8.1). Mean overall and section quality scores did not differ by PPR status; respective PPR-Yes vs. PPR-No overall scores were 61.2 and 63.4 (p=0.47).
5) We did not find improved adjusted R-square estimates in the reclassified condition-specific model when we examined the association between the quality score and the original versus reclassified PPR algorithms. However, specifically among PNA readmissions, we found that adding prior utilization data and vital signs to predict PPRs increased the c-statistic from 0.577 to 0.626. Prior utilization also significantly increased the odds of PPRs; in particular, the number of prior admissions had an odds ratio (OR) of 1.20, confidence interval (CI)=1.18-1.22. Finally, 9.2% of hospitals changed performance quartiles when supplemental clinical data were incorporated into the models.

IMPACT:
Given increasing reliance on readmission as a measure of hospital performance, it is critical for hospitals to be able to identify those readmissions that are more likely to be preventable and therefore better targets for quality improvement. Although the PPRs represent an attractive alternative to the CMS all-cause readmission measure, we did not find a significant difference in the quality of care (as measured by inpatient and post-discharge processes of care) between cases flagged as PPRs and non-PPRs. This may be due to: problems in using administrative data-based readmission measures; limitations in the data present in the EMR (e.g., patient-provider communication) which limit our ability to determine preventability; and the multitude of factors associated with readmission we did not examine (e.g., socioeconomic factors, psychosocial factors, access issues). Future studies at the hospital level are needed to further explore the utility of the PPRs in identifying high-risk groups of patients for QI/intervention. In addition, future studies should also explore other ways of identifying potentially preventable readmissions (i.e., through provider and/or patient perceptions or direct observation).

PUBLICATIONS:

Journal Articles

  1. Borzecki AM, Chen Q, Mull HJ, Shwartz M, Bhatt DL, Hanchate A, Rosen AK. Do Acute Myocardial Infarction and Heart Failure Readmissions Flagged as Potentially Preventable by the 3M Potentially Preventable Readmissions Software Have More Process-of-Care Problems? Circulation. Cardiovascular quality and outcomes. 2016 Sep 6; 9(5):532-41.
  2. Chen Q, Mull HJ, Rosen AK, Borzecki AM, Pilver C, Itani KM. Measuring readmissions after surgery: do different methods tell the same story? American journal of surgery. 2016 Jul 1; 212(1):24-33.
  3. Rosen AK, Chen Q, Shwartz M, Pilver C, Mull HJ, Itani KF, Borzecki A. Does Use of a Hospital-wide Readmission Measure Versus Condition-specific Readmission Measures Make a Difference for Hospital Profiling and Payment Penalties? Medical care. 2016 Feb 1; 54(2):155-61.
  4. Borzecki AM, Chen Q, Restuccia J, Mull HJ, Shwartz M, Gupta K, Hanchate A, Strymish J, Rosen A. Do pneumonia readmissions flagged as potentially preventable by the 3M PPR software have more process of care problems? A cross-sectional observational study. BMJ quality & safety. 2015 Dec 1; 24(12):753-63.
  5. Mull HJ, Brennan CW, Folkes T, Hermos J, Chan J, Rosen AK, Simon SR. Identifying Previously Undetected Harm: Piloting the Institute for Healthcare Improvement's Global Trigger Tool in the Veterans Health Administration. Quality management in health care. 2015 Jul 1; 24(3):140-6.
  6. O'Brien WJ, Chen Q, Mull HJ, Shwartz M, Borzecki AM, Hanchate A, Rosen AK. What is the value of adding Medicare data in estimating VA hospital readmission rates? Health services research. 2015 Feb 1; 50(1):40-57.
  7. Chen Q, Tsai TC, Mull HJ, Rosen AK, Itani KM. Using a composite readmission measure to assess surgical quality in the Veterans Health Administration: how well does it correlate with established surgical measures? JAMA surgery. 2014 Nov 1; 149(11):1206-7.
  8. Mull HJ, Chen Q, Shwartz M, Itani KM, Rosen AK. Measuring surgical quality: which measure should we trust? JAMA surgery. 2014 Nov 1; 149(11):1210-2.
  9. Rosen AK, Chen Q, Shin MH, O'Brien W, Shwartz M, Mull HJ, Cevasco M, Borzecki AM. Medical and surgical readmissions in the Veterans Health Administration: what proportion are related to the index hospitalization? Medical care. 2014 Mar 1; 52(3):243-9.
  10. Mull HJ, Chen Q, O'Brien WJ, Shwartz M, Borzecki AM, Hanchate A, Rosen AK. Comparing 2 methods of assessing 30-day readmissions: what is the impact on hospital profiling in the veterans health administration? Medical care. 2013 Jul 1; 51(7):589-96.
  11. Rosen AK, Loveland S, Shin M, Shwartz M, Hanchate A, Chen Q, Kaafarani HM, Borzecki A. Examining the impact of the AHRQ Patient Safety Indicators (PSIs) on the Veterans Health Administration: the case of readmissions. Medical care. 2013 Jan 1; 51(1):37-44.
Conference Presentations

  1. Borzecki AM, Mull HJ, Shwartz M, Labonte AJ, Rosen AK. Are Readmissions Flagged as Potentially Preventable More Likely to Have Process of Care Problems than Non-flagged Readmissions? Applying the 3M™ PPR Software to Acute Myocardial Infarction in the Veterans Health Administration. Presented at: AcademyHealth Annual Research Meeting; 2014 Jun 10; San Diego, CA.
  2. Chen Qi, Tsai TC, Mull HJ, Itani K, Rosen AK. Using A Composite Readmission Measure to Assess Surgical Quality in the Veterans Health Administration: How Well Does It Correlate with Established Surgical Measures? Presented at: AcademyHealth Annual Research Meeting; 2014 Jun 10; San Diego, CA.
  3. Rosen AK, Chen Qi, Restuccia JD, Shwartz M, O'Brien W, Borzecki AM. Supplementing Administrative Data with Prior Utilization and Clinical Information: Does it Contribute to Improving Prediction of Potentially Preventable Readmissions and Lead to Changes in VA Hospital Profiles? Presented at: AcademyHealth Annual Research Meeting; 2014 Jun 10; San Diego, CA.
  4. Chen Qi, Tsai TC, Mull HJ, Rosen AK, Itani K. Using A Composite Readmission Measure to Assess Surgical Quality in the Veterans Health Administration: How Well Does It Correlate with Established Surgical Measures? Presented at: VA Association of Surgeons Annual Meeting; 2014 Apr 8; New Haven, CT.
  5. Chen Q, Mull HJ, O'Brien W, Itani K, Rosen AK. Examining Potentially Preventable Readmissions after Surgical Procedures in the Veteran Health Administration. Poster session presented at: AcademyHealth Annual Research Meeting; 2013 Jun 26; Baltimore, MD.
  6. Mull HJ, Chen Qi, O'Brien W, Shwartz M, Borzecki AM, Hanchate AD, Rosen AK. Comparing Two Methods of Assessing 30-Day Readmissions in the Veterans Health Administration: What is the Effect on Hospital Reporting and Pay-for-Performance? Presented at: AcademyHealth Annual Research Meeting; 2013 Jun 25; Baltimore, MD.
  7. O'Brien W, Chen Qi, Mull HJ, Borzecki AM, Shwartz M, Hanchate AD, Rosen AK. Tracking Readmissions Across Healthcare Systems: Implications for Readmission Rates and Payment Penalties. Poster session presented at: AcademyHealth Annual Research Meeting; 2013 Jun 25; Baltimore, MD.
  8. Rosen AK, Chen Qi, O'Brien W, Borzecki AM, Frakt AB, Shwartz M, Bauer MS. Assessing the Impact of Serious Mental Illness (SMI) and Substance Use Disorders (SUD) on Risk of Readmission for Patients with Acute Myocardial Infarction, Heart Failure, and Pneumonia in the Veterans Health Administration (VA). Presented at: AcademyHealth Annual Research Meeting; 2013 Jun 25; Baltimore, MD.
  9. Chen Q, Borzecki A, O'Brien W, Mull HJ, Restuccia J, Rosen AK. Validating the 3M™ Potentially Preventable Readmissions Software in a Cohort of Veterans with Pneumonia. Poster session presented at: AcademyHealth Annual Research Meeting; 2013 Jun 24; Baltimore, MD.
  10. Mull HJ, Chen Qi, O'Brien W, Shin M, Rosen AK. Hospital-level Variation in Potentially Preventable Hospital Readmission Rates within the Veterans Health Administration. Poster session presented at: AcademyHealth Annual Research Meeting; 2013 Jun 24; Baltimore, MD.
  11. Chen Qi, Mull HJ, O'Brien W, Borzecki AM, Itani K, Rosen AK. How Many 30-Day Readmissions after Surgical Procedures Are Preventable? Application of the 3M Potentially Preventable Readmission Classification System in the Veterans Health Administration. Presented at: Boston University School of Medicine Department of Surgery Annual Grasberger Research Symposium; 2013 Feb 25; Boston, MA.
  12. Hynes D, Brown MM, Maciejewski ML, Wagner T, Rosen A. Current Research Applications and Future Directions in VA-CMS Data. Paper presented at: VA HSR&D / QUERI National Meeting; 2012 Jul 17; National Harbor, MD.
  13. O'Brien W, Chen Qi, Rosen AK. Adding Medicare Data to Assess Veterans’ Readmission Rates: More Trouble than it’s Worth? Presented at: VA HSR&D / QUERI National Meeting; 2012 Jul 15; National Harbor, MD.


DRA: Cardiovascular Disease, Lung Disorders, Health Systems
DRE: Prognosis, Treatment - Observational
Keywords: Predictive Modeling, Quality Indicators, Quality of Care, Risk Adjustment, System Performance Measures
MeSH Terms: none

Questions about the HSR&D website? Email the Web Team.

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.