skip to page content
Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website
Publication Briefs

Patient Safety Indicators Do Not Always Identify True Safety Events in VA Hospitals


BACKGROUND:
As increasing numbers of healthcare systems are adopting the Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) for public reporting/pay-for-performance, it is essential that the PSIs accurately reflect hospital safety performance. Although the PSIs underwent a rigorous development process, concerns about how well the indicators identify true events and accurately reflect hospital performance have heightened with their increasing use in public reporting. This retrospective study examined the positive predictive value (PPV – proportion of flagged cases confirmed by chart review to have PSI event) of 12 selected PSIs using data from VA’s electronic medical record as the gold standard. Investigators also sought to determine reasons why flagged PSI cases did not represent true events, and to recommend modifications to the PSI algorithms to improve PPVs. The 12 PSIs included: decubitus ulcer, foreign body left in during procedure, iatrogenic pneumothorax, central venous catheter-related bloodstream infections, post-operative hip fracture, post-operative hemorrhage or hematoma, post-operative physiologic and metabolic derangement, post-operative respiratory failure, post-operative pulmonary embolism or deep vein thrombosis, post-operative sepsis, post-operative wound dehiscence, and accidental puncture or laceration. Using VA data from FY03 through FY07, investigators identified PSI-flagged cases from 28 representative VA hospitals, focusing on hospital discharge information, e.g., diagnosis and procedure codes, and patient demographics.

FINDINGS:

  • Despite evidence on the accuracy and completeness of VA data, all PSIs misidentified true events to some extent, with considerable PPV variation across PSIs. PPVs ranged from 28% for post-operative hip fracture to 87% for post-operative wound dehiscence. This variation was due to coding inaccuracies or limitations (e.g., lack of precise or meaningful codes, poor documentation).
  • PSI rates were generally low. Ulcer and respiratory failure were the most commonly flagged PSIs, suggesting clinical areas for targeting and opportunities for hospital improvement.
  • VA PSI rates will be reported on both the VA and CMS Hospital Compare websites in the near future. However, results suggest that additional coding improvements are needed before the PSIs evaluated in this study are used for hospital reporting or pay-for-performance.

LIMITATIONS:

  • Investigators were unable to report other aspects of criterion validity, e.g., specificity, sensitivity, or negative predictive value of the PSIs.
  • The sample size was small, preventing investigators from examining whether there was variation in coding accuracy across hospitals.

AUTHOR/FUNDING INFORMATION: This study was funded by HSR&D (SDR 07-002). Dr. Rosen is part of HSR&D’s Center for Organization, Leadership and Management Research, Boston, MA.


PubMed Logo Rosen A, Itani K, Cevasco M, et al. Validating the Patient Safety Indicators in the Veterans Health Administration: Do They Accurately Identify True Safety Events? Medical Care January 2012;50(1):74-85.

Related Briefs

» next 27 Patient Safety Briefs...


What are HSR Publication Briefs?

HSR requires notification by HSR-funded investigators about all articles accepted for publication. These journal articles are reviewed by HSR and publication briefs or summaries are written for a select number of articles that are then forwarded to VHA Central Office leadership to keep them informed about important findings or information. Articles to be summarized are selected by HSR based on timeliness of the findings, interest of leadership, or potential impact on the organization. Publication briefs are written for only a small number of HSR published articles. Visit the HSR citations database for a complete listing of HSR articles and presentations.


Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.