Health Services Research & Development

Veterans Crisis Line Badge
Go to the ORD website
Go to the QUERI website

2006 HSR&D National Meeting Abstract


1021 — Identification of Patient Safety Events from VA Administrative Data: Is it Valid?

Author List:
Rosen AK (Center for Health Quality, Outcomes and Economic Research (CHQOER))
Rivard P (CHQOER)
Zhao S (CHQOER)
Tsilimingras D (CHQOER)
Loveland S (CHQOER)
Christiansen C (CHQOER)
Henderson W (University of Colorado)
Khuri S (Boston VA Healthcare System)
Elixhauser A (AHRQ)
Romano P (University of California at Davis)

Objectives:
The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) are currently used for quality improvement, public reporting, and pay-for-performance initiatives. Although the PSIs were designed to be used with administrative data to screen for potential patient safety events that compromise inpatient safety, their validity as screening measures has not been tested. The purpose of this study is to examine their validity against clinically-derived postoperative adverse events from the VA National Surgical Quality Improvement Program (NSQIP). We selected 5 of 16 PSIs (Postoperative Physiologic and Metabolic Derangements, Postoperative Respiratory Failure, Postoperative Pulmonary Embolism/Deep Vein Thrombosis (PE/DVT), Postoperative Sepsis, and Postoperative Wound Dehiscence) whose definitions, based on ICD-9-CM codes, were comparable to NSQIP’s clinically-defined outcomes.

Methods:
We used the Patient Treatment File (PTF) to identify all discharge hospitalizations in FY2001 with surgical DRGs. Specific algorithms were constructed to merge PTF surgical hospitalizations with NSQIP chart-abstracted surgical data, resulting in a data file containing 56,419 hospitalizations. Using NSQIP as the “gold standard,” we examined the sensitivities, positive predictive values (PPVs), and likelihood ratios of the five PSIs using: 1) original PSI definitions and 2) modified PSI definitions containing additional ICD-9-CM diagnoses/procedures to improve comparability with NSQIP’s definitions.

Results:
Sensitivities ranged from 19% to 56% for original PSI definitions; PPVs ranged from 22% to 74%. Modifications to the PSIs improved sensitivities of all PSIs; the greatest improvement was in “Postoperative Respiratory Failure” (19% to 67%). The PPVs had mixed results. Positive likelihood ratios ranged from 65 to 524 for original PSI definitions, suggesting that occurrence of a PSI was 65-524 times more likely among those hospitalizations with NSQIP events than those without NSQIP events.

Implications:
The PSIs show promise as valid screening tools for potential patient safety events that occur in the hospital setting. Slight modifications to the PSIs improved their ability to identify in-hospital patient safety events.

Impacts:
Administrative data offer a relatively inexpensive and efficient approach for flagging potential patient safety events compared to medical chart abstraction. Given the impact that public reporting can have on any given hospital, validation of the PSIs is critical for hospital-level comparisons.