Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

Health Services Research & Development

Veterans Crisis Line Badge
Go to the ORD website
Go to the QUERI website

2009 HSR&D National Meeting Abstract

Printable View

National Meeting 2009

3084 — Validating the Patient Safety Indicators (PSIs): Do They Reflect True Safety Events?

Rosen AK (Bedford COE), Borzecki A (Bedford COE), Kaafarani H (Boston COE), Hanchate A (Boston University), Loveland S (Bedford COE), Hartmann C (Bedford COE), Shwartz M (Boston COE), Shin M (Bedford COE), Shimada S (Bedford COE), Rivard P (Suffolk University)

Objectives:
Accurate measurement of patient safety-related events is challenging given their relative infrequency. The AHRQ Patient Safety Indicators (PSIs) use hospital administrative data to screen for potentially preventable complications, such as postoperative pulmonary embolism or deep vein thrombosis (PE/DVT) and iatrogenic pneumothorax (IP). Since there is little evidence that PSIs measure “true” safety events, we examined their criterion validity through chart review of flagged PSI cases. Site visits also will be conducted to examine attributional validity (i.e., the association between PSI rates and processes/structure of care). This multi-faceted approach will elicit important information regarding which PSIs should be used as performance measures and which should be used only as case-finding/screening tools.

Methods:
We applied the AHRQ PSI software to FY03-FY07 PTF files to obtain risk-adjusted PSI rates. We conducted a retrospective chart review of flagged PSI cases using standardized abstraction tools. We examined inter-rater reliability between nurses and calculated rates of true and false positives.

Results:
Two PSIs (PE/DVT and IP) have been validated to date. Of 112 cases abstracted for PE/DVT, 49 were true events, yielding a positive predictive value (PPV) of 43.8% (95% CI, 34.4-53.4%). PPV for IP was higher (50%, 95% CI, 35.5%-64.5%). Inter-rater reliability was > 90% for both PSIs. Various coding-related issues accounted for most of the false positives for both PSIs. However, for IP most of the false positives resulted from IPs following percutaneous lung biopsies that are meant to be excluded per the PSI algorithm.

Implications:
Improving PSI algorithms and expanding the codes used should increase the predictive value of the PSIs. PPV results suggest that with some relatively minor modifications (e.g., better procedure coding for lung biopsy), IP may be ready for use in performance measurement, whereas PE/DVT may be better suited as a screening tool. Site visit information should provide additional insights on how healthcare systems can use the PSIs. We are continuing to validate other PSIs and proceed with site visits.

Impacts:
As the PSIs are increasingly being used for hospital profiling and pay-for-performance, evaluating their accuracy is critical for determining which ones are most appropriate for these purposes.


Questions about the HSR&D website? Email the Web Team.

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.