Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

Health Services Research & Development

Veterans Crisis Line Badge
Go to the ORD website
Go to the QUERI website

2011 HSR&D National Meeting Abstract

Printable View

2011 National Meeting

1057 — Comparative Effectiveness Study of the Patient Safety Indicators and Natural Language Processing on Identifying Post-Operative Complications

Murff HJ (Tennessee Valley Healthcare System), FitzHenry F (Tennessee Valley Healthcare System), Matheny M (Tennessee Valley Healthcare System), Gentry N (Tennessee Valley Healthcare System), Kotter KL (Tennessee Valley Healthcare System), Dittus R (Tennessee Valley Healthcare System), Rosen A (Bedford Veterans Affairs Medical Center), Elkin PL (Mount Sinai School of Medicine), Brown SH (Tennessee Valley Healthcare System), Speroff T (Tennessee Valley Healthcare System)

Objectives:
The Patient Safety Indicators (PSI) are being used to identify patient safety occurrences; however, using natural language processing (NLP) of the electronic medical record might represent a more accurate means of identifying adverse events. The purpose of this study was to develop and evaluate a rule-based free-text search engine to identify post-operative surgical complications and compare this strategy to the PSI.

Methods:
We selected a sample of 2794 patients who underwent major surgical procedures in one of six Veterans Administration Medical Centers from 1999 to 2006. Surgical complications including acute renal failure, deep vein thrombosis, pulmonary embolism, sepsis, pneumonia, and myocardial infarction were determined by manual chart review performed as part of the VA Surgical Quality Improvement Program (VASQIP). We determined the sensitivity, specificity, and 95% confidence intervals of the text-based rules engine and the PSI rules.

Results:
For acute renal failure, the NLP approach had a sensitivity and specificity of 0.82 (0.67, 0.91) and 0.90 (0.89, 0.91) compared to the PSI sensitivity and specificity of 0.38 (0.25, 0.54) and 1.00 (1.00, 1.00). For deep vein thrombosis/pulmonary embolism, the NLP approach had a sensitivity and specificity of 0.70 (0.58, 0.79) and 0.87 (0.86, 0.89) compared to the PSI sensitivity and specificity of 0.46 (0.32, 0.60) and 0.99 (0.98, 0.99). For sepsis, the NLP approach had a sensitivity and specificity of 0.69 (0.56, 0.79) and 0.83 (0.80, 0.85) compared to the PSI sensitivity and specificity of 0.34 (0.24, 0.47) and 0.99 (0.98, 0.99). For pneumonia, the NLP approach had a sensitivity and specificity of 0.81 (0.75, 0.86) and 0.86 (0.84, 0.88) compared to the PSI sensitivity and specificity of 0.05 (0.03, 0.09) and 0.99 (0.99, 1.00). For myocardial infarctions, the NLP approach had a sensitivity and specificity of 0.94 (0.81, 0.98) and 0.83 (0.81, 0.84) compared to the PSI sensitivity and specificity of 0.89 (0.74, 0.96) and 0.99 (0.98, 0.99).

Implications:
A NLP approach to identify surgical complications is more sensitive but less specific compared to a strategy using the PSI.

Impacts:
NLP can potentially assist in prioritization, efficiency, and broadening scope of case finding for surgical quality review.


Questions about the HSR&D website? Email the Web Team.

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.