3084 — Automated Measurement of Care Delivery to Heart Failure Patients within the Department of Veterans Affairs

Garvin JH, Salt Lake City VA HCS; Kim Y, Salt Lake City VA HCS; Gobbel GT, Tennesee Valley VA HCS; Matheny ME, Tennesee Valley VA HCS; Redd A, Salt Lake City VA HCS; Heidenreich P, Palo Alto VA HCS; Bolton D, Salt Lake City VA HCS; Kalsy M, Salt Lake City VA HCS; Goldstein MK, Palo Alto VA HCS; Meystre S, Salt Lake City VA HCS

Objectives:
We used information extraction techniques to automate calculation of quality metrics for chronic heart failure (HF) within the Department of Veterans Affairs. We undertook formative evaluation to inform potential implementation and to determine clinical uses of the automated system.

Methods:
We used texts from 1390 VA inpatients discharged with HF and trained and tested a modular natural language processing (NLP) application, the Congestive Heart Failure Information Extraction Framework (CHIEF), to extract required data from text documents and combine them to classify the patient's documented care as meeting the quality measure. We conducted semi-structured interviews with interview guides using a snowball sampling technique. Validated interview summaries were used to generate themes using applied thematic analysis.

Results:
Using the NLP testing set of 742 patients, CHIEF classified each patient's care at the time of discharge as correctly with a sensitivity of 98.9% and a PPV of 98.7%. We compared our results with those obtained by EPRP with 45.73% of patients classified as meeting CHI 19 at the time of discharge during manual review based on EPRP results compared to CHIEF, in contrast, which classified a greater proportion of patients, 97.70%, as meeting CHI19 at the time at discharge.

Implications:
CHIEF automated processes, using only a specific types of text documents, accurately identified a greater percentage of patients with evidence-based care at the time of discharge compared to EPRP manual review despite having access to less data. This implies that automated quality measurement can potentially provide accurate, timelier data about the status of guideline-concordant care for a given patient especially if text notes and resulting data could be provided in near-real time at the point of care to treating providers.

Impacts:
Automation of quality measurement has the potential to provide actionable information at the point of care to prompt guideline-concordant care.