3060 — Using Information Extraction to Measure Adherence to TBI Treatment Guidelines

Finch DK, HSR&D Center of Innovation in Disability and Rehabilitation Research; Massengale JP, HSR&D Center of Innovation in Disability and Rehabilitation Research; McCart JA, HSR&D Center of Innovation in Disability and Rehabilitation Research; Luther SL, HSR&D Center of Innovation in Disability and Rehabilitation Research;

Objectives:
The VA/DoD Clinical Practice Guideline for Management of Concussion/Mild Traumatic Brain Injury (VA/DoDCPG) was created to assist clinicians in providing care for patients with mTBI. This study explores the potential of natural language processing (NLP) to extract information about adherence to the VA/DoDCPG guidelines.

Methods:
Clinicians reviewed 370 Plan of Care documents stratified across all VA Ploytrauma System of Care facilities (n = 23). Text representing symptoms and treatments were annotated, a vocabulary was constructed and a custom NLP system was constructed to extract this information. Symptoms included headaches, sleep disturbance, difficulties with concentration, decision making and memory, slowed thinking, depression, frustration, irritability, anxiety, and substance abuse. The system included functions to identify document sections, fuzzy string matching for minor spelling errors and a modified version of Negex. We achieved an overall F-measure of 0.86 compared to the annotated reference standard. The NLP system was applied system to all Plan of Care documents (n = 13,061) from the study's 6,572 patients. A clinical expert constructed a treatment-by-symptom matrix to designate which treatments were appropriate for each symptom according to the guideline. Treatments included all appropriate drug classes and clinical disciplines as well as specific treatments, such as "exposure therapy", "anger management" or "CPAP machine." We calculated the proportion (in percentages) of patients who had documentation of guideline care for each of the 13 symptoms.

Results:
We found that the most prevalent symptom experienced by far was headaches (67.07%), followed by sleep disturbance (39.24%) and memory problems (34.53%). Headaches were also the symptom with the lowest percentage of patients with documented guideline care (89.95%). This may be attributed to the complex relationship between headaches and sleep disturbance. Clinicians will often treat sleep disturbance in an effort to alleviate headache symptoms.

Implications:
Overall, the proportions of documented treatments for each symptom indicate a high level of adherence to the treatment protocol

Impacts:
The vocabulary from this study can be used in future NLP systems to extract symptoms and treatments of TBI. The results of this study will be used as pilot data for future studies evaluation the quality of assessments for headaches and sleep disturbance.