BACKGROUND/RATIONALE:
Traumatic Brain Injury (TBI) is referred to as the "signature injury" of modern warfare due to the types of explosives used and the improved survivability of those injured in combat. Most symptoms of mild TBI (mTBI) (e.g., headaches, cognitive impairment, and sleep disturbance) will resolve, although some continue to experience symptoms for long periods of time. The VA/DoD Clinical Practice Guideline for Management of Concussion/Mild Traumatic Brain Injury (VA/DoDCPG) was created to assist clinicians in providing care for patients with mTBI. While the VA electronic health record contains information about adherence to this guideline, much of it is stored in text-based documents and is not easily extracted or summarized.

OBJECTIVE(S):
This study explores the potential of natural language processing (NLP) to extract information about adherence to the VA/DoDCPG guidelines. The aims of this study are listed below.

Aim 1: Develop an annotated reference set of Individualized Rehabilitation and Community Reintegration Care Plan (Plan of Care) documents, containing information about persistent symptoms and treatments for mTBI to act as a reference standard for NLP development.

Aim 2: Create a valid and reliable NLP system to extract the information about persistent symptoms and treatments from the Plan of Care.

Aim 3: Compare the extracted persistent symptoms and treatments with recommended care by clinical practice guidelines and describe the level of adherence to the guidelines.

METHODS:
This study was a retrospective cohort study of Plan of Care documents of Veterans and service members receiving care for persistent symptoms of mTBI. We reviewed the two most common physical symptoms (headaches, and sleep disturbances) and behavioral symptoms related to cognitive difficulties and emotional difficulties. Symptoms included difficulties with concentration , decision making and memory, slowed thinking, depression, frustration, irritability, anxiety, substance abuse, sleep disturbance, headaches, and other cognitive and emotional symptoms not otherwise specified.

Clinicians reviewed 370 Plan of Care documents stratified across all of the Polytrauma System of care facilities and annotated specific text representing the targeted symptoms and treatments and a vocabulary was constructed from these annotations. Because of the way these items are documented in the notes, it was not possible to establish direct links between symptoms and prescribed treatments. Clinicians tend to list all symptoms in the "Current Problems" section and the plan for treating those symptoms in other sections of the document without specifically identifying the symptoms the treatments, referrals and consults were for.

An NLP system was created to extract the targeted information. The system was tested and refined until accuracy was maximized. We achieved an F-measure of .86 overall when the results were compared to the reference standard (based on annotation) (Aim 2).
We applied the NLP system to all Plan of Care documents in the study cohort which included 6572 patients and over 13,000 documents. Since there was often more than one plan of care document for each patient, we combined the symptoms and treatments from all of a patients documents to perform the guideline coverage calculations.

An expert in polytrauma care constructed a treatment by symptom matrix to designate which treatments were appropriate according to the guideline to evaluate adherence to treatment protocols. Symptoms were categorized into 13 individual symptoms and the treatments documented included 91 specific treatment types. Treatments included all appropriate drug classes and clinical disciplines as well as specific treatments, such as "exposure therapy" or "battlemind." We used the treatment by symptom matrix to compare the information extracted with VA/DoDCPG recommendations (Aim 3). We calculated the proportion (in percentages) of patients who had documentation of guideline care for each of the 13 symptoms found in their record.

FINDINGS/RESULTS:
For the employment topic, sample of 200 notes for 60 subjects were annotated and it was found that after removing the 20 subjects who were still on active duty, 75% (30 subjects out of 40) were unemployed.

Below is the summary statistics for the proportion of patients who had the specific symptoms documented in their record, and of those who had the symptom, the proportion who had guideline appropriate care also documented in there record.

Symptom SympPresent Treated
____________________________________
Anxiety 18.40% 92.97%
Irritability 7.99% 95.24%
Substance Abuse 21.68% 95.23%
Frustration 2.05% 98.52%
Depressed 20.88% 93.51%
Concentration 18.58% 97.13%
Decision Making 1.35% 97.75%
Memory 34.53% 98.02%
Slowed Thinking 4.26% 99.64%
Headache 67.07% 89.95%
Sleep Disturbance 39.24% 90.31%
Emotional-NOS 13.10% 96.17%
Cognitive-NOS 27.80% 98.91%

We found that the most prevalent symptom by far was headaches followed by sleep disturbance and memory problems. Headaches was also the symptom with the lowest percentage of patients with documented guideline care. This may be attributed to the complex relationship between headaches and sleep disturbance. Clinician will often treat sleep disturbance in an effort to alleviate headache symptoms. Overall, the proportions of documented treatments for each symptom indicate a high level of adherence to the the treatment protocol.

IMPACT:
The QUERI strategic plan states that "Within this goal area, we plan to work with our partners to convene a group of experts to identify key practices within the Concussion/Mild TBI CPG as well as identify methods for measuring these practices and associated outcomes." This project represents an opportunity to facilitate this effort by automatically extracting information about persistent symptoms and treatments of mTBI and comparing treatments provided with those recommended. This could identify areas for which interventions to improve adherence to the guideline or provide outcomes data from which the success of interventions could be measured.

The results of this study will be used as pilot data for two studies already under development. One to evaluate the quality of headache assessments and another to examine the internal referral practices for vocational rehabilitation in an effort to understand why so few veterans are consuming services.

---

External Links for this Project

NIH Reporter

Grant Number: I21HX001055-01
Link: https://reporter.nih.gov/project-details/8467093

---

PUBLICATIONS:

Journal Articles

1. Dillahunt-Aspillaga C, Finch D, Massengale J, Kretzmer T, Luther SL, McCart JA. Using information from the electronic health record to improve measurement of unemployment in service members and veterans with mTBI and post-deployment stress. PLoS ONE. 2014 Dec 26; 9(12):e115873. [view]

DRA: Brain and Spinal Cord Injuries and Disorders
DRE: Epidemiology
Keywords: none
MeSH Terms: none