skip to page content
Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website
Publication Briefs

Using Natural Language Processing to Evaluate Evidence-Based Treatment for Veterans with PTSD


BACKGROUND:
Given the high prevalence of PTSD among Veterans who use VA healthcare, the impairment associated with PTSD, and the amount of care VA provides for PTSD, it is essential that this care be as effective as possible. A major limitation in the ability to track the use of evidence-based therapies on a regional or national level has been the need to manually review the content of treatment notes to determine whether the type of psychotherapy being delivered follows practices shown to be effective for PTSD. This study examined whether automated coding of treatment notes using natural language processing could match the accuracy of manual chart review for classifying evidence-based PTSD treatments. Investigators analyzed the content of clinical notes using the automated retrieval console (ARC). ARC "learns" from a set of gold standard interpretations made by human chart reviewers and combines natural language processing and machine learning technology to replicate their judgments. Using this automated process, they evaluated the use of evidence-based psychotherapy treatment for over 1900 Veterans newly enrolled in six VA outpatient PTSD clinics in New England during FY10, following extensive national efforts to increase use of evidence-based psychotherapies for PTSD.

FINDINGS:

  • ARC identified psychotherapy notes and classified the type of therapy performed in a treatment session as reliably as expert human raters.
  • Evidence-based psychotherapies appear to be infrequently used as an initial treatment in VA outpatient PTSD clinics in New England. Of the Veterans in this study, ony 6% received at least one session of one of the evidence-based psychotherapies outlined in the VA/DOD guidelines for PTSD during the initial six months of treatment.

LIMITATIONS:

  • It is possible that patients in this study were receiving other evidence-based treatments, such as pharmacotherapy. Examination of pharmacy records was beyond the scope of this study.
  • This study was limited to Veterans receiving treatment in specialized PTSD clinics in New England.

IMPLICATION:

  • The ARC method uses a clinican-friendly interface, does not require custom programming, and is freely available to others, so could be used to evaluate treatment for other mental health disorders and in other healthcare settings that use electronic medical records.

AUTHOR/FUNDING INFORMATION:
This study was partly supported through VA/HSR&D's Quality Enhancement Research Initiative (QUERI: RRP 11-432) and through the VA Consortium for Health Informatics Research (CHIR). Drs. Shiner and Watts are part of the White River Junction VAMC and the VA New England Veterans Engineering Resource Center (VERC), White River Junction, VT. Dr. Shiner is also part of the VA National Center for PTSD, White River Junction, VT.


PubMed Logo Shiner B, D’Avolio L, Nguyen T, Zayed M, Young-Xu Y, Desai R, Schnurr P, Fiore L, and Watts B. Measuring Use of Evidence-Based Psychotherapy for Post-Traumatic Stress Disorder. Administration and Policy in Mental Health and Mental Health Services Research April 26, 2012; Epub ahead of print.

Related Briefs

» next 90 PTSD Briefs...


What are HSR Publication Briefs?

HSR requires notification by HSR-funded investigators about all articles accepted for publication. These journal articles are reviewed by HSR and publication briefs or summaries are written for a select number of articles that are then forwarded to VHA Central Office leadership to keep them informed about important findings or information. Articles to be summarized are selected by HSR based on timeliness of the findings, interest of leadership, or potential impact on the organization. Publication briefs are written for only a small number of HSR published articles. Visit the HSR citations database for a complete listing of HSR articles and presentations.


Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.