Comparison of six natural language processing approaches to assessing firearm access in Veterans Health Administration electronic health records.

Trujeque J, Dudley RA, Mesfin N, Ingraham NE, Ortiz I, Bangerter A, Chakraborty A, Schutte D, Yeung J, Liu Y, Woodward-Abel A, Bromley E, Zhang R, Brenner LA, Simonetti JA. Comparison of six natural language processing approaches to assessing firearm access in Veterans Health Administration electronic health records. Journal of the American Medical Informatics Association : JAMIA. 2025 Jan 1; 32(1):113-118.

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.    Search Dimensions for VA for this citation
* Don't have VA-internal network access or a VA email address? Try searching the free-to-the-public version of Dimensions

Search for Abstract from PubMed

---

Abstract:

OBJECTIVE: Access to firearms is associated with increased suicide risk. Our aim was to develop a natural language processing approach to characterizing firearm access in clinical records. MATERIALS AND METHODS: We used clinical notes from 36 685 Veterans Health Administration (VHA) patients between April 10, 2023 and April 10, 2024. We expanded preexisting firearm term sets using subject matter experts and generated 250-character snippets around each firearm term appearing in notes. Annotators labeled 3000 snippets into three classes. Using these annotated snippets, we compared four nonneural machine learning models (random forest, bagging, gradient boosting, logistic regression with ridge penalization) and two versions of Bidirectional Encoder Representations from Transformers, or BERT (specifically, BioBERT and Bio-ClinicalBERT) for classifying firearm access as "definite access", "definitely no access", or "other". RESULTS: Firearm terms were identified in 36 685 patient records (41.3%), 33.7% of snippets were categorized as definite access, 9.0% as definitely no access, and 57.2% as "other". Among models classifying firearm access, five of six had acceptable performance, with BioBERT and Bio-ClinicalBERT performing best, with F1s of 0.876 (95% confidence interval, 0.874-0.879) and 0.896 (95% confidence interval, 0.894-0.899), respectively. DISCUSSION AND CONCLUSION: Firearm-related terminology is common in the clinical records of VHA patients. The ability to use text to identify and characterize patients'' firearm access could enhance suicide prevention efforts, and five of our six models could be used to identify patients for clinical interventions.