4087 — NoteAid: A System That Translates Medical Terms to Lay Definitions to Support Veterans' Comprehension of Electronic Health Record Notes
Lead/Presenter: Hong Yu, COIN - Bedford/Boston
All Authors: Chen J (Department of Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, MA)
Druhl E (Bedford VA Medical Center, Bedford, MA)
Ramesha BP (Optum, Boston, MA)
Houston T (Bedford VA Medical Center, Bedford, MA; Department of Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, MA)
Brandt C (VA Connecticut Health Care System, West Haven, CT; Center for Medical Informatics, Yale University, New Haven, CT)
Donna ZM (VA Palo Alto Health Care System, Menlo Park, CA; Division of General Medical Disciplines, Stanford University School of Medicine, Stanford, CA)
Yu H (Bedford VA Medical Center, Bedford, MA; Department of Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, MA)
VA has made clinical notes in the electronic health record (EHR) available to veterans through the My HealtheVet portal since 2013. However, studies have shown that medical jargon in EHR notes can confuse patients, which may hinder patient-provider communication and patient engagement and self-care. We therefore developed NoteAid, a natural language processing system that translates medical terms to lay definitions targeting at or below the average adult literacy level, and report a formative evaluation conducted to improve the system.
NoteAid builds on two core units: CoDeMed, a lexicon of lay definitions, and MedLink, a computational unit that links medical terms to lay definitions. For quality assurance, lay definitions were curated and reviewed by domain experts including MDs. MedLink uses the lexical tool MetaMap to identify medical terms. We developed innovative computational methods, including adapted distant supervision and unsupervised ensemble ranking, to enhance NoteAid's lexicon and functionality. For iterative system development, we conducted a novel formative evaluation and asked physicians to assess the system's user interface as software users, and output quality as domain experts. The protocol includes a 1-hour think-aloud cognitive walkthrough session, where physicians used the system to process 10 clinical notes, and a 7-item post-session questionnaire. The sessions and surveys were audio-recorded and analyzed qualitatively using thematic analysis.
Nine physicians with diversified backgrounds evaluated NoteAid. They were in general satisfied with user interface and system output. Positive themes included: (1) easy to use; (2) good visual display; (3) adequate coverage; and (4) adequate lay definitions. Major issues identified included: (1) occasional system crashes when users hit the "simplify" button multiple times (we have fixed this problem); (2) need to cover more multi-word medical terms; (3) some definitions do not fit the specific context.
Physician evaluation yielded positive results and useful feedback for content validation and refinement of this innovative tool.
We have improved NoteAid based on physicians' feedback. Next steps include a study engaging veterans to test the system. NoteAid has the potential to improve patient EHR comprehension, which, when used concurrently with MyHealtheVet, can improve patient experience, engagement, and health knowledge.