2017 HSR&D/QUERI National Conference

4052 — Usability Evaluation of VA Consultation Order Templates: Identifying Problems to Prioritize Redesign Efforts

Lead/Presenter: April Savoy, COIN - Indianapolis
All Authors: Savoy AW (Center for Health Information and Communication (CHIC), Richard L. Roudebush VAMC, Indianapolis, IN) Patel H (CHIC) Flanagan ME (Indiana University School of Medicine) Weiner M (CHIC) Russ AL (CHIC)

Objectives:
In the VA consultation process, consultation order templates facilitate initial steps of referral communication. Although the design of these templates influences both clinical efficiency and patient safety, a formal evaluation of their usability has not been reported. Our objective was to evaluate consultation order templates, identify usability problems, and estimate their potential severity in clinical care.

Methods:
Three evaluators independently performed a heuristic evaluation of 26 consultation order templates collected from three VA Medical Centers in different Veterans Integrated Service Networks (VISNs). Evaluators compared template designs against 19 established design heuristics to identify usability principle violations. Each usability violation was mapped to usability goals. Additionally, transcripts from interviews, conducted separately with 16 referring clinicians about the consultation process, were analyzed. Clinicians' input about 10 templates was converted into dichotomized statements of referral difficulty (i.e., easy or difficult). Results of the heuristic evaluation were compared to clinicians' interviews, to assess agreement about usability findings.

Results:
Evaluators found 201 violations (mean = 7.7 per template). Fourteen heuristics were violated at least once. The most frequently violated heuristics were Aesthetic and Minimalist Design, Error Prevention , and Consistency and Standards. The most severely violated heuristics were Recognition Rather than Recall and Meet Referrers' Information Needs. Collectively, violations potentially inhibited five usability goals: efficiency, effectiveness, safety, learnability, and utility. Of these usability goals, violations were linked most often to efficiency (32%) and safety (24%). Finally, evaluators' ratings and referring clinicians' feedback agreed on 8 of 10 templates .

Implications:
Multiple usability violations were found in each consultation order template studied. The most frequent violations were associated with templates' display and interaction; the most severe were associated with omitted information desired by referring clinicians.

Impacts:
Usability problems with consultation order templates can affect referral communication because they impede the transfer of pertinent information such as reason for referral, patient's presentation, and medical history. This study identified usability design problems in current consultation order templates. Results can be used to prioritize redesign of the consultation process. Improvements to referrals may decrease the volume of cancelled consultation requests, decrease waiting times for appointments, and increase Veterans' access to consultants.